Experimental design in two-sided platforms: An analysis of bias
We develop an analytical framework to study experimental design in two-sided
marketplaces. Many of these experiments exhibit interference, where an intervention applied …
marketplaces. Many of these experiments exhibit interference, where an intervention applied …
Interference, bias, and variance in two-sided marketplace experimentation: Guidance for platforms
Two-sided marketplace platforms often run experiments (or A/B tests) to test the effect of an
intervention before launching it platform-wide. A typical approach is to randomize users into …
intervention before launching it platform-wide. A typical approach is to randomize users into …
[DOC][DOC] Online experimentation: Benefits, operational and methodological challenges, and scaling guide
In the past decade, online controlled experimentation, or A/B testing, at scale has proved to
be a significant driver of business innovation. The practice was first pioneered by the …
be a significant driver of business innovation. The practice was first pioneered by the …
Modeling interference using experiment roll-out
Experiments on online marketplaces and social networks suffer from interference, where the
outcome of a unit is impacted by the treatment status of other units. We propose a framework …
outcome of a unit is impacted by the treatment status of other units. We propose a framework …
Design and analysis of bipartite experiments under a linear exposure-response model
A bipartite experiment consists of one set of units being assigned treatments and another set
of units for which we measure outcomes. The two sets of units are connected by a bipartite …
of units for which we measure outcomes. The two sets of units are connected by a bipartite …
Statistical inference and A/B testing for first-price pacing equilibria
We initiate the study of statistical inference and A/B testing for first-price pacing equilibria
(FPPE). The FPPE model captures the dynamics resulting from large-scale first-price auction …
(FPPE). The FPPE model captures the dynamics resulting from large-scale first-price auction …
Adaptive experimental design with temporal interference: A maximum likelihood approach
Suppose an online platform wants to compare a treatment and control policy (eg, two
different matching algorithms in a ridesharing system, or two different inventory management …
different matching algorithms in a ridesharing system, or two different inventory management …
A/b testing for recommender systems in a two-sided marketplace
Two-sided marketplaces are standard business models of many online platforms (eg,
Amazon, Facebook, LinkedIn), wherein the platforms have consumers, buyers or content …
Amazon, Facebook, LinkedIn), wherein the platforms have consumers, buyers or content …
Correlated cluster-based randomized experiments: Robust variance minimization
Experimentation is prevalent in online marketplaces and social networks to assess the
effectiveness of new market intervention. To mitigate the interference among users in an …
effectiveness of new market intervention. To mitigate the interference among users in an …