Ido Bright, Arthur Delarue, Ilan Lobel
arXiv 4 May 2022 · Mathematics — Optimization · publishedManagement Science (2024) · 9 citations (OpenAlex)
arXiv:2205.02274 · PDF · DOI · OpenAlex · Extracted main text
Marketplace companies rely heavily on experimentation when making changes to the design or operation of their platforms. The workhorse of experimentation is the randomized controlled trial (RCT), or A/B test, in which users are randomly assigned to treatment or control groups. However, marketplace interference causes the Stable Unit Treatment Value Assumption (SUTVA) to be violated, leading to bias in the standard RCT metric. In this work, we propose techniques for platforms to run standard RCTs and still obtain meaningful estimates despite the presence of marketplace interference. We specifically consider a generalized matching setting, in which the platform explicitly matches supply with demand via a linear programming algorithm. Our first proposal is for the platform to estimate the value of global treatment and global control via optimization. We prove that this approach is unbiased in the fluid limit. Our second proposal is to compare the average shadow price of the treatment and control groups rather than the total value accrued by each group. We prove that this technique corresponds to the correct first-order approximation (in a Taylor series sense) of the value function of interest even in a finite-size system. We then use this result to prove that, under reasonable assumptions, our estimator is less biased than the RCT estimator. At the heart of our result is the idea that it is relatively easy to model interference in matching-driven marketplaces since, in such markets, the platform mediates the spillover.
appendix boundary found by appendix_command · 59% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Hannah Li, Geng Zhao, Ramesh Johari, and Gabriel Y Weintraub (2022) Interference, bias, and variance in two-sided marketplace experimentation: Guidance for platforms | 0.644 | 2 | 2 | 100% |
| 2 | Patrick Bajari, Brian Burdick, Guido W Imbens, Lorenzo Masoero, Jame… (2021) Multiple Randomization Designs | 0.585 | 3 | 1 | 100% |
| 3 | Ramesh Johari, Hannah Li, Inessa Liskovich, and Gabriel Y Weintraub (2022) Experimental Design in Two-Sided Platforms: An Analysis of Bias | 0.585 | 3 | 1 | 100% |
| 4 | Nicholas Chamandy (2016) Experimentation in a Ridesharing Marketplace | 0.511 | 2 | 1 | 100% |
| 5 | New York City Taxi and Limousine Commission (2022) Trip record data | 0.405 | 1 | 1 | 100% |
| 6 | Ravindra K Ahuja, Thomas L Magnanti, and James B Orlin (1993) Network flows: theory, algorithms, and applications, chapter 9, page 299 | 0.405 | 1 | 1 | 100% |
| 7 | Thomas Blake and Dominic Coey (2014) Why Marketplace Experimentation is Harder Than It Seems: The Role of Test-Control Interference | 0.405 | 1 | 1 | 100% |
| 8 | Iavor Bojinov, David Simchi-Levi, and Jinglong Zhao (2022) Design and Analysis of Switchback Experiments | 0.405 | 1 | 1 | 100% |
| 9 | Alex Chin (2019) Regression Adjustments for Estimating the Global Treatment Effect in Experiments with Interference | 0.405 | 1 | 1 | 100% |
| 10 | WG Cochran (1939) Long-Term Agricultural Experiments | 0.405 | 1 | 1 | 100% |
Showing the top 10 of 18 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.