Luofeng Liao, Christian Kroer, Sergei Leonenkov, Okke Schrijvers, Liang Shi, Nicolas Stier-Moses, Congshan Zhang
arXiv 11 Feb 2024 · Mathematics — Statistics Theory · 1 citations (OpenAlex)
arXiv:2402.07322 · PDF · DOI · OpenAlex · Extracted main text
Online A/B testing is widely used in the internet industry to inform decisions on new feature roll-outs. For online marketplaces (such as advertising markets), standard approaches to A/B testing may lead to biased results when buyers operate under a budget constraint, as budget consumption in one arm of the experiment impacts performance of the other arm. To counteract this interference, one can use a budget-split design where the budget constraint operates on a per-arm basis and each arm receives an equal fraction of the budget, leading to “budget-controlled A/B testing.” Despite clear advantages of budget-controlled A/B testing, performance degrades when budget are split too small, limiting the overall throughput of such systems. In this paper, we propose a parallel budget-controlled A/B testing design where we use market segmentation to identify submarkets in the larger market, and we run parallel experiments on each submarket. Our contributions are as follows: First, we introduce and demonstrate the effectiveness of the parallel budget-controlled A/B test design with submarkets in a large online marketplace environment. Second, we formally define market interference in first-price auction markets using the first price pacing equilibrium (FPPE) framework. Third, we propose a debiased surrogate that eliminates the first-order bias of FPPE, drawing upon the principles of sensitivity analysis in mathematical programs. Fourth, we derive a plug-in estimator for the surrogate and establish its asymptotic normality. Fifth, we provide an estimation procedure for submarket parallel budget-controlled A/B tests. Finally, we present numerical examples on semi-synthetic data, confirming that the debiasing technique achieves the desired coverage properties.
appendix boundary found by appendix_command · 52% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Vincent Conitzer, Christian Kroer, Debmalya Panigrahi, Okke Schrijve… (2022) Pacing equilibrium in first price auction markets self | 0.894 | 7 | 4 | 71% |
| 2 | Luofeng Liao, Yuan Gao, and Christian Kroer (2023) Statistical inference for fisher market equilibrium self | 0.737 | 4 | 4 | 50% |
| 3 | Vincent Conitzer, Christian Kroer, Eric Sodomka, and Nicolas E Stier… (2022) Multiplicative pacing equilibria in auction markets self | 0.737 | 4 | 3 | 50% |
| 4 | Luofeng Liao and Christian Kroer (2023) Statistical inference and a/b testing for first-price pacing equilibria self | 0.693 | 9 | 4 | 33% |
| 5 | Yuan Gao and Christian Kroer (2022) Infinite-dimensional fisher markets and tractable fair division self | 0.644 | 3 | 2 | 67% |
| 6 | Guillaume W. Basse, Hossein Azari Soufiani, and Diane Lambert (2016) Randomization and the pernicious effects of limited budgets on auction experiments | 0.511 | 2 | 2 | 50% |
| 7 | Min Liu, Jialiang Mao, and Kang Kang (2021) Trustworthy and powerful online marketplace experimentation with budget-split design | 0.511 | 2 | 2 | 50% |
| 8 | Lihua Chen, Yinyu Ye, and Jiawei Zhang (2007) A note on equilibrium pricing as convex optimization | 0.511 | 2 | 2 | 50% |
| 9 | Santiago Balseiro, Anthony Kim, Mohammad Mahdian, and Vahab Mirrokni (2017) Budget management strategies in repeated auctions | 0.405 | 1 | 1 | 100% |
| 10 | Han Hong, Aprajit Mahajan, and Denis Nekipelov (2015) Extremum estimation and numerical derivatives | 0.405 | 1 | 1 | 100% |
Showing the top 10 of 43 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.
| Citing paper | Intensity | Mentions | Sections | |
|---|---|---|---|---|
| 1 | Tackling Interference Induced by Data Training Loops in A/B Tests: A Weighted Training Approach | 0.405 | 1 | 1 |
| 2 | 2501.11996 | 0.405 | 1 | 1 |