Shan Huang, Chen Wang, Yuan Yuan, Jinglong Zhao, Brocco, Zhang
arXiv 16 Aug 2023 · Econometrics · publishedManagement Science (2023) · 4 citations (OpenAlex)
arXiv:2308.08152 · PDF · DOI · OpenAlex · Extracted main text
Estimating the effects of long-term treatments through A/B testing is challenging. Treatments, such as updates to product functionalities, user interface designs, and recommendation algorithms, are intended to persist within the system for a long duration of time after their initial launches. However, due to the constraints of conducting long-term experiments, practitioners often rely on short-term experimental results to make product launch decisions. It remains open how to accurately estimate the effects of long-term treatments using short-term experimental data. To address this question, we introduce a longitudinal surrogate framework that decomposes the long-term effects into functions based on user attributes, short-term metrics, and treatment assignments. We outline identification assumptions, estimation strategies, inferential techniques, and validation methods under this framework. Empirically, we demonstrate that our approach outperforms existing solutions by using data from two real-world experiments, each involving more than a million users on WeChat, one of the world's largest social networking platforms.
appendix boundary found by none_found · 100% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Athey, Susan and Chetty, Raj and Imbens, Guido W and Kang, Hyunseung (2019) The surrogate index: Combining short-term proxies to estimate long-term treatment effects more rapidly and precisely | 1.000 | 19 | 4 | 100% |
| 2 | Deng, Alex and Xu, Ya and Kohavi, Ron and Walker, Toby (2013) Improving the sensitivity of online controlled experiments by utilizing pre-experiment data | 0.928 | 4 | 3 | 100% |
| 3 | Yang, Jeremy and Eckles, Dean and Dhillon, Paramveer and Aral, Sinan (2023) Targeting for long-term outcomes | 0.874 | 5 | 2 | 100% |
| 4 | Duan, Weitao and Ba, Shan and Zhang, Chunzhe (2021) Online Experimentation with Surrogate Metrics: Guidelines and a Case Study | 0.843 | 3 | 3 | 100% |
| 5 | Stock, James H and Watson, Mark W (2001) Vector autoregressions | 0.843 | 3 | 3 | 100% |
| 6 | Xiong, Ruoxuan and Athey, Susan and Bayati, Mohsen and Imbens, Guido (2019) Optimal experimental design for staggered rollouts | 0.843 | 3 | 3 | 100% |
| 7 | Baiocchi, Michael and Cheng, Jing and Small, Dylan S (2014) Instrumental variable methods for causal inference | 0.737 | 3 | 2 | 100% |
| 8 | Joffe, Marshall M and Greene, Tom (2009) Related causal frameworks for surrogate outcomes | 0.737 | 3 | 2 | 100% |
| 9 | Prentice, Ross L (1989) Surrogate endpoints in clinical trials: definition and operational criteria | 0.737 | 3 | 2 | 100% |
| 10 | Weir, Christopher J and Walley, Rosalind J (2006) Statistical evaluation of biomarkers as surrogate endpoints: a literature review | 0.737 | 3 | 2 | 100% |
Showing the top 10 of 64 scored citations.