Shuze Chen, David Simchi-Levi, Chonghuan Wang
arXiv 29 Jul 2024 · Statistics — Methodology
arXiv:2407.19618 · PDF · DOI · OpenAlex · Extracted main text
Utilizing randomized experiments to evaluate the effect of short-term treatments on the short-term outcomes has been well understood and become the golden standard in industrial practice. However, as service systems become increasingly dynamical and personalized, much focus is shifting toward maximizing long-term outcomes, such as customer lifetime value, through lifetime exposure to interventions. Our goal is to assess the impact of treatment and control policies on long-term outcomes from relatively short-term observations, such as those generated by A/B testing. A key managerial observation is that many practical treatments are local, affecting only targeted states while leaving other parts of the policy unchanged. This paper rigorously investigates whether and how such locality can be exploited to improve estimation of long-term effects in Markov Decision Processes (MDPs), a fundamental model of dynamic systems. We first develop optimal inference techniques for general A/B testing in MDPs and establish corresponding efficiency bounds. We then propose methods to harness the localized structure by sharing information on the non-targeted states. Our new estimator can achieve a linear reduction with the number of test arms for a major part of the variance without sacrificing unbiasedness. It also matches a tighter variance lower bound that accounts for locality. Furthermore, we extend our framework to a broad class of differentiable estimators, which encompasses many widely used approaches in practice. We show that all such estimators can benefit from variance reduction through information sharing without increasing their bias. Together, these results provide both theoretical foundations and practical tools for conducting efficient experiments in dynamic service systems with local treatments.
appendix boundary found by none_found · 100% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Farias V, Li AA, Peng T, Zheng A (2022) Markovian interference in experiments | 1.000 | 19 | 8 | 100% |
| 2 | Pfeifer PE, Carraway RL (2000) Modeling customer relationships as markov chains | 1.000 | 6 | 3 | 100% |
| 3 | Tran A, Bibaut A, Kallus N (2023) Inferring the long-term causal effects of long-term treatments from short-term experiments | 1.000 | 5 | 4 | 100% |
| 4 | Simester DI, Sun P, Tsitsiklis JN (2006) Dynamic catalog mailing policies | 0.928 | 4 | 3 | 100% |
| 5 | Farias VF, Li H, Peng T, Ren X, Zhang H, Zheng A (2023) Correcting for interference in experiments: A case study at douyin | 0.874 | 5 | 2 | 100% |
| 6 | Hu Y, Wager S (2022) Switchback experiments under geometric mixing | 0.843 | 3 | 3 | 100% |
| 7 | Jones GL (2004) On the markov chain central limit theorem | 0.843 | 3 | 3 | 100% |
| 8 | Huang S, Wang C, Yuan Y, Zhao J, Zhang J (2023) Estimating effects of long-term treatments | 0.811 | 4 | 2 | 100% |
| 9 | Maystre L, Russo D, Zhao Y (2023) Optimizing audio recommendations for the long-term: A reinforcement learning perspective | 0.737 | 3 | 2 | 100% |
| 10 | Shi C, Wang X, Luo S, Zhu H, Ye J, Song R (2023) Dynamic causal effects evaluation in a/b testing with a reinforcement learning framework | 0.737 | 3 | 2 | 100% |
Showing the top 10 of 71 scored citations.