EconBase
← All papers

Switchback Experiments under Geometric Mixing

Yuchen Hu, Stefan Wager

arXiv 1 Sep 2022 · Statistics — Methodology · 11 citations (OpenAlex)

arXiv:2209.00197 · PDF · DOI · OpenAlex · Extracted main text

Abstract

The switchback is an experimental design that measures treatment effects by repeatedly turning an intervention on and off for a whole system. Switchback experiments are a robust way to overcome cross-unit spillover effects; however, they are vulnerable to bias from temporal carryovers. In this paper, we consider properties of switchback experiments in Markovian systems that mix at a geometric rate. We find that, in this setting, standard switchback designs suffer considerably from carryover bias: Their estimation error decays as $T^{-1/3}$ in terms of the experiment horizon $T$, whereas in the absence of carryovers a faster rate of $T^{-1/2}$ would have been possible. We also show, however, that judicious use of burn-in periods can considerably improve the situation, and enables errors that decay as $\log(T)^{1/2}T^{-1/2}$. Our formal results are mirrored in an empirical evaluation.

Citation extraction

43
references
101
in-text mentions
43
distinct cited
2
self-citations
12,439
main-text words

appendix boundary found by appendix_command · 46% of the source is main text. Read the extracted text to check this.

Most heavily cited references

The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.

ReferenceIntensityMentionsSectionsMain text
1Bojinov, Iavor and Simchi-Levi, David and Zhao, Jinglong (2023) Design and analysis of switchback experiments0.92314479%
2Glynn, Peter W and Johari, Ramesh and Rasouli, Mohammad (2020) Adaptive experimental design with temporal interference: A maximum likelihood approach0.87462100%
3Farias, Vivek and Li, Andrew and Peng, Tianyi and Zheng, Andrew (2022) Markovian interference in experiments0.8434375%
4Imbens, Guido W and Rubin, Donald B (2015) Causal Inference in Statistics, Social, and Biomedical Sciences0.8435560%
5Xiong, Ruoxuan and Chin, Alex and Taylor, Sean (2023) Bias-variance tradeoffs for designing simultaneous temporal experiments0.84333100%
6Künsch, Hans R (1989) The jackknife and the bootstrap for general stationary observations0.7373367%
7Neyman, Jersey (1923) Sur les applications de la théorie des probabilités aux experiences agricoles: Essai des principes0.7373367%
8Aronow, Peter M and Samii, Cyrus (2017) Estimating average causal effects under general interference, with application to a social network experiment0.73732100%
9Bojinov, Iavor and Shephard, Neil (2019) Time series experiments and causal estimands: exact randomization tests and trading0.73732100%
10Leung, Michael P (2022) Rate-optimal cluster-randomized designs for spatial interference0.73732100%

Showing the top 10 of 43 scored citations.

Cited by, within the corpus

arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.

Citing paperIntensityMentionsSections
1Data-Driven Switchback Experiments: Theoretical Tradeoffs and Empirical Bayes Designs1.00064
2What is the Long-Term Value of Reliability?0.73732
3Estimating Effects of Long-Term Treatments0.64422
4ARMA-Design: Optimal Treatment Allocation Strategies for A/B Testing in Partially Observable Experiments0.64422
5Can We Validate Counterfactual Estimations in the Presence of General Network Interference?0.64422
6On Evolution-Based Models for Experimentation Under Interference0.64422
7Tackling Interference Induced by Data Training Loops in A/B Tests: A Weighted Training Approach0.51121
8Validating Causal Message Passing Against Network-Aware Methods on Real Experiments0.51121
9Causal Estimation of User Learning in Personalized Systems0.40511
102501.119960.40511