EconBase
← Back to paper

Switchback Experiments under Geometric Mixing

Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.

80,052 characters · 10 sections · 56 citation commands

Rendered from LaTeX for readability, not typeset faithfully. Citation keys are highlighted; maths is left as source; figures, tables and equation environments are summarised rather than reproduced; unrecognised commands are greyed out so nothing is silently dropped. Email addresses are removed.

Switchback Experiments under Geometric Mixing

\RUNTITLE{Switchback Experiments under Geometric Mixing}

\TITLE{Switchback Experiments under Geometric Mixing}

\ARTICLEAUTHORS{ \AUTHOR{Yuchen Hu}

\AFF{Management Science & Engineering, Stanford University, \EMAIL{[email removed]}}

\AUTHOR{Stefan Wager}

\AFF{Graduate School of Business, Stanford University, \EMAIL{[email removed]}}

}

\ABSTRACT{ The switchback is an experimental design that measures treatment effects by repeatedly turning an intervention on and off for a whole system. Switchback experiments are a robust way to overcome cross-unit spillover effects; however, they are vulnerable to bias from temporal carryovers. In this paper, we consider properties of switchback experiments in Markovian systems that mix at a geometric rate. We find that, in this setting, standard switchback designs suffer considerably from carryover bias: Their estimation error decays as $T^{-1/3}$ in terms of the experiment horizon $T$, whereas in the absence of carryovers a faster rate of $T^{-1/2}$ would have been possible. We also show, however, that judicious use of burn-in periods can considerably improve the situation, and enables errors that decay as $\log(T)^{1/2}T^{-1/2}$. Our formal results are mirrored in an empirical evaluation. }

\KEYWORDS{Causal Inference; Experimental Design; Markov Decision Process; Non-Stationary Stochastic Process}

\else

\hypersetup{colorlinks,citecolor=blue,urlcolor=blue,linkcolor=blue}

\allowdisplaybreaks \usetikzlibrary{graphs}

\graphicspath{{./figures/}}

\theoremstyle{plain} \newtheorem{prop}{Proposition}

\newtheorem{conj}[prop]{Conjecture} \newtheorem{coro}[prop]{Corollary} \newtheorem{lemm}[prop]{Lemma} \newtheorem{theo}[prop]{Theorem}

\theoremstyle{definition} \newtheorem*{exam}{Example} \newtheorem{defi}{Definition} \newtheorem{assu}{Assumption} \newtheorem{proo}{Proof} \newtheorem{model}{Model}

\theoremstyle{remark} \newtheorem{comm}{Comment} \newtheorem{rema}{Remark}

document\begin{abstract} The switchback is an experimental design that measures treatment effects by repeatedly turning an intervention on and off for a whole system. Switchback experiments are a robust way to overcome cross-unit spillover effects; however, they are vulnerable to bias from temporal carryovers. In this paper, we consider properties of switchback experiments in Markovian systems that mix at a geometric rate. We find that, in this setting, standard switchback designs suffer considerably from carryover bias: Their estimation error decays as $T^{-1/3}$ in terms of the experiment horizon $T$, whereas in the absence of carryovers a faster rate of $T^{-1/2}$ would have been possible. We also show, however, that judicious use of burn-in periods can considerably improve the situation, and enables errors that decay as $\log(T)^{1/2}T^{-1/2}$. Our formal results are mirrored in an empirical evaluation. \end{abstract} \fi

Introduction

Switchback experiments involve repeatedly toggling a treatment of interest on and off. There are several reasons to consider such experiments. In early work, brandt1938tests and cochran1941double studied the effect of diet on milk yield in dairy cows by alternating different diets for the same cow. Here, the motivation for using switchbacks was variance reduction: Different cows may have vastly different baseline yields, and so using a switchback can improve precision relative to designs that only give a single diet to each animal. In clinical settings, a closely related paradigm is that of N-of-1 trials, where a single individual is repeatedly randomized to different treatments to estimate personalized causal effects (see, e.g., liang2025randomization and references therein). More recently, there has been an explosion of interest in using switchbacks for A/B testing in online marketplaces, where a target intervention is toggled on and off at the market level bojinov2020design,chamandy2016experimentation,glynn2020adaptive,kastelman2018switchback,kohavi2020trustworthy,xiong2022bias. When applied at the market level, switchbacks help mitigate bias due to cross-unit interference or spillovers: For example, if the intervention involves a new pricing scheme, then using a market-level switchback avoids market distortions that could arise from simultaneously using two different pricing schemes at the same time.

A key challenge in using switchback experiments, however, is the problem of temporal carryovers or lag effects bojinov2020design,cochran1941double. Treatments assigned at any specific time point can have not only an immediate effect on the current outcome but also a longer-term effect due to the change of the system's (potentially latent) state glynn2020adaptive. And any approach to estimation and inference in switchbacks that does not account for temporal carryovers is prone to bias.

As a concrete example of a problem setting where carryovers are likely to matter, consider a switchback used to compare greedy vs. optimized matching strategies in a two-sided market in which jobs arrive sequentially and need to be matched to available workers (consider, e.g., a ride sharing or grocery delivery platform). Suppose that the greedy algorithm matches each incoming job to the nearest available worker, while the optimized strategy seeks to preserve available resources when possible (e.g., if a job could reasonably be matched to one of two available workers and one of them is likely to be in high demand in the future, then the job could be matched to the other one---even if they are slightly further). The optimized strategy promotes good positioning of available workers, and so may cause abnormally good initial performance for the greedy algorithm right after the switch; conversely, the greedy algorithm may lead to anomalously poor initial performance for the optimized strategy. Thus, a switchback analysis that ignores carryovers may, in this setting, severely underestimate the benefit of the optimized strategy relative to the greedy one.

In this paper, we study switchbacks under an assumption that the system we are intervening on is a (non-stationary) Markov decision process with mixing time $t_{\mix}$. Under this assumption, actions taken at time $t$ may affect the state of the system at all times $t' > t$, but the strength of these effects decays as $\exp[-(t' - t) / t_{\mix}]$. We let the state evolution of the system be arbitrarily non-stationary (e.g., the system may respond arbitrarily to the time of the day or exogenous shocks like weather); however, we assume the mixing rate of the system to be uniformly bounded from above (see Section (ref) for a formal model).

We first consider the behavior of a standard switchback experiment, i.e., one that toggles a binary treatment at selected time points and then estimates the treatment effect by taking the difference of the average outcome in periods where treatment is on and the average outcome when treatment is off. Under our model, we show that this standard switchback is severely affected by carryover bias: Given a single time series observed for $T$ time periods, the error of the standard switchback cannot be made to decay faster than $1/\sqrt[3]{T}$. This is markedly slower than the $1/\sqrt{T}$ rate of convergence one could have achieved with $T$ time periods and no carryovers.

We also find, however, that judicious use of burn-in periods each time the treatment assignment is switched can alleviate this issue. Specifically, we consider a switchback design that, given a pre-specified burn-in time $b$, throws out all observations that are within $b$ time periods of the last treatment switch. We then show that, using this design, we can estimate the global treatment effect for non-burn-in periods with errors decaying as $\sqrt{\log(T)/T}$ with a simple difference-in-means estimator. Furthermore, we propose a bias-corrected, weighted estimator that can use data from a switchback experiment to estimate the global treatment effect across the entire experiment with errors also decaying as $\sqrt{\log(T)/T}$.

We provide central limit theorems for both estimators that remain valid in non-stationary environments. Finally, in numerical experiments, we also find that---as expected---the use of burn-in periods considerably helps improve the behavior of switchbacks under our Markovian model for carryover effects.

Related work

The problem of carryover effects in switchbacks has been considered by a number of authors, including cochran1941double, bojinov2020design, glynn2020adaptive, and liang2025randomization. cochran1941double propose addressing the issue using a regression model, with a regression coefficient that captures the lagged effect of past treatment on future periods. bojinov2020design allow for the presence of carryovers, but assume that there is a fixed (known) time horizon $m$ such that all carryover effects of an action taken at time $t$ disappear by time $t + m$. glynn2020adaptive consider a Markov model related to the one used here (although their model is stationary); however, they address the problem of temporal carryovers by fitting the Markov model by maximum likelihood rather than by adapting the switchback. Similar to us, liang2025randomization also allow for infinite carryover effects, but in their framework the carryovers are encoded through an impulse-response outcome model, rather than through nonparametric state transitions.

Relative to existing results, we believe our approach may be helpful in settings where researchers cannot assume that carryover effects will fully vanish after a finite amount of time (as is the case in generic Markovian systems) and do not trust a regression model to capture all carryovers, but still want to use a switchback design to estimate treatment effects. We also emphasize that, although we make Markovian assumptions for analytic purposes, we do not require the researcher to be able observe the state of the system; we only depend implicitly on this modeling assumption via the mixing time. In contrast, the maximum likelihood approach of glynn2020adaptive and the Differences-In-Q approach of farias2022markovian require observing the full state in order to compute the estimators. The problem of treatment effect estimation in Markov decision processes under general designs with sequential ignorability is further considered in liao2021off, liao2022batch, kallus2020double,kallus2022efficiently and mehrabi2024off, but these results again require observing the full state of the system.

Our approach to modeling non-stationarity builds on the well-known Neyman model for finite-population causal inference neyman1923applications. We assume that our switchback is run over $t = 1, \, \ldots, \, T$ time periods and---like in the Neyman model where each study participant can be different from the others---we allow for each time period $t$ to be different from the others. All we assume is that the system is Markovian (i.e., memoryless), and that it mixes over time (i.e., the effect of past events washes out over time); see Assumptions (ref) and (ref) for details. Closely related to this finite-population viewpoint is the framework of bojinov2019time, which provides the first pure design-based analysis of panel experiments, whereas our model additionally incorporates tools from dynamic processes that enable the estimation of global effects. Our central limit theorems build on statistical tools originally built for finite-population causal inference aronow2017estimating, bojinov2019time, bojinov2021panel, bojinov2020design, leung2022causal, li2017general, lin2013agnostic; however, to our knowledge this paper is the first to use these tools to study non-stationary Markov decision processes.

Finally we note that, at a high level, the switchback design can be regarded as a special case of the cluster randomized experiment imbens2015causal in the temporal setting. From this perspective, our work is relevant to the strand of the literature focusing on the optimal design of a clustered randomized experiment without the presence of well-defined clusters athey2018exact,harshaw2021design,leung2021rate,leung2022causal,savje2021causal,ugander2013graph,ugander2020randomized,viviano2020experimental. Much of this literature relies on the existence of some exposure mapping function that fully characterizes the interference structure aronow2017estimating,manski2013identification, with some recent exceptions including leung2021rate,leung2022causal and savje2021causal that allow for misspecified exposure mappings and instead only assume, e.g., a rate of decay on the spillover effects. In our Markovian model, the carryover bias never disappears---it only decays over time---and as such our results are closer to these recent papers allowing approximate or misspecified exposure mappings than to analyses that depend on a well-specified exposure mapping. In a recent advance, jia2024clustered build on a preprint version of this paper to develop a clustered switchback design that allows for both spatial spillovers and Markovian carryovers.

A Markovian Model for Temporal Carryovers

We collect data from following a single system for a time period of length $T$. At each time point $t=1,\dots,T$, we assign a binary treatment $W_t\in\{0,1\}$ to the system and observe an immediate outcome $Y_t\in \RR$. In switchback experiments, the horizon is divided into blocks of equal length $l$, and treatments are assigned such that $W_t=W_s$ if $t$ and $s$ are from the same block. For simplicity, we assume that there exists $k\in \mathbb{Z}^+$ such that $kl=T$.

We assume that we are in a setting where there exists a (potentially unobserved) state variable $S_t\in \mathcal{S}$, and that the triples $(S_t, W_t,Y_t)$ form a non-stationary Markov decision process with transition operators $P_t(\cdot | \cdot)$. This transition operator can vary arbitrarily across time and captures the influence of all exogenous events on the system. The MDP assumption is illustrated in Figure (ref).

figure[figure omitted — 1,645 chars of source]
assuFor all time points $t = 1, \, \ldots, \, T$, the evolution of the underlying state $S_t$ is governed by a pair of (deterministic) state transition distributions $P_t^{0}(\cdot | \cdot)$ and $P_t^{1}(\cdot | \cdot)$ that \begin{equation} S_{t+1} \cond \ff_t, Y_t \sim P_t^{W_t}\p{\cdot | S_t}, \end{equation} where $\ff_t =\sigma\p{S_t, \, W_t, \,Y_{t-1}, \, S_{t-1}, \, W_{t-1}, \, \cdots} $ contains all information available from time $-\infty$ to time $t$. Furthermore, there exists a set of functions $\mu_t:\mathcal{S}\times\mathcal{W}\to\RR$ such that $Y_t$ is a noisy observation of $\mu_t(S_t, W_t)$, i.e., $Y_t=\mu_t(S_t, W_t)+\epsilon_t$ for some mutually independent noises $\epsilon_t$ satisfying $\EE{\epsilon_t\cond \ff_t}=0$ and $\Var{\epsilon_t\cond \ff_t}\le \sigma^2<\infty$.

Although the system is not stationary in time, we assume that the system “mixes” at a rate of at least $1/t_{\mix}$ at each time point. Mixing assumptions are common in the literature in contextual bandits and reinforcement learning, and a mixing assumption of the form (ref) is used and discussed by, e.g., van1998learning, even2005experts and hu2021off. In particular, hu2021off use such an assumption for off-policy evaluation in a partially observed Markov decision process. Unlike us, however, these papers consider mixing assumptions in a stationary environment (i.e., where $P_t^w = P^w$ is the same for all time periods $t$), whereas here we allow the problem dynamics to change arbitrarily from one period to the next. Intuitively, this mixing assumption implies that, for any realized sequence of environments (or formally, transition kernels $\{P_t\}_t$), the impact of past treatment assignments decays exponentially fast. In this sense, the system gradually ‘forgets’ its history. This excludes dynamics in which the effects of early actions can accumulate or persist.

assuThere exists a mixing time $t_{\mix}<\infty$ such that, for all time points $t = 1, \, \ldots, \, T$ and $w\in\{0,1\}$, \begin{equation} \Norm{f'P_t^{w}-fP_t^{w}}_{TV}\le \exp\p{-1/t_{\mix}}\Norm{f'-f}_{TV} \end{equation} for any pair of distributions $f$ and $f'$ on $S_t$.

\ifms

exam[Example: Ridesharing Platform] Consider a ridesharing marketplace in which the platform repeatedly dispatches drivers to rider requests: \begin{itemize} • Trearment $W_t$: At each time, the platform selects a dispatching policy, for instance, a greedy nearest-driver rule ($W_t=0$) or an optimized rule that accounts for future demand patterns ($W_t=1$). • State $S_t$: The latent system state collects all operational conditions that influence future outcomes. This may include, but is not limited to, all online drivers’ locations and routes, outstanding requests, drivers’ beliefs, and accumulated constraints from earlier dispatches. This state is high-dimensional, difficult to track in practice, and may be only partially observed or even fully unobserved by the analyst. • Outcome $Y_t$: The observed outcome at time $t$ may be a performance metric such as revenue, profit, fulfillment rate, or rider wait time. \end{itemize} This setting naturally exhibits nonstationarity: exogenous conditions such as weather, time of day, day of week, special events, and holidays can alter rider arrival patterns and driver supply in ways that vary arbitrarily across time. These environmental factors may not be constant or mixing at all. At the same time, the mixing assumption captures the idea that, conditional on a fixed treatment policy and a fixed sequence of environment, the operational state of the system gradually forgets past actions. For example, the impact of a suboptimal dispatch decision dissipates as drivers complete their trips and new requests arrive. Thus, the latent state can mix at a geometric rate even though the exogenous environment introducing nonstationarity may evolve unpredictably. This combination of arbitrary environmental shifts and geometric mixing in the operational state is characteristic of many online marketplaces.

\else

exam[Ridesharing Platform] Consider a ridesharing marketplace in which the platform repeatedly dispatches drivers to rider requests: \begin{itemize} • Trearment $W_t$: At each time, the platform selects a dispatching policy, for instance, a greedy nearest-driver rule ($W_t=0$) or an optimized rule that accounts for future demand patterns ($W_t=1$). • State $S_t$: The latent system state collects all operational conditions that influence future outcomes. This may include, but is not limited to, all online drivers’ locations and routes, outstanding requests, drivers’ beliefs, and accumulated constraints from earlier dispatches. This state is high-dimensional, difficult to track in practice, and may be only partially observed or even fully unobserved by the analyst. • Outcome $Y_t$: The observed outcome at time $t$ may be a performance metric such as revenue, profit, fulfillment rate, or rider wait time. \end{itemize} This setting naturally exhibits nonstationarity: exogenous conditions such as weather, time of day, day of week, special events, and holidays can alter rider arrival patterns and driver supply in ways that vary arbitrarily across time. These environmental factors may not be constant or mixing at all. At the same time, the mixing assumption captures the idea that, conditional on a fixed treatment policy and a fixed sequence of environment, the operational state of the system gradually forgets past actions. For example, the impact of a suboptimal dispatch decision dissipates as drivers complete their trips and new requests arrive. Thus, the latent state can mix at a geometric rate even though the exogenous environment introducing nonstationarity may evolve unpredictably. This combination of arbitrary environmental shifts and geometric mixing in the operational state is characteristic of many online marketplaces.

\fi

With the model in place, we can now formalize the causal effects we aim to learn. In simple randomized controlled trials without interference, it is customary to focus on estimating the average treatment effect imbens2015causal, $\text{ATE} = \frac{1}{T} \sum_{t = 1}^T \EE{Y_t(1) - Y_t(0)}$, where $Y_t(w)$ denotes the potential outcome we would observe in the $t$-th period by setting $W_t = w$. In the presence of temporal carryovers, however, these potential outcomes (and the induced average treatment effect) are no longer meaningful because the distribution $Y_t$ doesn't just depend on $W_t$, but can also depend on past actions $\{W_{t-1},W_{t-2},\dots\}$.

figure[figure omitted — 434 chars of source]

Figure (ref) illustrates difficulties associated with non-stationarity and temporal carryovers in our model. Shaded regions indicate times when treatment is on $(W_t = 1)$ while clear regions indicate times when treatment is off ($W_t = 0$). Meanwhile, dashed vertical lines indicate exogenous shocks to the system unrelated to the treatment; these shocks cause the functions $P^t(\cdot|\cdot)$ and $\mu_t(\cdot)$ in Assumption (ref) to change. Given this setting, the dashed blue line shows the average behavior we would get if the system were given control assignment throughout, the dotted green one shows average behavior with treatment throughout, and the solid red line shows average behavior with treatment toggled as in a switchback. Thanks to our mixing condition (Assumption (ref)), the red line eventually converges to the blue or green lines after each time we toggle the treatment---but this change is not instantaneous.

Intuitively, we can think of the average gap between the green and blue curves in Figure (ref) as quantifying an average effect we want to estimate. To formalize this notion, for any treatment sequence $w \in \{0, 1\}^{\mathbb{Z}_+}$, we write

equation[equation omitted — 105 chars of source]

\sloppy{i.e., $\law_w^t$ is the long-term outcome distribution at time $t$ under the treatment assignment sequence $w$, which is conditioned implicitly on the sequence of transitional kernels $\{P^1_t,P^0_t,P^1_{t-1},P^0_{t-1},\cdots\}$. We use short-hand $\law_0^t$ and $\law_1^t$ for distributions indexed by “pure” histories, i.e., where $w$ is all $0$s or all $1$s. Given this notation, we define the following estimands.}

defiUnder Assumption (ref), the stable treatment effect at time $t$ is\footnote{Notice that this is equivalent to a potential outcome specification, where $Y_t = Y_t(w)$ and $w$ indexes the full history.} \begin{equation} \tau_t= \EE[\law_1^t]{Y_t}-\EE[\law_0^t]{Y_t}, \end{equation} and the global average treatment effect (GATE) is \begin{equation} \tau_{\GATE} = \frac{1}{T}\sum_{t=1}^T \tau_t. \end{equation}

The global average treatment effect is a direct analogue to the estimand considered in ugander2013graph and xiong2022bias for the Markov setting. In a cross-sectional randomized trial with spillover effects between the units, GATE is the average difference in outcomes when all units are exposed to treatment versus when all units are exposed to control, conditionally on the group of recruited subjects. Here in the longitudinal setting, our estimand of interest is the average difference in outcomes when the system is always exposed to treatment versus when the system is always exposed to control, conditionally on the time period during which the experiment is conducted. We emphasize that this estimand is only well-defined conditionally on environments before and during the experiment, i.e., on the sequence of transition kernels from time $-\infty$ to time $T$. This is also related to the long-run average reward studied in glynn2020adaptive, except that in our setting the system is never stationary due to the exogenous and arbitrary transition operator.

\ifms

exam[Example: Ridesharing Platform Continued] To further understand the conditional estimand in Definition (ref), it is helpful to revisit the ridesharing example introduced above. Recall that the state $S_t$ captures operational conditions that evolve as a consequence of the dispatching policy, which a decision-maker will typically expect to behave similarly in future periods if the same treatment policy were adopted. For example, if the platform were to switch permanently to an optimized dispatching policy, under similar environmental conditions, the steady-state distribution of driver locations and queue lengths would be determined by that policy. By contrast, the exogenous environmental conditions may change unpredictably and need not repeat in future deployment environments. These environmental factors are embedded in the time-varying transition operators $\cb{P_t}$. Because they are external to the system’s operational response and may vary arbitrarily over time, it is natural for the estimand to be conditional on these environmental variables for robustness. Consequently, the GATE captures the expected difference in outcomes between always using the treatment versus always using the control in the same sequence of environmental conditions under which the experiment was conducted. For instance, if the experiment is run throughout the month of December 2025, then GATE represents the effect of adopting the treatment for the month of December 2025 (with its own pattern of holidays and weather), but not necessarily the effect for an entire future year. This distinction is important for decision-making. Managers are typically interested in understanding how a new policy would perform in future periods with similar environmental profiles to those observed during the experiment. Meanwhile, extrapolating the treatment effect to environments not represented in the experiment (e.g., inferring July effects from December data) would require additional modeling assumptions about how $\cb{P_t}$ might change outside the experimental window. By conditioning on the realized sequence of environmental conditions but averaging over the operational factors that the policy itself influences, GATE aligns naturally with the causal question that switchback experiments are designed to answer.\footnote{Mathematically, the random outcome noise $\epsilon_t$ could be absorbed into the time-indexed outcome function without affecting the numerical value of the estimand. However, doing so changes the philosophical interpretation of the estimand, and specifically, what sources of randomness are being averaged over. This also alters the variance structure of the estimator, which may in turn lead to a different optimal design.}

\else

exam[Ridesharing Platform Continued] To further understand the conditional estimand in Definition (ref), it is helpful to revisit the ridesharing example introduced above. Recall that the state $S_t$ captures operational conditions that evolve as a consequence of the dispatching policy, which a decision-maker will typically expect to behave similarly in future periods if the same treatment policy were adopted. For example, if the platform were to switch permanently to an optimized dispatching policy, under similar environmental conditions, the steady-state distribution of driver locations and queue lengths would be determined by that policy. By contrast, the exogenous environmental conditions may change unpredictably and need not repeat in future deployment environments. These environmental factors are embedded in the time-varying transition operators $\cb{P_t}$. Because they are external to the system’s operational response and may vary arbitrarily over time, it is natural for the estimand to be conditional on these environmental variables for robustness. Consequently, the GATE captures the expected difference in outcomes between always using the treatment versus always using the control in the same sequence of environmental conditions under which the experiment was conducted. For instance, if the experiment is run throughout the month of December 2025, then GATE represents the effect of adopting the treatment for the month of December 2025 (with its own pattern of holidays and weather), but not necessarily the effect for an entire future year. This distinction is important for decision-making. Managers are typically interested in understanding how a new policy would perform in future periods with similar environmental profiles to those observed during the experiment. Meanwhile, extrapolating the treatment effect to environments not represented in the experiment (e.g., inferring July effects from December data) would require additional modeling assumptions about how $\cb{P_t}$ might change outside the experimental window. By conditioning on the realized sequence of environmental conditions but averaging over the operational factors that the policy itself influences, GATE aligns naturally with the causal question that switchback experiments are designed to answer.\footnote{Mathematically, the random outcome noise $\epsilon_t$ could be absorbed into the time-indexed outcome function without affecting the numerical value of the estimand. However, doing so changes the philosophical interpretation of the estimand, and specifically, what sources of randomness are being averaged over. This also alters the variance structure of the estimator, which may in turn lead to a different optimal design.}

\fi

remaOutcomes in temporal settings are inherently dynamic, but using a fully dynamic framework typically forces the decision-maker to impose strong assumptions about how the system evolves over time. In contrast, traditional finite-population, design-based approaches treat potential outcomes as fixed and attribute all randomness to the treatment assignment; this ensures robustness to adversarial changes in the potential outcomes neyman1923applications. We extend this idea to a dynamic setting by conditioning on the sequence of environmental conditions (equivalently, the sequence of transition kernels) while still allowing randomness to arise from the latent state transitions and outcome generation. In this way, the framework remains robust to adversarial changes in the environment while capturing the stochastic temporal dependence inherent in dynamic systems. As will become clearer later, this hybrid perspective enables robust design-based inference while accommodating the flexibility needed in nonstationary operational settings. Our semi-design-based formulation is conceptually related to the finite-population time-series framework of bojinov2019time and bojinov2021panel, in that both condition on an exogenous sequence (potential outcomes versus environment transitions). However, the goals differ largely: their lag-p dynamic effect compares specific, finite windows of treatment paths, whereas the GATE characterize the long-run distributional effect of policy-level interventions. Under our framework, when the system mixes sufficiently quickly and $p$ is chosen large enough, the two estimands can be numerically close.

Difference-in-Means Estimation

In this section, we develop estimation guarantees for the estimands in Definition (ref) using switchback experiments. We begin by formalizing the switchback design and the associated natural treatment effect estimator.

defiThe regular Bernoulli switchback design is characterized by a block-length $l > 1$ and a time horizon $T$, and assigns treatment as \begin{equation} W_t \cond W_{t-1} = \begin{cases} Bernoulli(0.5) & if $t = (i-1)l + 1$ for some $i = 1, \, \ldots, \, \lfloor T/l \rfloor$, \\ W_{t-1} & else. \end{cases} \end{equation} For convenience, we write $Z_i = W_{(i-1)l + 1}$ to the treatment given to the $i$-th block, and write $k = \lfloor T/l \rfloor$ for the total number of blocks. The regular switchback estimator takes data from a regular switchback along with an (optional) burn-in length $0 \leq b < l$, and estimates the overall treatment effect via a difference-in-means that discards burn-in periods (if a non-zero burn-in length is used),\footnote{We follow the convention to set $0/0$ to be $0$.} \begin{equation} \begin{split} &\htau_{DM}^{(l,b)}=\frac{1}{k_1(l - b)}\sum_{\cb{i : Z_i = 1}} \sum_{s = b + 1}^l Y_{(i - 1)l + s} - \frac{1}{k_0(l - b)}\sum_{\cb{i : Z_i = 0}} \sum_{s = b + 1}^l Y_{(i - 1)l + s}, \\ &k_w = \abs{\cb{i = 1, \, \ldots, \, \lfloor T/l \rfloor : Z_i = w}}. \end{split} \end{equation}

Our definition of the regular switchback design and estimator is closely related to the one used in bojinov2020design, in that we both consider experiments that may randomly switch treatment according to Bernoulli draws at pre-determined time-points, and both consider estimators that discard observations right after switches to mitigate bias. However, our specification of the estimator (ref) differs from the one used in bojinov2020design in that it is purely algorithmic: the estimator \smash{$\htau_{\text{DM}}^{(l,b)}$} does not depend explicitly on the model (Assumptions (ref) and (ref)); rather, it is a function of the block length $l$ and burn-in period $b$ that one could seek to justify from a number of perspectives. In contrast, bojinov2020design start by specifying conditions under which carryover effects are guaranteed to vanish, and then consider estimation using the natural Horvitz-Thompson estimator induced by their modeling assumptions. We note that in our setting, i.e., under Assumptions (ref) and (ref), carryover effects never fully vanish---they simply decay at an exponential rate. Thus, a literal application of the construction of bojinov2020design would not be consistent, because no data could ever be used after the first switch. Beyond the work of bojinov2020design, we are not aware of prior studies of switchback experiments that consider using burn-in periods or other analogous estimation techniques.

Our first result is an error decomposition for regular switchback estimators under our geometric mixing assumptions. The estimator \smash{$\htau_{\text{DM}}^{(l,b)}$} has two sources of bias. First, it has bias due to the long-term effect carryover from the past treatments. As can be observed from Figure (ref), when the treatment condition switches from $w$ to $1-w$, the curve will take time to converge from the mean outcome curve under $w$ to the mean outcome curve under $1-w$. Thus, there is always a bias due to the mixing of the process after each switchback, and the closer it is to the switch point, the larger the bias we will encounter in estimating $\tau_t$. The use of burn-in periods can mitigate carryover bias---but this results in a different source of potential bias due to ignoring outcomes in the burn-in periods.

theoUnder Assumptions (ref) and (ref), suppose in addition that the conditional expectations are bounded almost surely by a constant $\Lambda$ such that \[\abs{\EE{Y_t|S_t,W_t}}= \abs{\mu_t(S_t, W_t)}\le_{a.s.}\Lambda, \] and that the heterogeneity in treatment effects is bounded by a constant $\Psi$, i.e., \[\abs{\tau_t-\tau_s}\le \Psi \] for arbitrary $t,s\in\{1,\dots,T\}$. Given the block length $l$ and the burn-in period $b$, define $k=k_1+k_0=\lfloor T/l \rfloor$, and let \begin{equation} \tau_{\FATE}^{(l, b)} := \frac{1}{k(l-b)}\sum_{i=1}^k \sum_{s=b+1}^l \tau_{(i-1)l+s} \end{equation} denote the Filtered Average Treatment Effect (FATE) that only considers non-burn-in periods. Then, the bias of the regular switchback estimator $\htau_{\text{DM}}^{(l,b)}$ can be bounded as \begin{equation} \abs{\EE{\htau_{DM}^{(l,b)}}-\tau_{\FATE}^{(l, b)}}\leq \underbrace{\frac{4\Lambda}{1-\exp\p{-1/t_\mix}}\cdot\frac{\exp\p{-b/t_{mix} }}{l-b}}_{mixing bias} + \oo\p{2^{-k}}, \end{equation} and furthermore \begin{equation} \abs{\tau_{\FATE}^{(l, b)} - \tau_{\GATE}} \leq \underbrace{\Psi \, b/l}_{burn-in bias}. \end{equation} Meanwhile, the variance of \smash{$\htau_{\text{DM}}^{(l,b)}$} can be bounded as \begin{equation} \begin{split} \Var{\htau_{DM}^{(l,b)}}&\le \frac{12\Lambda^2}{k} +\frac{4\sigma^2}{k(l-b)} +\frac{16\Lambda^2\exp\p{-b/t_\mix}}{(1-\exp\p{-1/t_\mix})^2}\cdot\frac{1}{k(l-b)^2} +\oo\p{\frac{1}{k^2}}, \end{split} \end{equation} where the three leading terms in the above bound represent the variance from the clustered treatment assignment, the unpredictable outcome noise, and covariance due to carryover effects respectively.

Theorem (ref) highlights the tradeoffs introduced by burn-in periods in the presence of carryover effects. The upper bound on bias reflects how burn-ins simultaneously mitigate mixing bias, bias caused by switchbacks generating treatment histories inconsistent with the always-treat and never-treat regimes we aim to evaluate, while at the same time inducing burn-in bias, since discarding early observations implicitly replaces $\tau_{t}$ for $t\in\text{burn-in}$ with with $\tau_{t}$ for $t\notin\text{burn-in}$. The existence of the carryover effect also inflates the variance by inducing a positive correlation between mean outcomes observed at different times points. In addition, the variance depends on the noise level in the outcomes and on how clustered the treatment assignments are. Nevertheless, as long as $l(T)-b(T)$ is bounded away from zero, the variance will be dominated by the term from the clustered assignment; this mirrors results available in the context of generic cluster-randomized experiments leung2021rate.

Having characterized how burn-in periods and block length affect the bias and variance of the DM estimator, we now turn to the practical question of choosing $l$ and $b$ to optimize estimation accuracy. One might expect burn-in periods to be beneficial; indeed, they are widely used in industry to mitigate carryover effects in switchback experiments. However, for the purpose of estimating GATE, we show that this intuition is misleading: any positive burn-in length $b>0$ is in fact rate-suboptimal. The optimal strategy is to set $b=0$ and rely exclusively on long blocks to control mixing bias, except in cases where the burn-in bias is negligible.

coroUnder the conditions of Theorem (ref), with the choice that \begin{equation} l \asymp T^{1/3}\qquad and \qquad b=0, \end{equation} $\htau_{\text{DM}}^{(l,b)}$ achieves the error bound \begin{equation} \EE{\p{\htau_{DM}^{(l,b)}-\tau_{\GATE}}^2} = \oo\p{T^{-2/3}}. \end{equation} Furthermore, no regular switchback estimator as given in Definition (ref) can guarantee a faster-than-$\mathcal{O}\p{ T^{-2/3}}$ rate of convergence in this setting.
coroUnder the conditions of Theorem (ref), if the treatment effect is sufficiently stable so that \begin{equation} \abs{\tau_{\FATE}^{(l, b)} - \tau_{\GATE}} = \smallO(\sqrt{\log T\cdot T^{-1}} ) \end{equation} for the chosen design $(l,b)$, then for any bounded constant $C_1>0$, with the choice that \begin{equation} l= b+C_1 \quad and \quad b = \frac{t_\mix}{2} \log T, \end{equation} $\htau_{\text{DM}}^{(l,b)}$ achieves the error bound \begin{equation} \EE{\p{\htau_{DM}^{(l,b)}-\tau_\GATE}^2}= \oo\p{ \log T\cdot T^{-1}}. \end{equation}

Given side by side, these two results present a comprehensive characterization of the role of burn-in periods in regular Bernoulli switchbacks as specified in Definition (ref). When the treatment effects inside and outside burn-in periods differ substantially, the burn-in bias from using a non-trivial $b$ grows faster than the carryover bias decays, thus making the choice of $b = 0$ (i.e., no burn-in period) rate optimal. Consequently, the optimal strategy for estimating $\tau_{\GATE}$ using regular Bernoulli switchbacks is then to use a very long block length of $T^{1/3}$, resulting in an $\mathcal{O}(T^{-1/3})$ rate of convergence in root-mean squared error. In contrast, when the environment is sufficiently stable so that the burn-in bias is negligible (e.g., when the environment inside and outside the burn-in period behaves similarly, or when the decision-maker adopts a super-population perspective in which environmental variation is regarded as random rather than systematic), using a non-trivial burn-in period becomes beneficial: it removes the leading carryover bias and allows the estimator to achieve an $\mathcal{O}(T^{-1/2})$ rate up to logarithmic factors. In Supplementary material, we provide further characterization of optimal designs for the DM estimator and discuss settings where $\tau_{\FATE}^{(l, b)}$ is close to $\tau_{\GATE}$ or can be regarded as a reasonable estimand for downstream decision-making.

We end this section by giving conditions where the error bounds from Theorem (ref) can be sharpened into a central limit theorem. This provides an asymptotic theory for semi-design-based inference in dynamic systems, which has remained largely unexplored in the literature. To state our result, we use the following notation: For all blocks $i=1,\dots,k$, let

equation[equation omitted — 81 chars of source]

where as usual $l$ denotes the block length. Furthermore, for $w = 0, \, 1$, let $\bY_i(w)$ denote the potential outcome that would be observed for block $i$ if its treatment assignment were set to $Z_i=w$ while holding the rest of the treatment history unchanged. We also write

equation[equation omitted — 146 chars of source]

i.e., $\bmu_i(w)$ would be the expectation of $\bY_i$ in a system that always receives treatment $w$, and $\bM_i(w)$ is the expectation of $\bY_i$ in our switchback given $Z_i = w$. We note that, following (ref), we have

equation[equation omitted — 116 chars of source]

Finally, for simplicity of exposition we assume that, although our system is non-stationary, relevant empirical variances converge in large samples; similar assumptions are often made in finite population central limit theorems aronow2017estimating,lin2013agnostic.

assuWe consider a regular switchback with $l$ and $b$ chosen such that the following limits exist: \begin{equation} \begin{split} &\frac{1}{k} \sum_{i = 1}^k \p{\bmu_i(0) - \frac{1}{k} \sum_{j = 1}^k \bmu_j(0)}^2 \rightarrow V_0,\quad\frac{1}{k} \sum_{i = 1}^k \p{\bmu_i(1) - \frac{1}{k} \sum_{j = 1}^k \bmu_j(1)}^2 \rightarrow V_1, \\ &\frac{1}{k} \sum_{i = 1}^k \p{\bmu_i(1)-\frac{1}{k} \sum_{j = 1}^k \bmu_j(1)}\p{\bmu_i(0)-\frac{1}{k} \sum_{j = 1}^k \bmu_j(0)} \rightarrow V_{01}. \end{split} \end{equation}
theoUnder the conditions in Theorem (ref), suppose in addition that Assumption (ref) holds, that $Y_t$ are upper bounded such that $\abs{Y_t}\le \Gamma_0$ for some $\Gamma_0<\infty$, and that there exists a constant $\sigma_0>0$ such that $\Var{\epsilon_t\cond \ff_t}\ge \sigma_0^2$. Then provided that we choose the burn-in length $b$ so that $l-b=\oo\p{1}$ and \begin{equation} k \exp\p{-2b/t_{mix}} \rightarrow 0, \end{equation} as $k\to\infty$, we have \begin{equation} \sqrt{k} \p{\htau_{DM}^{(l,b)}-\tau_{\FATE}^{(l, b)}} \to_d \nn\p{0, \, V}, \ \ \ V = V_0 + V_1 +2V_{01} + \Sigma_\Delta. \end{equation} where \begin{equation} \Sigma_\Delta=\EE{\p{{\cb{\bY_i(1)-\bM_i(1)}\frac{Z_{i}}{0.5}-\p{\bY_i(0)-\bM_i(0)}\frac{(1-Z_{i})}{0.5}}}^2}. \end{equation}

As in the classical Neyman finite-population causal inference framework imbens2015causal, the variance expression in (ref) does not in general admit an unbiased estimator. The challenge arises from the cross-term $V_{01}$, whose estimation would require observing both treatment assignments within the same block; this is impossible by design and therefore unidentifiable from the realized data. Nevertheless, conservative confidence intervals can still be constructed by bounding the cross-term via the Cauchy–Schwarz inequality, which ensures that the resulting intervals have at least the nominal coverage probability. In Section (ref) of the supplemental materials, we present a jackknife variance estimator miller1974jackknife and show that, under our semi-design-based framework, it converges to such a conservative variance bound.

A Bias-Corrected Estimator

In the previous section, we found that a regular switchback estimator can only achieve an error rate of $\oo\p{T^{-1/3}}$ in estimating $\tau_{\GATE}$. We also showed that, when considering a design with burn-in periods, it is possible to overcome the challenge brought by carryover bias and achieve a rate of $\oo(\sqrt{\log T/T})$ if the environment doesn't vary much inside and outside the burn-in periods. However, because burn-in periods deterministically exclude observations, introducing burn-ins can also create substantial bias for estimating $\tau_{\GATE}$ in general settings. This tension raises a natural question: Is it fundamentally hard to get good estimates of $\tau_{\GATE}$ using data collected from a switchback design, or could we achieve better results by analyzing the data produced by the switchback using a different estimator?

The answer to this last question turns out to be affirmative: We can also achieve a $\oo(\sqrt{\log T/T})$ rate of convergence for $\tau_{\GATE}$ by using an inverse-propensity-type bias-corrected estimator that takes data collected from a regular Bernoulli switchback design as its input. Our proposed bias-corrected estimator utilizes the fact that, with Bernoulli randomization, the assigned treatment does not actually change at the beginning of every block, and thus observations within burn-in periods need not always be thrown away. Using shorthand $k_{ww'}=\sum_{i=2}^kI(Z_i = w, \, Z_{i-1} = w')$, we then estimate $\tau_{\GATE}$ as

equation[equation omitted — 342 chars of source]

The intuition behind our proposed estimator is as follows. In a Bernoulli switchback, a switch at the beginning of each block occurs randomly with a probability of $0.5$. Each block in a switchback is divided into two parts: a burn-in part and a focal part. The observations from the focal part are always included in the estimation with weight $1$, while the observations from the burn-in part are included with a probability of $0.5$ (when a switch does not occur), and are given a weight of approximately $2$. Hence, on average, observations from all periods are weighted equally, but those from the initial few time periods after a realized switch are still discarded. The following result establishes the claimed rate of convergence for $\tau_{\GATE}$ using this bias-corrected estimator.

theoUnder the assumptions of Theorem (ref), the bias of $\htau_{\text{BC}}^{(l,b)}$ can be bounded as \begin{equation} \abs{\EE{{\htau_{BC}^{(l,b)}}}-\tau_{\GATE}}\le \frac{4\Lambda}{1-\exp\p{-1/t_\mix}}\cdot\frac{\exp\p{-b/t_{mix} }}{l}+\frac{2\Lambda}{k}+\oo\p{2^{-k}}. \end{equation} Meanwhile, the variance of $\htau_{\text{BC}}^{(l,b)}$ can be bounded as \begin{equation} \begin{split} \Var{\htau_{BC}^{(l,b)}}&\le \frac{28\Lambda^2}{k} + \frac{16\Lambda^2\exp\p{-b/t_\mix}}{(1-\exp\p{-1/t_\mix})^2}\cdot\frac{1}{kl^2}+\frac{8\sigma^2}{kl}+\oo\p{\frac{1}{k^2l}}+\oo\p{2^{-k}}. \end{split} \end{equation} Furthermore, for any bounded constant $C_2>0$, with the choice that \begin{equation} l=b+C_2 \quad and \quad b=\frac{t_{\mix}}{2}\log T \end{equation} $\htau_{\text{BC}}^{l,b}$ achieves the error bound \begin{equation} \EE{\p{\htau_{BC}^{(l,b)}-\tau_\GATE}^2}\le 14\Lambda^2t_\mix\cdot \log T\cdot T^{-1} +\smallO\p{ \log T\cdot T^{-1}} \end{equation} in estimating $\tau_\GATE$.

One question that is left open by Corollary (ref) is how to precisely choose $l$ and $b$ when targeting \smash{$\tau_{\FATE}^{(l, b)}$}. Qualitatively, the insight is that we should use $l \sim \log(T)$, and that “most” periods should be devoted to burn-in, i.e., $b = l - C$ for some constant $C$ that does not scale with $T$. In practice, however, $\log(T)$ may not be materially larger than relevant constants for reasonable values of $T$, and so the optimal choice of $b$ may be a non-trivial multiple of $l$ (e.g., $b = l/2$ may be reasonable). Furthermore, optimal choices of both $b$ and $l$ depend on $t_\mix$, which may be challenging to estimate from data; this mirrors the findings of xiong2022bias, which highlights the role of prior information in designing an efficient switchback experiment. In practice, our main recommendation is that analysts using switchbacks to estimate treatment effects in systems that may exhibit carryovers should consider using burn-in periods to mitigate carryover bias. Optimal choices of $l$ and $b$ are likely to depend on specifics of the application setting, and we recommend using a mix of experimental and semi-synthetic validation to pick $l$ and $b$ on a case-by-case basis.

We also provide a central limit theorem analogous to the one given in Theorem (ref) for the regular switchback estimator. The bias-corrected estimator has a more complex form, and so we need to require convergence of further moments in addition to Assumption (ref).

assuFor $i=1,\dots,k$, let $\bmu_i(w)$ denote average “pure treatment” potential outcomes for the focal period as in Assumption (ref), and let \begin{equation} \bmu^b_i(w) = \frac{1}{b}\sum_{t=1}^b\EE[\law_w^{(i-1)l+t}]{ Y_{(i-1)l+t}} \end{equation} denote the analogous quantity for the burn-in periods. We assume that the centered second moment of the block-averaged potential outcomes in the burn-in periods ($b$) and focal periods ($f$) converge as the number of blocks $k$ gets large, \begin{equation} \frac{1}{k} \sum_{i = 1}^k \begin{pmatrix} \bmu^b_i(w) - \frac{1}{k} \sum_{j = 1}^k \bmu^b_j(w) \\ \bmu_i(w) - \frac{1}{k} \sum_{j = 1}^k \bmu_j(w) \end{pmatrix}^{\otimes 2} \rightarrow \begin{pmatrix} V_{w}^b & V_{w}^{bf} \\ V_{w}^{bf} & V_{w}^{f} \end{pmatrix}, \end{equation} for both $w = 0, \, 1$. We also assume that the associated cross-terms converge in the same limit: \begin{equation} \frac{1}{k} \sum_{i = 1}^k \begin{pmatrix} \bmu^b_i(0) - \frac{1}{k} \sum_{j = 1}^k \bmu^b_j(0) \\ \bmu_i(0) - \frac{1}{k} \sum_{j = 1}^k \bmu_j(0) \end{pmatrix} \begin{pmatrix} \bmu^b_i(1) - \frac{1}{k} \sum_{j = 1}^k \bmu^b_j(1) \\ \bmu_i(1) - \frac{1}{k} \sum_{j = 1}^k \bmu_j(1) \end{pmatrix}' \rightarrow \begin{pmatrix} V_{01}^b & V_{01}^{bf} \\ V_{10}^{bf} & V_{01}^{f} \end{pmatrix}. \end{equation} Finally, we assume that all cross products of between the block-averaged potential outcomes in one block and those in the burn-in period of the subsequent block ($s$) also converge, \begin{equation} \frac{1}{k-1} \sum_{i = 1}^{k-1} \begin{pmatrix} \bmu^b_i(w) - \frac{1}{k} \sum_{j = 1}^k \bmu^b_j(w) \\ \bmu_i(w) - \frac{1}{k} \sum_{j = 1}^k \bmu_j(w) \end{pmatrix} \p{\bmu^b_{i+1}(w') - \frac{1}{k} \sum_{j = 1}^k \bmu^b_j(w')} \rightarrow \begin{pmatrix} V_{ww'}^{bs} \\ V_{ww'}^{fs} \end{pmatrix}, \end{equation} for all pairs $w, \, w' \in \cb{0,1}$.
theoUnder the assumptions of Theorem (ref), suppose in addition that $Y_t$ are upper bounded such that $\abs{Y_t}\le \Gamma_0$ for some $\Gamma_0<\infty$, and that Assumption (ref) holds. Then provided that we choose $b$ and $l$ so that $l\to\infty$, \begin{equation} \frac{k}{l^2} \exp\p{-2b/t_{mix}} \rightarrow 0, \end{equation} and there exists a constant $\beta\in [0,1]$ such that $b/l\to\beta$ as $k\to\infty$, we have \begin{equation} \sqrt{k} \p{\htau_{BC}^{(l,b)}-\tau_{\GATE}} \to_d \nn\p{0, \, \Tilde{V}} \end{equation} where \begin{equation} \begin{split} \Tilde{V} &= (1-\beta)^2\p{V^f_0+V^f_1+2V^f_{01}}\\ &\qquad\qquad +\beta^2 \p{3V^b_0+3V^b_1+2V^b_{01}+2V^{bs}_{01}+2V^{bs}_{10}+2V^{bs}_{00}+2V^{bs}_{11}}\\ &\qquad\qquad +2\beta(1-\beta) \p{{V}^{bf}_{0}+{V}^{bf}_{1}+{V}^{bf}_{01}+{V}^{bf}_{10}+{V}^{fs}_{00}+{V}^{fs}_{11}+{V}^{fs}_{01}+{V}^{fs}_{10} }. \end{split} \end{equation}

Although the form of the asymptotic variance in (ref) may appear complex at first glance, it is in fact the natural analogue of (ref) once the burn-in periods and their inverse-propensity reweighting are incorporated. As in (ref), the first term in (ref) corresponds to the variation arising from the focal-period difference-in-means component, scaled by the factor $(1-\beta)^2$ reflecting the proportion of focal periods. The second term plays a similar role for the burn-in correction, except that this part of the estimator is constructed using an inverse-propensity reweighting, which introduces additional cross-block covariance among consecutive burn-in averages. Finally, the remaining term captures the covariance between the focal and burn-in components, arising from the fact that the estimator combines information from both parts of the block in a single weighted statistic.

As with the simpler case in (ref), the asymptotic variance $\Tilde{V}$ in (ref) generally does not admit an unbiased estimator. In particular, several of the cross-terms in (ref) involve covariances that are unobservable under the realized design, and therefore cannot be estimated empirically. As before, conservative confidence intervals can be constructed by applying the Cauchy–Schwarz inequality to bound these unidentified components. Motivated by the jackknife procedure we develop for (ref) and block-resampling methods for correlated data kunsch1989jackknife, we outline a block jackknife procedure in Section (ref) of the supplemental materials. In our simulation studies, this block jackknife estimator performs well in practice, and we expect its good empirical behavior to hold more broadly, although a full theoretical characterization remains an interesting direction for future investigation.

Numerical Experiments

In this section, we show that the asymptotic rates derived in the previous sections approximate well the optimal rates that can be achieved by the estimators in estimating both the average global policy effect and the average filtered policy effect in simulations.

Simple Illustration

We start with a very simple setting in which all carryovers are mediated by a hidden state variable $H_t\in \{0,1,\dots,20 \}$ that evolves according to a random walk with drift. The evolution of $H_t$ is governed by the assigned treatment, as well as an exogenous market condition variable $M_t\in\{1,2,3\}$. We generate $M_t$ with the following dynamic process:

equation[equation omitted — 123 chars of source]

i.e., the market condition will switch with probability $0.5$, and once it switches, the new market condition is a uniform random draw from $\{1,2,3\}$. Conditionally on the sequence of market conditions, the state variable $H_t$ is generated as follows: If $W_t=1$,

equation[equation omitted — 171 chars of source]

if $W_t=0$,

equation[equation omitted — 171 chars of source]

Given $(H_t, W_t)$, $Y_t$ is then generated as

equation[equation omitted — 58 chars of source]

where $\epsilon_t\sim N(0,\sigma^2)$. Throughout, we fix $\sigma=3$, and vary the horizon $T$ from $400$ to $25,600$.

figure[figure omitted — 523 chars of source]

First, we evaluate the performance of $\htau_{\text{DM}}^{(l,b)}$ in estimating $\tau_\GATE$. Motivated by Corollary (ref), we fix $b$ to be always zero, and vary $l$ from $60$ to $480$. The left panel of Figure (ref) displays how MSE changes with different choices of $l$ and $T$ on a log scale across $5000$ iterations. In general, $\htau_{\text{DM}}^{(l,0)}$ with a larger $l$ achieves a quicker rate of convergence in estimating $\tau_\GATE$, while $\htau_{\text{DM}}^{(l,0)}$ with a smaller $l$ starts off with a smaller error. Consequently, as the horizon $T$ gets larger, the optimal $l$ also grows gradually from $l=60$ to $l=480$. Nevertheless, the mean-squared error achieved even with the best choices of $l$ will never surpass a $T^{-2/3}$ rate of convergence in large samples.

In the middle panel of Figure (ref), we plot the performance of \smash{$\htau_{\text{DM}}^{(l,b)}$} in estimating \smash{$\tau_{\FATE}^{(l, b)}$}. This is equivalent to the setting when the burn-in bias is negligible. Motivated by Corollary (ref), we vary $b$ from $10$ to $80$, and consider the set of $l$ such that $l=b+30$. We observe a similar pattern in the relationship between the horizon and the length of the burn-in periods, with the optimal $b$ growing gradually from $10$ to $20$ as $T$ increases. However, we can notice that, when there is no burn-in bias, we are able to achieve a much better error rate using the estimator using burn-in periods. This is especially important if the practitioners are planning for an experiment over a relatively long period of time. Furthermore, comparing the plot to the left panel, we notice that the performance is relatively robust to the choice of $b$. We also considered the set of $l$ such that $l=b+50$ and obtained similar results. Our results thus suggest that, in addition to improving asymptotic behavior relative to standard switchbacks, using burn-in periods (when there is no burn-in bias) also makes the performance of the experiment more stable across different choices of tuning parameters.

Next, we investigate the performance of \smash{$\htau_{\text{BC}}^{(l,b)}$} in estimating \smash{$\tau_{\GATE}$}, as represented in the right panel of Figure (ref). Once again, we vary $b$ from $10$ to $80$, and consider the set of $l$ such that $l=b+30$. The overall performance of \smash{$\htau_{\text{BC}}^{(l,b)}$} in estimating \smash{$\tau_{\GATE}$} closely mirrors that of \smash{$\htau_{\text{DM}}^{(l,b)}$} in estimating \smash{$\tau_{\FATE}^{(l, b)}$}, both achieving an error rate of \smash{$\log(T)\cdot T^{-1}$} in mean-squared error for estimating their targets. Overall, we find that the empirically optimal choice of $l$ grows very differently across the estimators. For the difference-in-means estimator without burn-in, the optimal $l$ increases rapidly with $T$: it grows from about $l^*=60$ when $T=400$ to $l^*=480$ when $T=6400$. In contrast, for the estimators with burn-ins, the optimal $l$ grows much more slowly: it increases from roughly $l^*=40$ when $T=400$ to only around $l^*=70$ when $T=6400$. This pattern aligns with our rate results in the preceding sections, where the estimator without burn-ins requires $l^*$ to grow on the order of $T^{1/3}$, while with burn-ins $l^*$ only needs to grow at a logarithmic rate.

{

table[table omitted — 587 chars of source]

}

Finally, we assess the coverage provided by confidence intervals associated with the three estimators: $\htau_{\text{DM}}^{(l,0)}$, $\htau_{\text{DM}}^{(l,b)}$, and $\htau_{\text{BC}}^{(l,b)}$. Throughout this analysis, we set $T=20,000$ and $l=200$, while varying $b$ from $50$ to $150$. To estimate the variance of the difference-in-means estimators, we employ a jackknife resampling procedure miller1974jackknife, which iteratively excludes each block and computes the variance of the estimator based on the remaining observations. For the bias-corrected estimators, we utilize a block-jackknife resampling procedure kunsch1989jackknife, which excludes two blocks simultaneously to account for the significant correlation between blocks introduced by weighting with the treatment assigned to the preceding block.\footnote{We experimented with excluding different numbers of blocks in the block-jackknife estimator, but did not find this to meaningfully affect performance here.} We present details of the jackknife variance estimators in Section (ref) of the supplemental materials.

The coverage of 95% confidence intervals around the three estimators can be found in Table (ref). We immediately see that the difference-in-means estimator without burn-in periods has zero coverage for $\tau_{\GATE}$. This anomaly arises because of the slow mixing of the Markov chain under study, leading to substantial bias; see Table (ref) in the supplementary material. In contrast, the two estimators employing a burn-in period exhibit significantly improved performance and provide reasonable coverage despite the presence of a substantial carryover effect. Moreover, these estimators also demonstrate robustness to variations in the length of the burn-in period. The bias-corrected estimator shows slight under-coverage; however, as seen in Table (ref) this is not due to bias, and instead this appears to be due to a finite-sample right-skew of the block-jackknife variance estimate here (see Figure (ref)). In Section (ref) of the supplemental materials we also consider an analogous-but-easier setting with faster mixing, and verify that all estimators do well in that setting.

A Ride-Sharing Simulator

We also evaluated our estimators using a large-scale ride-sharing simulator adapted from farias2022markovian. The simulator generates drivers and riders based on data from the NYC taxi trip records dataset tlcrecord. In this simulator, drivers enter the system continuously, each with a fixed capacity of 3 riders. Their initial positions are randomly selected from the trip records dataset, and the duration of their shifts follows an exponential distribution. Once a driver completes their shift, they go offline.

At any given time, a rider may initiate a ride request, with pick-up and drop-off locations randomly drawn from the trip records dataset and their value-for-time parameter (which informs offer acceptance) drawn from a lognormal distribution. When a request is initiated, the system dispatches a driver according to the current dispatching policy. Upon dispatch, an offer is made to the rider based on the expected cost of the dispatched driver's service. If the dispatched driver is a pool driver, an additional discount of 50% is applied to the offer. The rider then compares the offer with their outside option before deciding whether to accept. If accepted, the dispatched driver's route is updated accordingly.

As in farias2022markovian, we experiment on dispatching policies. Specifically, we examine a class of policies that determine whether to dispatch a pool driver based on the cost comparison with an idle driver. In these policies, a pool driver is dispatched only if the cost of the candidate pool driver is less than $\theta_d$ times the cost of the candidate idle driver, where $\theta_d$ is a threshold parameter determining the dispatching decision. We investigate two policies corresponding to different values of $\theta_d$: $\theta_d=0.5$ (treatment arm, $W_t=1$), indicating a more stringent dispatching criterion, and $\theta_d=1$ (control arm, $W_t=0$), representing a less stringent criterion. Upon completion of the ride, both the rider and the ride-sharing company incur costs and payments, respectively. The time horizon is discretized into 400-second intervals. The outcome of interest we study is the aggregated profit, which is defined as the total difference between the price charged to riders and the cost incurred by the ride-sharing company for completed requests. We refer the readers to Section (ref) of the supplemental material for more detailed information about the ride-sharing simulator.

The ride-sharing system can be naturally modeled as an MDP, with a large latent state space $S_t$ encompassing drivers' positions and routes, potential rider's locations, traffic conditions, riders' willingness-to-pay, outside options, and more. The outcome $Y_t$, which is the aggregated profit, is a function of both the current treatment $W_t$ and the unobserved state variable $S_t$. The dispatch policy can have a long-term effect by altering the state variable: If a change in the dispatching policy results in a different driver being dispatched to a request, it will impact this driver's route, subsequently affecting their future position and the requests they will be dispatched to. This carryover effect may persist indefinitely and may also influence nearby drivers.

figure[figure omitted — 678 chars of source]

We illustrate these carryovers in Figure (ref). To this end, we run three coupled simulators, one under the never-treat policy, one under an always-treat policy, and one with a switchback experiment. Each of the coupled simulators has the same random realizations of driver arrivals and rider requests, but then uses different dispatch policies. The figure displays aggregated profit over time from these 3 coupled simulators across a single switch from control to treatment. We see that the red solid curve representing the observed-in-switchback outcomes starts by approximating the never-treat trajectory before the switch; then, after the switch, it slowly diverges from it and eventually starts to approximate the always-treat trajectory. The coupling doesn't become perfect, though, since the effect of past dispatch decisions on the current system state never fully vanish on any reasonable timescale.

figure[figure omitted — 455 chars of source]

To choose the block length for the switchback experiment, we examine the distribution of the rides’ expected durations using a historical always-control dataset containing approximately 400,000 seconds of operations. As shown in the left panel of Figure (ref), the majority of rides are completed within 4,000–6,000 seconds. This suggests that a block length of roughly 4,000 seconds should be sufficient to ensure that the carryover effect from the previous treatment assignment has largely dissipated. In Section (ref) of the supplementary material, sensitivity analyses shows that experiments using a shorter block length of 2,000 seconds perform substantially worse, while a longer block length of 6,000 seconds performs only slightly better and at a comparable overall level. In practice, we recommend selecting the block length conservatively based on domain knowledge (e.g., the typical ride duration) so that any cross-block carryover effects are essentially negligible.

With a block length of 4,000 seconds ($l=40$), we run the experiments for a total duration of 400,000 seconds ($k = 100$). We examine three estimators: the difference-in-means estimator $\hat{\tau}_{\text{DM}}^{(l,0)}$, the difference-in-means estimator with burn-in periods $\hat{\tau}_{\text{DM}}^{(l,b)}$, and the bias-corrected estimator $\hat{\tau}_{\text{BC}}^{(l,b)}$. To guide the choice of the burn-in period length $b$, we inspect how $\hat{\tau}_{\text{DM}}^{(l,b)}$ and $\hat{\tau}_{\text{BC}}^{(l,b)}$ change as $b$ increases using a single experimentation trajectory sample. From the right panel of Figure (ref), both estimators increase together for small $b$, which is likely due to the common reduction in burn-in bias. The estimators stabilize around $b\approx 8$. Beyond this point, however, they begin to behave very differently as $b$ continues to increase, which is likely due to the dominance of variance. This is unsurprising, as the outcomes themselves are highly noisy (Figure (ref)). These observations motivate choosing a burn-in period of roughly $b=8$. In practice, this inspection can be performed using a previous experiment with treatments that induce similar mixing behavior, while applying it to the same trajectory used for analysis could invalidate inference and should therefore be avoided when confidence statements are desired.

{

table[table omitted — 750 chars of source]

}

Table (ref) presents the bias, variance, mean squared error, and coverage achieved by those estimators with two different lengths of burn-in periods: $b=8$ and $b=16$ respectively. As expected, we notice that the use of burn-in periods considerably reduces the bias of the treatment effect estimation. Although the variance of the estimators with burn-in periods increases, the decrease in bias still results in a large decrease in the overall mean squared error. Furthermore, including burn-in periods leads to more accurate inferential results, with confidence intervals having coverage close to the nominal level. Again, we observe that the performance of the estimators with burn-in periods remains relatively stable regardless of the chosen length of the burn-in period. In Section (ref) of the supplementary material, we report additional results examining the performance of the estimators under a wider range of $(l,b)$ choices, as well as a comparison with a Horvitz–Thompson–type estimator in the spirit of bojinov2020design. Overall, the choice suggested by the heuristics in Figure (ref) are never much worse than any of the alternatives we tried, and the results are fairly stable across all reasonable choices of $(l,b)$. If the practitioner has access to a simulator similar to ours, another option for selecting $(l,b)$ in practice is to replicate this type of simulation study and simply choose the design that performs best in the simulated environment.

Discussion

We studied switchback experiments under a generic, time-heterogeneous Markovian model for carryover effects. We found that, under this model, regular switchback estimators as they are often implemented in practice---that is, without any burn-in periods---suffer from a severe bias problem that limits the best attainable rate of convergence. We also showed that there exist a number of practical solutions to this bias problem. If researchers are willing to change their statistical target to a filtered average treatment effect, then the bias problem can be side-stepped by using burn-in periods with a regular switchback estimator. Meanwhile, researchers who want to target the global average treatment effect can achieve good performance by using a modified, bias-corrected estimator to process the data generated from a Bernoulli switchback design.

More broadly, our results suggest promise in using dynamic stochastic modeling techniques to understand and improve popular tools for causal inference. The switchback is a simple and intuitive experimental design that can be used without explicitly writing down a stochastic model. However, we found that modeling the underlying system as a Markov decision process enabled us to get new insights on how and why switchback experiments enable accurate causal inference---and these insights can then be put into practice without requiring researchers to make large changes to their analytic approach (and, in particular, without requiring researchers to fit a Markov decision process). It is likely that we could also improve our understanding of other (at face value model-free) causal estimators via a stochastic modeling approach.

\ifms

APPENDIX{Supplemental Materials} \else

Funding

This research was supported by NSF grant SES-2242876.