Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
134,376 characters · 14 sections · 64 citation commands
Double Machine Learning for Causal Inference under Shared-State Interference
\ificml \twocolumn[ \icmltitle{{Double Machine Learning for Causal Inference under Shared-State Interference}}
\icmlaffiliation{mit}{Institute for Data, Systems and Society, MIT} \icmlaffiliation{sloan}{Sloan School of Management and Department of Electrical Engineering and Computer Science, MIT}
\icmlcorrespondingauthor{Chris Hays}{[email removed]}
\icmlkeywords{Machine Learning, ICML}
\vskip 0.3in ] \printAffiliationsAndNotice \else \fi
In causal inference, interference --- where individuals' treatment assignments, outcomes or other characteristics impact others' outcomes --- is everywhere: On short-form video platforms, the consumption of one individual is used as an input to recommendation algorithms that are subsequently used to serve content to others. In ride-sharing or housing rental services, providing a discount to some people can cause them to increase their usage of the service and increase wait times or prices. In such settings, failure to account for interference may lead to biased estimates of causal effects, even in randomized controlled trials.
Across many settings, like those of recommender systems and markets, interference between units often follow a common pattern: the outcomes of individuals depend on others through some shared state. In recommender systems, the shared state might be the outputs of recommender systems that are used to generate users' feeds. In marketplaces, the shared state may be prices, wait times, or availability of inventory. In each of these settings, individuals both influence and are influenced by the shared state.
Assumptions are required to perform causal inference in the presence of interference. Prior work has studied this kind of interference in particular applications johari_experimental_2021,basse_randomization_2016,wager_causal_2024,munro_causal_2024,simchi-levi_pricing_2023,dhaouadi_price_2023,farias_correcting_2023,farias_markovian_2022,li_experimenting_2024. In some cases, they propose either experimental designs or analysis methods that account for interference and allow for valid inference. However, these approaches typically assume a parametric model of the system or domain-specific assumptions.
\paragraph{Our contributions.} We provide a general framework to reason about causal inference under shared-state interference. We define a formal model in (ref), where individuals arrive sequentially. Their covariates (i.e., unit characteristics), treatment assignments, and outcomes may influence the outcomes of future individuals through some shared state.
In (ref), we provide conditions under which efficient inference is possible in the framework. We extend methods from double machine learning (DML) chernozhukov_doubledebiased_2018, which has shown that it is possible to achieve efficient inference using expressive machine learning methods without parametric functional form assumptions. A key assumption we rely on is that the shared state progresses according to a Markov chain with particular properties. We also provide a consistent variance estimator, which is necessary for constructing valid confidence intervals and running hypothesis tests.
Next, we instantiate our framework with two applications: first, we show how our framework can be used to estimate average direct effects (ADE) in observational settings; then, we provide a variance reduction strategy for estimation of the global average treatment effects (GATE) in switchback experiments. Intuitively, the ADE measures the expected difference between treatment and control for a unit drawn uniformly at random, keeping the treatment assignment distributions of other units fixed. The GATE measures the mean difference between outcomes when all units are assigned to treatment versus when all are assigned to control. Key to our approach in these sections is constructing structural models that capture settings of interest and estimators for the treatment effects which satisfy the properties necessary to apply the DML theorem from (ref). In each instantiation, we provide simulations validating that our method produces estimators that concentrate around the true treatment effects in finite samples. We also show our consistent variance estimators can be used to construct confidence intervals with the desired coverage probability.
Our work sits at the intersection of the study of inference under interference and double machine learning (DML) methods.
\paragraph{Shared-state interference.} Several prior works have explored causal inference in settings captured by or similar to our formulation of shared-state interference johari_experimental_2021,basse_randomization_2016,wager_causal_2024,munro_causal_2024,holtz_reducing_2024,simchi-levi_pricing_2023,dhaouadi_price_2023,farias_correcting_2023,farias_markovian_2022,li_experimenting_2024,brennan_reducing_2024,bajari_experimental_2023,bright_reducing_2023. Some of these papers have analyzed bias resulting from naive estimators or experimental designs and proposed less biased alternatives johari_experimental_2021, farias_markovian_2022,brennan_reducing_2024. Other works have proposed parametric models of settings that allow for inference in the presence of interference wager_experimenting_2021, li_experimenting_2024. Several works consider a Markov model of interference similar to ours in an experimental setting and explore less biased estimation methods farias_correcting_2023,farias_markovian_2022,glynn_adaptive_2022.
\paragraph{Double machine learning.} The double machine learning framework was introduced in chernozhukov_doubledebiased_2018, drawing on a rich literature in semiparametric statistics (see, e.g., kennedy_semiparametric_2023 for an overview). DML methods have been extended to several other settings, such as those with continuous treatments kennedy_non-parametric_2017, estimation of quantile effects kallus_localized_2022 and settings with limited unobserved confounding rambachan_robust_2024. A few works, like ours, have developed semiparametric methods for settings with interference. Several works have explored interference channeled through individuals' social networks emmenegger_treatment_2023,ogburn_causal_2024. Our setting is different because each unit may be dependent on each of the units before it (whereas their network-based approaches require the network to be sparse), and interference in our setting is anonymous (i.e., independent of the identities of the units). munro_causal_2024 develops a DML theorem for settings like auctions where interference is channeled through a centralized allocation mechanism. zhan_estimating_2024 develops a DML theorem for a discrete choice model of content consumption under a neural network-based recommender system. In contrast, our work relaxes these assumptions and instead requires a general Markovian assumption. ballinari_semiparametric_2024 develops a DML theorem for a time-series model where a single unit is observed over time, but where observations across time obey a mixing condition. Their method allows for estimating impulse response functions, which measure the effect of an intervention at a particular time on the outcome at a future time. Our work also relies on mixing conditions for our convergence results, but our model assumes all dependence between observations is channeled through the shared-state, and our model instantiations and estimands in (ref) are different from theirs. We supply an extended comparison to some key related works in (ref).
We first give a high-level description of our setting. Our model begins with a set of sequentially arriving units. Each unit has covariates and a binary treatment assignment drawn iid from a joint distribution, as in canonical causal inference settings. There is also a (possibly vector-valued) observed shared state through which all interference is channeled: each arriving unit has an outcome of interest that may depend on the shared state, their covariates and their treatment assignment; then the shared state at the next time step may depend on the previous shared state, as well as the previous unit's outcome, covariates and treatment assignment. The key assumption that makes our setup tractable is that the shared state has a Markov property: that is, conditional on the shared state at the previous time step and the data of the unit that arrived in the previous time step, the shared state is independent of the previous data. We next introduce the model and notation formally.
\paragraph{Notation.} Capital letters will denote random variables. For a vector ${x}$ and indices $i,j$, ${x}_{i:j}$ indicates the column vector of entries $(x_{i}, \dots, x_{j})'$ where $'$ indicates the transpose. For a constant $c$, the vector $(c, c, \dots, c)'$ (of length induced by context) will be denoted using bold font $\mathbf{c}$. For a scalar random variable $V \sim P$, we will denote the $L^q(P)$ norm $\left\| {V} \right\|_{L^q(P)} = \mathbb{E}_P[{\left| V \right|^q}]^{1/q}$. For two functions $f$ and $g$ taking the same argument $x$, we may write $(f+g)(x) = f(x) + g(x)$ to denote their sum.
\paragraph{Model.} We will consider a sequence of units indexed $t=1,2,\dots,$ and observed up to time $T$. Each unit $t$ will have features $X_t$ drawn iid from a sample space $\mathbb{R}^{p_X}$ for finite dimension $p_X$ and a binary treatment assignment $D_t$. We will let $H_t$ on $\mathbb{R}^{p_H}$, for finite dimension $p_H$, denote the shared state. The observed outcome for each unit will be real-valued and denoted $Y_t$.
We will adopt notation so that potential outcomes are a (stochastic) function of treatment and the shared-state (i.e., $Y_t(D_t, H_t)$) similar to the exposure mapping approach of aronow_estimating_2017.\footnote{Exposure mappings are typically defined to be functions of the set of treatment assignments, but in our setting, as we will discuss, $H_t$ may depend on other units' (stochastic) outcomes.} The dependence of outcomes on the shared state is the primary difference between this model and canonical iid causal inference settings, and the avenue through which interference is assumed to occur. We will assume $D_t$ is independent of all data from prior units but that it may depend on $X_t$. The observed data associated with time step $t$ will be denoted $W_t = \{ X_t, D_t, H_t, Y_t \}$ on the sample space $\mathcal{W} = \mathbb{R}^{p_X } \times \{ 0, 1\} \times \mathbb{R}^{ p_H} \times \mathbb{R}$. We will denote the full observed data $W_{1:T} = \{ W_t \}_{t=1}^T \in \mathcal{W}^T$ and the (unknown, joint) data distribution of $W_{1:\infty} = \{W_t\}_{t=1}^\infty$ as $P \in \mathcal{P}$ for some set of distributions $\mathcal{P}$. The set $\mathcal{P}$ specifies the set of probability distributions in which the true distribution may fall, and we will impose assumptions that must hold for all $P \in \mathcal{P}$.
We will assume that $H_t$ depends on the data from the previous time step (i.e., $W_{t-1}$) but is independent of $X_t$, and is conditionally independent of the data at time steps before time $t-1$, i.e., that $H_t$ has the Markov property: For all $P \in \mathcal{P}$ and $P$-measurable $A \subseteq \mathbb{R}^{p_H}$, and $t = 1, 2, \dots$
This is assumption satisfied in settings where $H_t$ is an update to $H_{t-1}$, like when inventory at time $t$ is a function of inventory at time $t-1$ and whether the customer at time $t-1$ made a purchase.
Our setting allows for the shared state at time $t$ to depend on the shared state at time $t-1$, allowing for dependencies over time, as long as these dependencies are exclusively mediated by the shared state. For example, the purchasing decisions of two individuals may be correlated if inventory is low during the time interval in which they arrive. The shared-state at time $t$ can also depend on the outcome at time $t-1$. This is different from many other models for causal inference under interference, which allow for dependencies only on other units' treatment assignments. For example, the canonical neighborhood treatment response assumption eckles_design_2014, informally, says that, for a particular network, a node's potential outcomes depend only on its treatment assignment and its neighbors' treatment assignments. Thus, our method has the desirable property that individuals' outcomes can depend on prior individuals' behavior, not just prior treatment assignments. The dependency structure assumed by a shared-state interference setting is summarized in (ref).
We will also assume that the distribution of $H_t$ and $D_t$ conditional on $W_{t-1}$ is invariant of $t$. Thus, we can specify a transition probability kernel $K_H$ where, for $w \in \mathcal{W}$ and measurable event $A$ on $\mathcal{W}$ and all $t \in [T]$,
There will be an initial (known) shared-state $H_0$ generated according to an arbitrary, unknown and possibly deterministic distribution $P_{0}$. The $t$-step kernel will be denoted $K_H^t$, where $K_H^t(w, A) = P(W_{s+t} \in A \; | \; W_{s} = w)$.
The fact that $H_t$ has the Markov property, $X_t$ is drawn iid, and $D_t$ is independent of the prior data trivially implies that $\{ W_t\}_{t=1}^\infty$ itself is a Markov chain. Indeed, since all transition probabilities are $t$-invariant, $\{ W_t\}_{t=1}^\infty$ is homogeneous. $\mathcal{W}$ may be uncountable, so the chain may be defined on a general (not necessarily finite) state space. We will denote the (unknown) transition probability kernel of $W_{1:\infty}$ as $K$ where $K(w, A) = P\left( {W_t \in A} \;\middle|\; {W_{t-1} = w} \right)$. Note that $K$ is distinct from $K_H$; the former denotes the transition probabilities for $W_t$ and the latter only for $H_t$. As with $K_H$, the $t$-step kernel will be denoted $K^t$.
When the Markov chain has a unique stationary distribution, we will denote it $K^\infty$, since we will only be considering settings where, for all $w, A$ it holds $K^\infty(A) = \lim_{T \to \infty} K^T(w, A)$. (We drop the argument $w$ in $K^\infty$ since $K^\infty$ does not depend on the starting state.) When $K^\infty$ exists, we will require $K^\infty \in \mathcal{P}$.
We will also require that $\{ W_t \}_{t=1}^\infty$ satisfies natural Markov chain conditions.
Geometric ergodicity is a generalization (to the general state space setting) of aperiodicity and irreducibility in finite-state Markov chains; we define it formally in (ref), and refer the reader to standard references like meyn_markov_2009 for a complete treatment of general state space Markov chains. (ref)(ref) implies that the chain has a unique steady state transition probability distribution and that the chain converges to that distribution from any starting state meyn_markov_2009. Thus, writing $K^\infty$ in (ref) is well-defined. Intuitively, detailed balance says that, for continuous distributions, the steady state probability density around a point $w$ times the probability density of transitioning from $w$ to $z$ is equal to the same expression with the roles of $w,z$ reversed.
The $m$-dependence condition states that data observed more than $m$ steps apart are independent. Observe that any $m$-dependent sequence can be written as a Markov chain, using the last $m$ observations as the state. We will assume throughout when invoking (ref)(ref) that (a finite upper bound on) $m$ is known. We refer the reader to the literature on switchback experiments for methods of estimating $m$ from data bojinov_design_2022.
\paragraph{Estimand.} We will assume there is a scalar-valued functional of interest $\psi^*$ which depends on the data distribution $P$; e.g., in (ref), $\psi^*$ will be the average direct effect and in (ref), it will be the global average treatment effect. We will assume there exists a function $\varphi^* \; : \; \mathcal{W} \to \mathbb{R}$ so that $\psi^*$ can be written as an asymptotic average over expected values of $\varphi^*$. I.e.,
We will assume that $\varphi^*$ is continuous.
\paragraph{Examples of shared-state interference.}
Our model is suited to describing marketplaces where interference is induced by limited supply or demand. For example, johari_experimental_2021 proposes a Markov chain model of a rental platform consisting of a sequence of arriving customers and a set of listing types, each with a fixed number of available listings. If a customer books a listing, it becomes unavailable for a time, temporarily decreasing the number of listings of the type by 1. Several other models of interference in specific markets can be well-captured by shared-state interference: for example, models of freelance labor markets wager_experimenting_2021 and ride-sharing markets li_experimenting_2024. Moreover, $m$-dependence is assumed in the large literature on switchback experiments bojinov_design_2022.
In this section, our main result is a double machine learning (DML) theorem for causal inference under shared state interference (SSI). Double machine learning is a meta-algorithm which allows for efficient inference with the use of expressive machine learning methods. It was proposed in chernozhukov_doubledebiased_2018, building on a large literature in semiparametric statistics. We provide background on the method in (ref) and give an informal overview here. We refer the reader to chernozhukov_doubledebiased_2018 for a full treatment of the topic in the iid setting and to chernozhukov_applied_2024 for a gentle introduction.
At a high level, the method consists of two steps. First, an expressive machine learning algorithm is used to approximate any nuisance parameters. Nuisance parameters often take the form of estimated conditional expectation functions (the expected outcome conditional on covariates and treatment) or propensity scores (probability of treatment conditional on covariates) which are necessary for estimation of treatment effects but not of interest on their own. A predictor for the expected outcome conditional on covariates could be used to create plug-in estimators by simply taking the average of predicted outcomes under treatment versus under control. However, such plug-in estimators may converge at slower-than-parametric rates and may be very biased in finite samples. Thus, second, the estimator is constructed so as to satisfy a first-order insensitivity property, in a distributional sense that we will precisely specify later. This property can be used to show that the convergence rates of the estimator depend only on products of convergence rates of the ML estimators, which allows for weaker conditions on the convergence rates for any one estimator.
We note that a naive approach to inference in our setting would be to treat the shared-state as a covariate and apply DML methods as if the data were iid. However, depending on the estimand, this may yield inconsistent treatment effect or variance estimators, as we will see in our simulations in (ref).
An important part of any DML procedure is that the nuisance parameter estimators must be independent of the data used to construct the target estimator. (Otherwise, typical $L^2$ convergence rates are not sufficient to ensure that the nuisance estimators yield consistent target estimators.) In iid settings, this independence is usually achieved via sample splitting: the data is split into $k$ folds, the ML estimator is trained on all but one fold and then the estimator is constructed using the held-out fold. In our setting, in the vein of angelopoulos_prediction-powered_2023, ballinari_semiparametric_2024 and many similar methods in statistics and machine learning, we instead assume that the nuisance estimators are generated via an auxiliary sample of data that is independent of the data to be used for inference. We make this simple alternate assumption because sample splitting would not guarantee independence between folds of the data in our setting, so predictors trained on some folds may still not be independent of the others.
If the data is $m$-dependent, then sample splitting can be used to generate a predictor that is independent of the data used to construct the estimator. Additionally, if the structural equation models assumed about the data generating process are Donsker,\footnote{Donsker classes, intuitively, are function classes that are simple enough that they cannot arbitrarily overfit on training data. Vapnik-Chervonenkis (VC) classes and bounded monotone function classes, among many others, are Donsker. See kennedy_semiparametric_2016 for detailed discussion of this approach.} no auxiliary sample is necessary. Finally, we note that in our interference setting, units at far apart time steps --- even without $m$-dependence --- have approximately uncorrelated outcomes, so it may be possible to split the data into approximately uncorrelated subsamples. We leave for future work the exploration of whether predictors trained on approximately uncorrelated data splits can be used for valid inference.
For our theorem in this section, we will assume that there is a known estimator $\psi$ that depends on $W_{1:T}$ and a set of data-dependent nuisance parameters/functions $\eta \in \mathcal{S}$ for some convex set $\mathcal{S}$. We can imagine $\eta$ to be the parameters in a neural network, the coefficients on a possibly high-dimensional linear regression, or the feature splits and thresholds for trees in a random forest. We will need that $\psi$ identifies $\psi^*$; i.e., that $\mathbb{E}_{P}\left[ {\psi(W_{1:T}; \eta^*)} \right] = \psi^*(P)$ for unknown true nuissances $\eta^* \in \mathcal{S}$. We will also require that $\psi$ is an empirical mean of a time-$t$ estimator $\varphi$, i.e., that
In (ref), $\psi$ will be an estimator for the average direct effect (ADE) and global average treatment effect (GATE), respectively, which we define formally under the structural models in each of those sections. The estimator will be specially constructed so as to satisfy the double-robustness property characteristic of DML methods. Much of the challenge in applying this section's main theorem involves specifying a structural model that fits a context of interest and identifying an estimator with the double-robustness properties.
We will assume access to a nuisance function estimator $\hat \eta$ (usually imagined to have been trained by a flexible machine learning method like a neural network or random forest). We will also require that $\hat \eta$ belongs to a convex set containing $\eta^*$, with high probability. This is for technical reasons that will become clear later and are common to all DML methods: the conditions required for our main theorem necessitate reasoning over convex combinations of $\hat \eta$ and $\eta^*$ so it is useful to define a convex set within which $\hat \eta$ and $\eta^*$ fall. Finally, we will require that the convex set is shrinking around $\eta^*$ asymptotically in $T$. This ensures that $\hat \eta$ is converging to $\eta^*$ (in an appropriate sense) as $T$ scales. Formally, for a constant $0 \leq \gamma < 1$, we will require that the nuisance function estimator $\hat \eta$ belongs to a convex set $S_T \subseteq \mathcal{S}$ with probability $1 - \gamma$ where it is also assumed $\eta^* \in S_T$. The DML procedure for our context is summarized in (ref).
We will need several assumptions about the data generating process and nuisance function estimation. These assumptions are analogous to those necessary for the results in chernozhukov_doubledebiased_2018. The first assumption, (ref), requires that $\psi$ is a smooth function of $\eta$, is Neyman orthogonal and satisfies regularity conditions. We state it formally next.
(ref)(ref) requires that second Gateaux derivatives of $\psi$ with respect to $\eta \in S_t$ exist and are continuous. The Gateaux derivative is defined in (ref) and, informally, is equivalent to the usual derivative with respect to $r \in (0,1)$ for $\psi(W_{1:T}; r \eta^* + (1-r) \eta) $ for $\eta \in S_T$. (ref)(ref) requires Neyman orthogonality, which is defined in (ref) and, informally, is the requirement that the Gateaux derivative above, evaluated at $r=0$, is zero. {In applications, using our theorem in this section will require carefully designing estimators so that these conditions hold.}
The second assumption, (ref), requires that the nuisance estimators obey regularity conditions and converge at appropriate rates, for all $\eta \in S_T$.
(ref) requires that the $L^q$ norm of $\varphi(W_t, \eta)$ must be bounded for all $t \in [T]$ for $q > 4$. (ref) requires the average over $t$ of the $L^2$ norm of $\varphi(W_t; \eta)-\varphi(W_t, \eta^*)$ must converge to zero as $T \to \infty$. Finally, (ref) requires the second Gateaux derivative of $\psi$ with respect to $\eta$ must go to zero at faster-than-$\sqrt{T}$ rates. These are analogous to standard regularity and convergence rate conditions for DML theorems chernozhukov_doubledebiased_2018.
We are now ready to state our theorem, which says that the assumptions above are sufficient for efficient inference and consistent variance estimation under shared-state interference.
We briefly comment on the fact that our variance estimators under (ref)(ref) and under (ref)(ref) are different. Under (ref)(ref), our consistent variance estimator is not a plug-in estimator. In fact, plug-in variance estimators in Markov chains are not in general consistent jones_fixed-width_2006. On the other hand, for (ref)(ref), the plug-in variance estimator is consistent.
The proof of our (ref) follows the pattern in that of Theorem 3.1 of chernozhukov_doubledebiased_2018. We have simplified some of their setting for the sake of space and clarity, but our analysis in this paper could be extended to accommodate the additional generality in their theorems.
To prove the result, we must account for covariance between different terms $\varphi(W_t)$ in our analysis; chernozhukov_doubledebiased_2018 do not have to handle any covariances since each observation is independent. To control covariance, we appeal to either the combination of geometric ergodicity and detailed balance or $m$-dependence. In either of these cases, correlation between terms goes to zero sufficiently fast that these terms do not dominate. After these covariances are handled, we may replace the iid central limit theorem used in chernozhukov_doubledebiased_2018 with an appropriate Markov chain central limit theorem to complete the result. To prove consistency of the variance estimates, under (ref)(ref) we apply a consistent variance estimation theorem from the Markov chain literature. Under (ref)(ref), we show that the plug-in variance estimator is consistent by showing the plug-in estimates of each variance and covariance term for $\varphi(W_t; \eta)$ are themselves consistent.
\paragraph{Applying the theorem.} At a high level, instantiating (ref) in specific settings is a matter of verifying (ref). Once these are satisfied, (ref) tells us that the DML procedure in (ref) can be used to generate asymptotically normal treatment effect estimators using the provided variance estimators.
In the next two sections, we apply (ref) for two structural models of independent interest. In each section, we examine a specific semiparametric structural model of outcomes given covariates, treatment and the shared state and a specific causal estimand. In each case, verifying (ref) amounts to ensuring all interference is channeled through a shared state, making appropriate regularity assumptions and proving the following three lemmas: first, that the estimator is Neyman orthogonal with respect to the nuisance estimators; second, that the average over $t$ of the $L^2$ norm of $\varphi(W_t; \eta)-\varphi(W_t, \eta^*)$ must converge to zero as $T \to \infty$; third, that the second Gateaux derivative of the estimator has order smaller than $\sqrt{T}$. The regularity assumptions amount to support conditions requiring a lower bound on the probabilities of each treatment assignment and requirements that nuisance estimates and parameters are bounded in probability.
\ifproofsinbody Proof (ref). Observe:
from applying the triangle inequality twice. Then applying (ref) and (ref) implies
Applying (ref), we observe
which, with, e.g., van_der_vaart_asymptotic_2000 Theorem 2.7(iv), implies the first statement in the result. Lastly, (ref) proves $\hat \sigma^2_T \overset{p}{\to} \sigma^2,$ and applying Slutsky's theorem completes the proof. \qed
In the proof of (ref), we will use the following two central limit theorems. The first is for geometrically ergodic Markov chains on general state spaces, and the second is for $m$-dependent sequences. (ref) generalizes standard central limit theorems for Markov chains in finite state spaces, where the analogous requirements are that the chain is irreducible and aperiodic. See (ref) for further description of geometrically ergodic Markov chains. (ref) states a central limit theorem for $m$-dependent sequences.
(ref) says that the oracle estimator approaches the estimand at $\sqrt{T}$ rates and is asymptotically normal. The proof of (ref) applies (ref). To apply the CLT, we just need to verify that we can apply one of the two central limit theorems above: (ref) for (ref)(ref) and (ref) for (ref).
Proof of (ref). From (ref), under part (ref), we have that $\{ W_t \}$ is a geometrically ergodic Markov chain. Also, $\varphi(W_t; \eta^*)$ is bounded in $L^{2+\delta}(P)$ by (ref), (ref). So $\psi(W_{1:T}; \eta^*) - \psi^*$ is bounded in probability by the triangle inequality:
Thus, the result under (ref)(ref) follows directly from the application of (ref).
For (ref)(ref), since $\{ W_t \}_{t=1}^\infty$ is $m$-dependent, so is $\{ \psi(W_t; \hat \eta) \}_{t=1}^\infty$. By (ref), (ref), we have that $\varphi(W_{t}; \eta^*)$ is bounded in $L^3$. Thus, we can apply (ref).
\qed
(ref) says that the deviation from its mean of the difference between the oracle estimator and our estimator approaches zero at faster-than-$\sqrt{T}$ rates. Our proof pattern is similar to that of chernozhukov_doubledebiased_2018: our strategy is to prove that the variance of the expression goes to zero at faster-than-$T$ rates, so that we can apply Chebyshev's inequality, which implies that the expression itself goes to zero at faster-than-$\sqrt{T}$ rates. The core difficulty of proving the lemma, compared to the analogous result in chernozhukov_doubledebiased_2018, is that we must account for the covariance between observations at different times. chernozhukov_doubledebiased_2018 assumes independence between observations and thus does not have to account for such covariances. We handle the covariances separately for (ref)(ref) and for (ref)(ref). Under (ref)(ref), we can invoke a theorem saying that geometrically ergodic Markov chains satisfying detailed balance have correlations between observations that decrease to zero at an exponential rate. This implies that the sum of correlations is bounded by a constant, so they do not dominate. Under (ref)(ref), there are only finitely many non-zero correlations, so their sum is also bounded by a constant.
Proof of (ref). First, we will introduce new notation. For all $t \in [T]$, let
Let $\mathcal{E}_T$ be the event that $\hat \eta \in S_T$ (which happens with probability $1 - \gamma$ by (ref)). Also, $\hat\eta$ is assumed to be a deterministic function of $W^{\mathrm{aux.}}$. On the event $\mathcal{E}_T$,
where the last equality comes from the {independence of $W^{\mathrm{aux.}}$ from $W_{1:T}$.} Now, for all $t \in \mathbb{N}$, let $\mathcal{F}_{t}^\infty$ be the $\sigma$-algebra generated by $ W_{t:\infty}$ and let $\mathcal{F}_1^t$ be the $\sigma$-algebra generated by $W_{1:t}$. Recall the definition of $\rho$-mixing (see, e.g., bradley_chapter_2007):
where the correlation, covariance and variance for functions has the usual definition:
and $\mathrm{Var}(f) = \mathrm{Cov}(f,f)$. Next, we state a lemma that will allow us to bound the covariance between terms in the above sum under (ref)(ref).
On the other hand, under (ref)(ref), $W_{1:\infty}$ is $\rho$ mixing with $\rho(\ell) = 0$ for all $\ell > m$. Note that these facts imply $W_{1:\infty}$ is $\rho$-mixing with $\rho(t) = O(e^{-c T})$ for some $c > 0$. Also, for all $t \in [T]$, $\eta \in S_T$ and $\ell < t$, it holds ${\varphi(W_{t}; \eta) - {{\varphi_t^*} - \mathbb{E}_{P}[\varphi(W_{t}; \eta) - {\varphi_t^*} ]}} \in L^2(\mathcal{F}_{t}^\infty)$ and ${\varphi(W_{t-\ell}; \eta) - { \varphi_{t-\ell}^* - \mathbb{E}_{P}[\varphi(W_{t-\ell}; \eta) - \varphi_{t-\ell}^*]}} \in L^2(\mathcal{F}_{1}^{t-\ell})$ (where expectations are taken with respect to $P$). To see this, note that for all $t \in [T]$ and $\eta \in S_T$, it holds $\left\| {{\varphi(W_{t}; \eta)}} \right\|_{L^2(P)} < \infty$ by (ref), (ref). Also, since the expressions just depend on $W_t$ and $W_{t-\ell}$, respectively, this implies $\left\| {{\varphi(W_{t}; \eta)}} \right\|_{L^2(P)} = \left\| {{\varphi(W_{t}; \eta)}} \right\|_{L^2(\mathcal{F}_{t}^\infty)}$ and similarly $\left\| {{\varphi(W_{t-\ell}; \eta)}} \right\|_{L^2(P)} = \left\| {{\varphi(W_{t-\ell}; \eta)}} \right\|_{L^2(\mathcal{F}_1^{t-\ell})}$. Also, it is assumed $\eta^* \in S_T$, so by the same reasoning $\left\| {{\varphi^*_t}} \right\|_{L^2(\mathcal{F}_{t}^\infty)} < \infty$ and $\left\| {{\varphi^*_{t-\ell}}} \right\|_{L^2(\mathcal{F}_1^{t-\ell})} < \infty$.
Thus,
This implies we can rewrite (ref) as
Now, notice $$T^{-1} \sum_{t=1}^T \left\| {{\varphi(W_{t}; \eta) - {{\varphi_t^*} }}} \right\|^2_{L^2(P)} = o(1)$$ by (ref), (ref), so $$T^{-2} \sum_{t=1}^T \left\| {{\varphi(W_{t}; \eta) - {{\varphi_t^*} }}} \right\|^2_{L^2(P)} = o(T^{-1}).$$ Next, since $\rho(\ell) = O(e^{-c \ell})$ for some $c > 0$ (by (ref)) and $$T^{-1} \sum_{t=1}^T \left\| {{\varphi(W_{t}; \eta) - {{\varphi_t^*} }}} \right\|_{L^2(P)}= o(1)$$ for all $t \in [T]$, it holds
where the last line comes from the fact that {$\sum_{t=1}^T e^{-\theta t} \leq 1/(1- e^{-\theta})$}. Thus, applying Chebyshev's inequality, we have
Finally, the fact that conditional convergence implies unconditional convergence (Lemma 6.1 of chernozhukov_doubledebiased_2018) implies the desired result. \qed
(ref) says that the absolute value of the expectation of the difference between the estimator and oracle goes to zero at faster-than-$\sqrt{T}$ rates. The proof is nearly identical to the analogous argument in chernozhukov_doubledebiased_2018: we apply Taylor's theorem by differentiating on the path between $\hat \eta$ and $\eta^*$ to show that small deviations of nuisances $\eta$ around $\eta^*$ do not dramatically affect the deviation of the estimator from the oracle estimator. The first-order term is zero by the assumption of Neyman orthogonality. The second-order term is small by assumption ((ref)).
Proof of (ref). We use Neyman orthogonality. Define
Then, notice
is the quantity we want to bound in probability. By Taylor's Theorem
for some $\tilde r \in (0,1)$. Notice that $f(0) = 0$ since $\psi(W_{1:T}; \eta^*)$ is independent of $W^{\mathrm{aux.}}$. Also, notice that $\partial f(0) / \partial r = 0$ by (ref)(ref). Finally, on the event $\mathcal{E}_T$,
where the convergence comes from (ref). \qed
(ref) states the consistency of our variance estimator $\hat \sigma^2$. We prove consistency for the estimators under (ref)(ref) and (ref)(ref) separately. For (ref)(ref), we can directly apply a consistent variance estimation result from prior work. For (ref)(ref), we prove consistency of the plugin variance estimator directly. We handle the variance terms and covariance terms in $\psi$ separately. For the variance terms, we follow the argument in chernozhukov_doubledebiased_2018. For the covariance terms, we follow a similar pattern to the argument for the variance terms: show that the average of the covariances converge in probability to covariance estimates computed using oracle nuisances, and then showing that the covariance estimates computed using oracle nuisances converge to their expectation.
Proof. We split the analysis into two cases. The first case handles when (ref)(ref) holds and the second handles when (ref)(ref) holds.
\paragraph{Case 1: Under (ref)(ref).} We will apply the following result adapted from jones_fixed-width_2006. For simplicity, we will use $T_1, T_2$ as defined in the statement of the result.
Notice that by (ref), (ref) and the fact that $K_\infty \in \mathcal{P}$, we have assumed $\left\| {\varphi(W_t; \eta)} \right\|_{L^{4+\delta}(K^\infty)} < C$. Thus, we can apply (ref)} and the result under (ref)(ref) follows.
\paragraph{Case 2: Under (ref)(ref)} Define
We will prove
which will imply the result, since $\sigma_T^2 \to \sigma^2$ trivially. Using the notation from (ref), notice
by definition. We will bound (ref) and (ref) separately. First, for (ref),
by adding and subtracting terms. Next, to bound (ref) in probability, notice
The first (in)equality comes from rearranging; the second comes from the Cauchy-Schwarz inequality. Now, we can bound (ref) as
where the first inequality comes from the fact that $(a+b)^2 \leq 2(a^2 + b^2)$ for all $a,b$; the second comes from adding and subtracting the conditional expectation and then again applying $(a+b)^2 \leq 2(a^2 + b^2)$. (ref) is $o_{P}(1)$ by Markov's inequality and (ref) since
(ref) is $o_{P}(1)$ by the following sequence.
The last line comes by Markov's inequality and the fact that $\mathbb{E}_{P}\left[ {\left( {\hat\varphi_t - \mathbb{E}_{P}\left[ {\hat\varphi_t} \right]} \right)^2} \right]right)^2} = O(1)$ for all $t$ by (ref) and the fact that $m$ is a constant independent of $T$. (ref) is $o(1)$ by the following sequence.
The first line comes by the fact that $\mathbb{E}_{P}\left[ {\psi(W_{1:T}; \eta^*) \; | \; W^{\mathrm{aux.}}} \right] = \psi^*$, the second is by definition of $\psi$, and the last is (ref). Thus, (ref) is $o_{P}(1)$. We can bound (ref) in probability as
where in each inequality, we apply the fact that $(a+b)^2 \leq 2a^2 + 2b^2$ and in the last line we apply Markov's inequality with (ref): Namely, from (ref) there exists a constant $C$ such that, for all $s, t \in [T]$,
(The last line applies the Cauchy-Schwarz inequality.) The fact that (ref) are $o_{P}(1)$ and $O_{P}(1)$ respectively imply that the expression in (ref) is $o_{P}(1)$. Next, we bound (ref) in probability. Notice
where in the last line we apply the triangle inequality $\left\| {{{\varphi_t^*} - \psi^*}} \right\|_{L^4(P)} \leq \left\| {{{\varphi_t^*}}} \right\|_{L^4(P)} + \left| \psi^* \right|$ and (ref). Thus, the fact that (ref) is $o_{P}(1)$ and that (ref) is $o_{P}(1)$ imply that (ref) is $o_{P}(1)$. To bound (ref),
To bound (ref),
The first equality comes from the fact that for all $a,b,a',b'$, it holds $aa' - bb' = (a-b)a' + (a'-b')b$; the second comes from rearranging; the third comes from (ref). Now, we just need to argue that ${\varphi_t^*} - \psi^* = O_{P}(1)$ and $\hat\varphi_{t} - \psi(W_{1:T}; \hat \eta) = O_{P}(1)$ for all $t$ to show that that (ref) is $o_{P}(1)$.
by (ref) so Chebyshev's inequality completes the argument. To bound (ref),
Thus, with Chebyshev's inequality, (ref) is $o_{P}(1)$. Then, the fact that (ref) is $o_{P}(1)$ and (ref) is $o_{P}(1)$ imply that (ref) is $o_{P}(1)$. Finally, the fact that (ref) and (ref) are each $o_{P}(1)$ completes the proof of the lemma. \qed \fi
In this section, we instantiate (ref) to estimate average direct effects in observational settings with shared-state interference. The average direct effect (ADE), informally, is the mean difference between treatment and control outcomes for each individual, keeping all other individuals' treatment assignments the same. It is also sometimes called the expected average treatment effect savje_average_2019. We first define a structural model of outcomes under interference in (ref), apply our DML theorem to this setting in (ref), and then validate our method for finite samples via simulations in (ref).
Informally, our structural model will consist of data generated according to a standard observational setup for units indexed $t=1,\dots,T$ --- with an outcome $Y_t$, covariates $X_t$ and treatment assignment $D_t$ --- except that we will allow for each outcome to depend on a shared state $H_t$, which is dependent on data up to time $t-1$. The shared state, which may be vector-valued, will be the channel through which spillovers may occur in the data: outcomes are dependent on shared states and shared states may be dependent on previous outcomes, covariates, treatments and shared states. In particular cases, the shared state might represent available inventory in a market, like the number of drivers available on a ride-sharing platform, or public information, such as the popularity of songs on a music streaming platform.
We will have, for each individual $t$, potential outcomes, shared-state and treatment given according to
where $Y_t(\cdot, \cdot)$ is a potential outcome depending on unit $t$'s outcome and the shared state, $H_t(\cdot)$ is the potential shared state depending on the data from time $t-1$, $D_t$ is the treatment assignment and $X_t$ are covariates. We use $Y_t$ and $H_t$ to denote the random variables realized from the data generating process (as opposed to $Y_t(\cdot, \cdot)$ and $H_t(\cdot)$ which describe the response surface of potential random variables under different treatment assignments and shared state or prior data). The structural model posits that $Y_t$ is determined by a function $f^*$ plus a stochastic residual term $\tilde Y_t$. Similarly, $H_t$ is a function of $h^*$ and residual $\tilde H_t$ and $D_t$ is a function of $m^*$ and residual $\tilde D_t$. We will assume the residual terms $\tilde Y_t, \tilde H_t, \tilde D_t$ obey
Thus, $m^*$ is a propensity score function since $P(D_t = 1 \; | \; X_t) = \mathbb{E}_P(D_t \; | \; X_t) = m^*(X_t)$.
By assumption, $H_t$ depends on data from prior time steps only through the observations at time $t-1$. Thus, $H_t$ satisfies the Markov property. We also note that this model allows for $H_t$ to be stochastic, even conditional on $W_{t-1}$, through the residual term $\tilde H_t$. Thus, the shared state may fluctuate due to factors exogenous to the previously arriving units' behavior. We also assume that $H_t$ satisfies either (ref)(ref) or (ref)(ref). This implies that the whole chain, consisting of observations $\{ W_t \}_{t=1}^T$ satisfies (ref).
Our estimand in this section will be the average direct effect (ADE). It is defined:
where the expectation is taken over $W_{1:T}$. In other words, this is the average effect on each individual of changing their treatment from control to treatment, without altering the distribution of others' treatments. (Note that the expectation $\mathbb{E}_P$ marginalizes over values of $H_t$, so the estimand is not conditional on realizations of $H_t$.)
Recall from (ref) that, in our setting the estimator $\psi(W_{1:T};\eta)$ is the sample average of functions $\varphi(W_t; \eta)$. In this section (and the next), the functional form of the time-$t$ estimator $\varphi$ can be written as the sum of two components: a plug-in term and a debiasing term. For convenience, we will denote these $\varphi^{\mathrm{pi}}$ and $\varphi^{\mathrm{db}}$; i.e., $\varphi(W_t; \eta) = \varphi^{\mathrm{pi}}(W_t; \eta) + \varphi^{\mathrm{db}}(W_t; \eta)$. These will be defined in this section as
where the nuisance estimators are $\eta = (f, m)$. The function $\varphi$ mirrors the augmented inverse probability weighting (AIPW) estimator robins_estimation_1994 that is standard in the semiparametric inference literature (see, e.g., wager_causal_2024 for background and history), with the exception that our outcome nuisance estimator $f$ takes the shared state $H_t$ as an argument. One way to interpret our estimator is as a covariate adjustment for the shared state: omitting the shared-state at a given time may result in a form of confounding between units' outcomes. However, it is not sufficient to treat $H_t$ as a covariate and apply DML methods as if the data were iid, since there is dependence between units which needs to be accounted for. We will see the contrast between our method and an approach that treats the shared state as if they were iid covariates in our simulations, and we note that, in (ref), treating the shared state as if they are covariates will not in general yield a consistent treatment effect estimator.
Before we state our main result in this section, we formalize assumptions sufficient for the result to hold.
In (ref), we require that each individual has true and predicted probability of assignment to treatment and to control bounded away from zero, almost surely. This is standard in settings that depend on estimation of the propensity score, since otherwise treatment assignments that occur with small probability may have a large influence on the estimator, making it unstable even as the sample size grows. We also require that the outcome function be bounded in $L^q$ norm, for $q > 4$ and that the error terms $\tilde Y_t$ are bounded in $L^2$ norm. These ensure that the concentration properties of the data $W_{1:T}$ propagate to the estimator. We state it formally next.
We note that since $m^*, f^* \in S_T$ by assumption, the regularity conditions (ref) also impose restrictions on the true nuisances.
In (ref), we require that the nuisance estimators $\hat m$ and $\hat f$ converge in $L^2$ to their true values $m^*$ and $f^*$ and that the product of their convergence rates vanishes at $o_P({T}^{-1/2})$ rates. Recall that for $q \geq 1$ the $L^q$ norm of a random variable $Z$ is $\mathbb{E}[|Z|^q]^{1/q}$.
The assumption is quantified over all $(f,m)$ in the nuissance realization set $S_T$. (ref) says that, if on average over units $t=1,\dots,T$, the expected squared difference between $m$ and $m^*$ must go to zero. (ref) says the same for $f$ and $f^*$. (ref) says that the average of the product in $m-m^*$ and $f-f^*$ over the data must go to zero at faster-than-$\sqrt{T}$ rates.
The required rates for (ref) can be achieved if, for example, when averaging over time steps, $\left\| {(m - m^*)(X_t)} \right\|_{L^2(P)} = o(T^{-1/4})$ and $\left\| {(f - f^*)(D_t, X_t, H_t)} \right\|_{L^2(P)} = o(T^{-1/4})$. On the other hand, if the propensity score function $m^*$ is known (if, e.g., it is determined via a randomized experiment controlled by the researcher), then $\left\| {(f - f^*)(D_t, X_t, H_t)} \right\|_{L^2(P)}$ can converge at an arbitrarily slow rate and still yield an efficient estimator.
There is a rich literature giving learning rates for machine learning estimators trained on dependent data. A key implication of our Markov chain assumptions is that the data are $\rho$-mixing, which intuitively means that observations that are far apart have diminishing correlations (see, e.g., bradley_chapter_2007 for formal definitions of mixing conditions). Learning rates for mixing sequences have been proved for neural networks ma_theoretical_2022, random forests goehry_random_2020 and several other machine learning methods irle_consistency_1997,steinwart_learning_2009,lozano_convergence_2014,wong_lasso_2020. See ballinari_semiparametric_2024 for further discussion.
With these two assumption in hand, we are ready to state our efficient inference result for this section.
(ref) instantiates (ref) for the model defined in (ref). It replaces the general assumptions required for (ref) with specific regularity and rates assumptions in (ref).
\ifproofsinbody Proof of (ref). If we can verify that (ref) hold, then we can apply (ref). Recall that (ref) is verified by assumption in the definition of the model in (ref). (ref)(ref) is trivially verified from the definition of $\psi$. (ref)(ref) is given by (ref). (ref), (ref) is trivially verified by the definition of $\psi$ and (ref). (ref), (ref) is given by (ref). (ref), (ref) is given by (ref). \qed
(ref) states that the estimator $\psi$ must be Neyman orthogonal with respect to the nuisance realization set $S_T$. The argument is standard to any proof of Neyman orthogonality of an augmented inverse probability weighted (AIPW) estimator.
Proof. Notice, for all $t$
\qed
(ref) states that, on average over time, $\varphi(W_t; \eta)$ must converge to $\varphi(W_t; \eta^*)$ in $L^2$, uniformly over the nuisance realization set $S_T$. The proof consists of showing that convergence of $\varphi$ reduces to conditions on convergence of $f$ to $f^*$ and $m$ to $m^*$, which we assumed in (ref).
Proof. Notice, for $\eta \in S_T$,
Thus, by the triangle inequality:
Next, the assumptions (ref) and (ref) combined imply
And finally
by (ref). \qed
(ref) states that the second-order term in Taylor's theorem converges to zero at faster-than-$\sqrt{T}$ rates. This ensures we can apply Taylor's theorem and establishes that slow rates of convergence of $\hat\eta$ to $\eta^*$ do not lead to sub-optimal rates on convergence of $\psi(W_{1:T}, \hat\eta)$ to $\psi^*$. The proof relies on showing that the second Gateaux derivative of $\psi$ on the path from any $\eta$ in the nuisance realization set $S_T$ to $\eta^*$ reduces to conditions on the products of convergence rates of $m$ to $m^*$ and $f$ to $f^*$. Thus, it is sufficient for each of $m$ and $f$ to converge to their true values at faster-than-$T^{1/4}$ rates, or for $m$ to be known a priori while $f$ converges at arbitrarily slow rates.
Proof. Observe, for $\eta \in S_T$ and $r \in (0, 1)$,
where we first switch the derivative with the expectation using (ref) and the dominated convergence theorem, and then we evaluate the derivatives. This implies
\qed
\fi
Next, we validate our results through simulations demonstrating the performance of the double machine learning estimator versus naive alternate estimators.
\paragraph{Setup for treatment effect estimator comparisons.} We compare our estimator, which we will sometimes call the DML4SSI estimator, to a naive plug-in estimator $\psi^{\mathrm{pi}}$, defined as
where $\varphi^{\mathrm{pi}}$ is defined as in (ref). Plug-in estimators constructed using machine learning models will converge at slower-than-$\sqrt{T}$ rates, and in finite samples may be very biased. We also compare our results to a naive difference-in-means Horvitz-Thompson (HT) estimator $\psi^\mathrm{HT}$, defined as
The naive HT estimator does not account for the shared state and thus may be inconsistent and very biased in finite samples. Finally, we include a comparison with a naive DML estimator (DML-N) which treats the data as if it were iid and fails to account for the shared state. The naive DML estimator $\psi^{\mathrm{DML-N}}$ is defined as the canonical iid AIPW estimator, ignoring the shared state:
where
The nuisances are $\eta' = (f', \hat m)$, where we learn an outcome model $f'$ by learning a predictor of $Y_t$ given $X_t$ and $D_t$ but omitting the shared state, and learn $\hat m$ as before by predicting $D_t$ as a function of $X_t$. (We use the notation $f'$ to disambiguate from $\hat f$, which takes $H_t$ as an argument.) To learn $f'$, we use an auxiliary sample as in (ref). Like the HT estimator, $\psi^{\mathrm{DML-N}}$ does not account for influence of the shared state and thus may in general be inconsistent and very biased in finite samples.
In our simulations, we generated data according to a smooth function of both $D_t, X_t$ and of $H_t$. For all machine learning predictors $\hat f$, $\hat m$ and $f'$, we used random forests with default parameters trained on auxiliary data of size $T$ sampled independently of the data used for inference. Then, we computed each of the estimators $\psi(W_{1:T}; \hat \eta), \psi^{\mathrm{HT}}(W_{1:T}; \hat \eta)$, $\psi^{\mathrm{pi}}(W_{1:T}; \hat \eta)$ and $\psi^{\mathrm{DML-N}}(W_{1:T}; \eta')$ and created a density plot for the bias of the resulting estimates in (ref). Full details of the simulation are available in (ref) and the code used to generate them is available at \href{https://github.com/johnchrishays/dml4ssi}{https://github.com/johnchrishays/dml4ssi}.
\paragraph{Results for treatment effect estimator comparisons.} In (ref), we observe that all the distributions of each of the naive estimators are substantially biased. By contrast, the DML estimator is centered around the true treatment effect. The bias in the plug-in estimator occurs despite the smoothness of $f^*$ and $m^*$ and the fact that the data is low dimensional. The bias in the HT and DML-N estimators comes from the fact that they do not account for the effect of the shared state and thus will in general be inconsistent. The direction and magnitude of the bias for each naive estimator is idiosyncratic to our synthetic data generating process and machine learners; in other settings the sign of the biases may be reversed, and the relative performance of different naive estimators may change.
\paragraph{Setup for confidence interval comparisons.} We next investigate the coverage of confidence intervals constructed from treatment effect and variance estimators. Confidence intervals for an arbitrary pair of treatment effect, variance estimators $(\hat \psi, \hat \sigma^2)$ are constructed as
where $z_\alpha$ is the $(1-\alpha/2)$-th quantile of a standard normal distribution. To compute coverage rates, we run many simulations, constructing a confidence interval for each one and calculating the proportion of confidence intervals that contain the estimand $\psi^*$.
The DML4SSI confidence intervals are constructed using $\psi(W_{1:T}, \hat\eta)$ and the consistent variance estimator $\hat \sigma^2$ in (ref). The Horvitz-Thompson (HT) confidence intervals use the treatment effect estimator $\psi^{\mathrm{HT}}(W_{1:T}; \hat \eta)$ as defined in (ref) and a standard variance estimator $\hat\sigma^2_{\mathrm{HT}}$ defined as
where $\overline{Y}^{(\ell)}$ for $\ell \in \{0, 1\}$ is the mean observed outcome conditional on $D_t = \ell$. The plug-in confidence intervals use $\psi^{\mathrm{pi}}(W_{1:T}; \eta)$ as defined in (ref) and the variance estimator defined in (ref), substituting $\varphi^{\mathrm{pi}}$ for $\varphi$ and $\psi^{\mathrm{pi}}$ for $\psi$. The naive DML (DML-N) estimator uses the treatment effect estimator $\psi^{\mathrm{DML-N}}$ defined as
In addition to the HT, plug-in and DML-N estimators compared above, we also compare our method to one where we treat the shared-state as covariates and construct treatment effect and variance estimates accordingly. We call these the shared-state as covariates (SSAC) estimators. In this section, the SSAC treatment effect estimator is mechanically the same as our treatment effect estimator, since both of them involve learning a conditional expectation function estimate $\hat f$. (In (ref), this is not true, and our treatment effect estimator and the SSAC estimators will be different, leading to inconsistent SSAC treatment effect estimates.) However, the SSAC variance estimator ignores covariances between terms and treats the data as if there were no covariances between terms. This is different from our variance estimator, which does account for covariances between terms. The SSAC variance estimator $\hat\sigma^2_{\mathrm{SSAC}}$ is defined as in (ref), substituting $\varphi$ for $\varphi^{\mathrm{DML-N}}$ and $\psi$ for $\psi^{\mathrm{DML-N}}$. These are the variance estimates that would have been constructed had the shared states been treated like iid covariates.
\paragraph{Results for confidence interval comparisons.} In (ref), we run simulations for different values of $T$ and construct confidence intervals for each estimator. In the left plot, we show coverage rates of the confidence intervals, and in the right plot, we show the confidence interval widths. We observe that the DML4SSI confidence intervals approach the target 95% coverage as $T$ increases, while each of the other estimators fall substantially below the target coverage. In particular, each of the HT, plug-in and DML-N confidence are substantially biased as we saw in (ref), leading to confidence intervals with coverage close to zero. The SSAC confidence intervals use the same (consistent) treatment effect estimator as the DML4SSI but have variance estimates that do not account for covariance between observations over time. This leads to underestimates of the variance, confidence intervals that are too narrow and lower-than-desired coverage. In this case, the SSAC 95% confidence intervals result in coverage between 0.6 and 0.7, and do not seem to be converging to the target coverage as $T$ increases.
In this section, we turn our attention to estimating the global average treatment effect (GATE) using switchback experiments. The GATE, as we will define formally shortly, is the difference in an the mean outcome of interest when all units are assigned to treatment versus when all units are assigned to control. Switchback experiments, which were formalized in bojinov_design_2022, allow for valid inference in settings where interference is $m$-dependent.
Switchback experiments are implemented by assigning treatments in sequential blocks so that $\ell > m$ units in a row receive the same (randomized) treatment. For example, for a block of $\ell$ sequential units, the experimenter may use a Bernoulli random draw to assign all units in the block to treatment or to control. Intuitively, switchback experiments account for interference by arranging for the observation of some units where all $m$ previous units received the same treatment assignment as the current unit. Our framework incorporates doubly robust regression adjustments that allow for lower variance estimates in settings where covariates and shared-state variable exert significant influence on outcomes (or, equivalently, in cases where treatment effect sizes are small). The structure of the remainder of this section is the same is in (ref): we define our setting in (ref), state an efficient inference result in (ref), and validate our results for finite samples via simulations in (ref).
Our model in this section is similar to that of (ref): for $t=1,\dots,T$, we have an outcome $Y_t$ that depends on a shared state $H_t$, covariates $X_t$ and treatment assignment $D_t$. This section is different from the previous in that we require $H_t$ to consist of the last $m$ observations of treatment assignments and covariates. This allows for satisfying the $m$-dependence condition required for switchback experiments to yield unbiased estimates of treatment effects bojinov_design_2022. Also, since we are considering switchback experiments, we assume $D_{1:T}$ to come from a known distribution determined by the experimenter.
Formally, we will analyze the following potential outcomes model on a population indexed $t=1,2,\dots$. For a constant $m \in \mathbb{N}$ and for each individual $t$ there is some binary treatment of interest $D_t \in \{ 0, 1 \}$, iid covariates $X_t \in \mathbb{R}^{d}$, shared state $H_t \in \mathbb{R}^{d_H}$ and outcome $Y_t \in \mathbb{R}$ given by
where $\mathcal{D}$ is a distribution determined by the experimenter. We will drop the arguments to $H_t$ when $D_{(t-m):(t-1)}$ are the observed treatment assignments. The residual terms $\tilde Y_t$ will be assumed to be mean zero and obey
Notice that the above implies that shared states are $m$-dependent as defined in (ref). For an index set $\mathbf{t} = (s:t)$, $s,t \in [T]$ and boolean vector $\mathbf{b} = \{ 0, 1 \}^{|\mathbf{t}|}$, define
to be the probability that the subvector $D_\mathbf{t}$ is equal to $\mathbf{b}$. The estimand will be the GATE, defined as
As in the previous section, our estimator $\psi$ will be defined by the sample average of the sum of a plug-in term $\varphi^{\mathrm{pi}}(W_t, \eta)$ and a debiasing term $\varphi^{\mathrm{db}}(W_t, \eta)$. We define these terms for this section as follows:
Note that the estimator has a form similar to the AIPW estimator of the previous section with a few differences. First, in the plug-in component of the estimator, we plug in the all treatment or all control vector for treatment assignments. Second, in the debiasing component of the estimator, we substitute in indicator functions for the $m$ past treatment assignments being equal to the current units' treatment assignment. These ensure that the estimator identifies the effect of all treatment versus all control.
{We will assume $\pi^*({(t-m):t}, \mathbf{1}), \pi^*({(t-m):t}, \mathbf{0}) > \zeta$ for all $t$. This is not always the way that switchback experiments are defined: in some contexts, the switching points may be deterministic, which would mean that any $t$ less than $m$ time steps after a (deterministic) switch point would have $\pi^*({(t-m):t}, \mathbf{1})= \pi^*({(t-m):t}, \mathbf{0}) = 0$. One way to achieve the constraint is to pick a switching length $\ell > m$ and then pick a first switch uniformly at random from the first $\ell$ time steps. Then for each block of constant treatment assignments, Bernoulli randomize. This will give $P(D_{(t-m):t} = \mathbf{1}) = P(D_{(t-m):t} = \mathbf{0}) \geq (\ell-m)/2\ell$.}
The assumptions necessary for efficient inference in switchback experiments under $m$-depen\-dence are analogous to those in (ref) and are deferred to the appendix. In (ref), we require that the probability that any $m+1$ sequential treatment assignments are the same (either all treatment or all control) is bounded away from zero. We also require that the conditional expectation estimator $f$ and the true conditional expectation function $f^*$ be bounded in $L^q$ norm for $q > 4$.
In (ref), we require that the time-average of the $L^2$ norm of $f-f^*$ is going to zero as $T \to \infty$.
(Since treatment assignments are randomized by the experimenter, propensity scores are known. This means we do not need to impose rate conditions on the convergence of $f$ to $f^*$, in line with the implications of standard DML theorems for inference in experiments chernozhukov_doubledebiased_2018.)
\ifproofsinbody Proof of (ref). As in the proof of (ref), we just need to verify (ref) and (ref) and apply (ref). (ref)(ref) is trivially verified by the definition of $\psi$. (ref)(ref) is verified by (ref). For (ref), (ref) is verified by (ref) and the definition of $\varphi$. (ref) is verified by (ref). (ref) is verified by (ref). \qed
(ref) states that our estimator is Neyman orthogonal with respect to the nuisance realization set. The proof resembles standard arguments of AIPW estimators, except that in our case the inverse-propensity is the replaced with the probabilities that the last $m$ observations where all assigned to treatment or control.
Proof of (ref). Notice:
where the first equality follows from (ref) and applying the dominated convergence theorem. The second equality follows by the fact that $\mathbb{E}\left[ {\tilde Y_t\; | \; D_{(t-m):t}} \right] = 0$ and rearranging. Now, using the facts that
and iterated expectations, the Gateaux derivative is zero. \qed
(ref) states the $L^2$ convergence of $\varphi(W_t;\eta)$, uniformly over the nuisance realization set $S_T$, to $\varphi(W_t; \eta^*$. Similar to the previous section, verifying the result reduces to conditions on the convergence of the nuisances, which in this case is satisfied by the a requirement that $f$ converges to $f^*$ and treatment probabilities are known.
Proof of (ref) Notice
Now, since, by (ref) and (ref), it holds
and by the fact that
almost surely, the triangle inequality proves the result. \qed
(ref) states the condition that second-order terms on the path from $\eta$ in $S_T$ to $\eta^*$ do not dominate. In this case, the condition is easily verified by the fact that the setting is experimental and therefore treatment probabilities are known.
Proof of (ref). The result holds trivially by (ref) and applying the dominated convergence theorem (to switch the derivative and expectation) and the fact that the second derivative is 0. \qed
\fi
We next validate the DML method for estimation of the GATE in switchback experiments.
\paragraph{Setting.} To guarantee $m$-dependence, the data generating process used for the plots in this section is different from that in (ref). In particular, the data generating process for $H_t$ in (ref) allowed for dependence on prior shared states, which allows for long-range dependencies between observations over time. In this section, $H_t$ is a function only of the last $m$ observations of treatments $D_{t-m:t-1}$ and covariates $X_{t-m:t-1}$. The setup is otherwise similar to that in (ref): $f^*$ is a smooth function of $D_t, X_t, H_t$ and we generate the predictor $\hat f$ using a random forest with default parameters. The switching length is chosen arbitrarily $\ell = 2m$. Full details of the simulations are deferred to (ref). We compare the performance of our estimator against naive estimators in (ref) and against an unbiased estimator for treatment effects in switchback experiments in (ref).
\paragraph{Treatment effect estimator comparisons.} In (ref), we compare our DML estimator to naive estimators. As in (ref), we compare our estimator to a naive Horvitz-Thompson estimator, a plug-in estimator and a naive DML estimator. Each estimator is defined the same way as in (ref), except that we plug in the known true propensity score $m^*$ instead of the estimated propensity score $\hat m$ since $m^*$ is known in an experiment. Additionally, we show the shared state as covariates estimator defined as
$\psi^{\mathrm{SSAC}}$ is the estimator obtained by treating the shared state as iid covariates.
Note that the naive estimators are biased away from the true GATE, while the DML estimator is approximately centered at the true effect. The SSAC estimator in this section is biased, intuitively, because it controls for the spillover effects between units, rather than including them in the effect estimate: the global average treatment effect includes both the direct effect of treatment and the indirect effects of others' treatments.
In (ref), we compare our estimator with an unbiased switchback estimator proposed in bojinov_design_2022, which is defined as \ificml
\else
\fi Note that both the DML and switchback HT estimator distributions are centered at the true effect, but the DML estimator has substantially lower variance. This is in accordance with the fact that some of the variation in outcomes is explained by the shared state and covariates (and is learned by expressive machine learning models), and the fact that the DML estimator uses all of the data while the switchback HT estimator drops the first $m$ observations after any switch. We leave more detailed exploration of the performance of the estimand (compared to (1) the Horvitz-Thompson-style switchback estimator and (2) across different choices of $\ell$) for future work.
\paragraph{Confidence interval comparisons.} We next explore the coverage of confidence intervals constructed from treatment effect and variance estimators. We use the same procedure to generate confidence interval coverage (ref) as we do for (ref): we run many simulations where we generate confidence intervals and compute the fraction of them that contain the true effect $\psi^*$. The definitions of the treatment effect and variance estimators for the naive HT, plug-in, naive DML estimators are the same as in (ref). We use our DML treatment effect estimator $\psi(W_{1:T}; \hat\eta)$ specified by (ref) and the consistent variance estimator from (ref). For the switchback HT estimators, we use the treatment effect estimator from (ref) and the conservative variance estimator provided in bojinov_design_2022. For the SSAC estimators, we use the treatment effect estimator from (ref) and the naive variance estimator as defined in (ref).
We observe that the DML4SSI confidence intervals are close to the target 95% coverage, while the switchback HT estimator provides substantially greater than 95% coverage. This is consistent with the fact that their variance estimator is conservative and will not be consistent in general. The mean confidence interval width constructed from the SB estimator (yellow) is between 3 and 7.5 times the width of that constructed from the DML4SSI estimator (red): put another way, in these simulations, the DML4SSI estimator requires many fewer samples to produce confidence intervals the same width as the SB estimator for a given time horizon $T$. Finally, we note that the naive estimators and plugin estimators all provide coverage near zero. This occurs as a result of the biases of each of these estimators for estimation of the treatment effect.
In this paper, we formally define shared-state interference and develop methods for efficient estimation of treatment effects using double machine learning. Shared-state interference occurs when spillovers in a system are channeled through a low-dimensional statistic like prices, inventory or information. Our motivating examples of shared-state interference are marketplaces and recommender systems, but there are many contexts where researchers and practitioners wish to measure treatment effects and where shared-state interference is present. Our method allows for the use of expressive machine learning estimation of nuisance parameters while achieving $\sqrt{T}$-rates. We also provide a consistent variance estimator.
In future work, it would be valuable to explore a number of theoretical and empirical questions around shared-state interference. Theoretically, it would be interesting to explore what other estimands (e.g., the average indirect effect as in munro_treatment_2023 or time-discounting as in farias_correcting_2023) or structural models allowing for inference using our (ref). It would also be valuable to characterize alternate forms of interference that have a similar structure: for example, in some situations, a shared state may influence treatment assignments (which we assume does not occur in our structural models). Empirically, it would be interesting to apply our methods to markets or information systems that exhibit shared-state interference. We hope that this work inspires further investigations of tractable interference structures in semiparametric inference.
\ificml \else \fi