Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
83,027 characters · 25 sections · 41 citation commands
Network Synthetic Interventions: A Causal Framework for Panel Data Under Network Interference
\ifarxiv
\else } \fi
There is growing interest in the identification and estimation of causal effects in the context of spillover on networks, in which the outcomes of a unit are affected by the treatments assigned to other units, known as the unit’s “neighbors.” Here, a unit could be an individual, customer cohort, or region, and correspondingly, treatments could be recommendations, discounts, or legislation. For example, whether an individual gets COVID-19 is a function of not only the individual’s vaccination status but also the vaccination status of that individual’s social network. In the setting of e-commerce, the number of goods sold of a particular product is a function of not only whether that product gets a discount, but the discount level of other products that are substitutes or complements of it. That is, there is network interference.
In this work, we focus on network inference with {\em panel data}, a ubiquitous manner in which data is structured, where we collect multiple measurements of different units, and each unit can undergo a different sequence of treatments. See Figure (ref) for an example of panel data and the type of causal question we are interested in. Causal inference with panel data has recently received significant attention, and a popular class of estimators in such settings are known as matching estimators, where one represents the outcomes of one unit as some combination of other units to answer counterfactual questions. Such estimators have been very popular in practice due to their flexibility and simplicity in addition to the fact that they provide valid causal estimates under unobserved confounding with appropriate assumptions. Some examples of matching estimators with panel data include Difference-in-Differences (DiD) bertrand2004much, Synthetic Controls (SC) abadie2021using, and variants thereof. However, such matching estimators rely on the Stable Unit Treatment Value Assumption (SUTVA), which implies that there is no spillover across units, i.e., the treatment applied to one unit does not affect the outcomes of other units. Failing to account for spillovers can lead to biased estimates.
We propose a novel latent factor model---which is a generalization of models studied in the panel data literature---that accounts for network interference. Given this model, we establish an identification result where the counterfactual potential outcome for a given unit and its neighbors can be written as a linear combination of the observed outcomes of a carefully selected set of other units. This identification result leads to a natural estimator, which we call Network Synthetic Interventions (NSI), a simple two-step procedure, that estimates the mean counterfactual potential outcome for a given unit. We then show that, given our latent factor model, the NSI estimator is finite-sample consistent and asymptotically normal under suitable conditions. NSI and our analysis of it can be viewed as a generalization of the Synthetic Interventions agarwal2020synthetic and, in turn, Synthetic Controls frameworks to account for network interference.
We furnish two validity tests that verify whether the treatment assignment pattern and the observed data have enough variation such that valid counterfactual estimates can be produced. Motivated by these tests, we provide a novel graph-based experiment design.
To explain the efficacy of the experiment design and the NSI estimator, we consider the setting of a regular network graph with degree $d \geq 2$. We show that the proposed experiment design requires only $O(d^3)$ training samples in order to guarantee that the training data has enough variation such that it is possible to generalize to a given target counterfactual treatment. Further, NSI obtains an estimate within error $\varepsilon$ with high probability under the proposed experiment design when $O\big({\sf poly}(d) / \varepsilon^4\big)$ training samples per unit are available. This is a significant improvement over the $O({\sf exp}(d) / \varepsilon^2)$ training samples that a naive procedure would require.
We conclude with simulations showing that NSI is robust to spillovers under which existing estimators are biased.
The literature on causal inference with network interference or spillover effect has mostly considered the setting of a single measurement per unit, whether in the setting of a randomized experiment or an observational study. Under fully arbitrary interference, it has been shown that it is impossible to estimate any desired causal estimands as the model is not identifiable Manski13, AronowSamii17, BasseAiroldi17, karwa2018systematic. Subsequently, various models have been proposed in the literature that impose restrictions on the exposure functions Manski13, AronowSamii17, viviano2020experimental, auerbach2021local, li2021causal, interference neighborhoods UganderKarrerBackstromKleinberg13, bargagli2020heterogeneous, SussmanAiroldi17, pmlr-v115-bhattacharya20a, parametric structure ToulisKao13, BasseAiroldi15, cai2015social, GuiXuBhasinHan15,EcklesKarrerUgander17, two-sided platforms johari2022experimental, bajari2021multiple or a combination of these, each leading to a different solution concept. A comprehensive review on network interference models is given by de2017econometrics. In this work, we focus on network interference that is additive across the neighbors, referred to in the literature as the joint assumptions of neighborhood interference, additivity of main effects, or additivity of interference effects SussmanAiroldi17, yu2022graph, cortez2022exploiting, cortez2022graph.
Distinct to our work is that we consider a {\em panel data} setting in which there are multiple measurements (e.g., a time series) for each unit. The potential outcomes function is thus also dependent on both the unit and the measurement. Additionally, we allow for the estimation of unit-specific counterfactuals under {\em multiple treatments}, whereas the existing literature has largely focused on binary treatments. Key to our approach is a novel latent factor model that takes into account network interference and is a generalization of the factor models typically used in panel data settings. Previous work has focused on causal estimands that capture population-level effects, such as the average direct treatment effect (the average difference in outcomes if only one unit and none of its neighbors get treated BasseAiroldi15, JagadeesanPillaiVolfovsky17, SavjeAronowHudgens17, SussmanAiroldi17, leung2019causal, ma2021causal) and the average total treatment effect (the average difference in outcomes if all units get treated versus if they do not UganderKarrerBackstromKleinberg13, EcklesKarrerUgander17, chin2019regression, yu2022graph, cortez2022exploiting, cortez2022graph). Alternately there has been some literature that focuses on hypothesis testing for the presence of network interference Aronow12, BowersFredricksonPanagopoulos12,AtheyEcklesImbens17,PougetAbadieSaveskiSaintJacquesDuanXuGhoshAiroldi17,saveski2017detecting; these results do not immediately extend to estimation as they are based on randomization inference with a fixed network size, and focus on testing the sharp null hypotheses.
While a majority of the literature focuses on randomized experiments, there is a growing interest in the literature to account for network interference when analyzing observational studies. The existing literature generally assumes partial interference, where the network consists of many disconnected sub-communities TchetgenVanderWeele12, perez2014assessing, liu2016inverse, DiTraglia2020, vazquez-bare2022. Without this strong clustering condition, other works impose strong parametric assumptions on the potential outcomes function, assuming that the potential outcomes only depend on a known statistic of the neighborhood treatment, e.g. the number or fraction of treated verbitsky2012causal, chin2019regression, ogburn2017causal. This reduces estimation to a regression task under requirements of sufficient diversity in the treatments. belloni2022neighborhood also consider a setting in which the exposure mapping is known but allow the “radius” of interference to vary across units, then learn this radius from data to devise a doubly robust estimator. forastiere2021identification consider a general exposure mapping model alongside an inverse propensity weighted estimator, but the estimator has high variance when the exposure mapping is complex. de2018recovering and de2019identifying derive identification conditions when the observational panel data contains no information about the social ties (i.e., network). Further, building on recent works in panel data agarwal2020synthetic, causalmatrixcompletion, we allow for unobserved confounding in treatment assignment as long as there exist low-rank latent factors that mediate the unobserved confounding, i.e., there is “selection on latent factors”.
We begin with some notation. Let $[X] \defeq \{1,\dots, X\}$ for any positive integer $X$. For vector $\ba \in [D]^N$ and set $S \subseteq [N]$, let $\ba_S \in [D]^{|S|}$ denote the vector containing the elements of $\ba$ indexed by $S$ and $a_i \in [D]$ denote the $i$-th element of $\ba$. Let $\bbI_x$ denote the $x \times x$ identity matrix and $\otimes$ denote the Kronecker product. Let $\ind(\cdot)$ denote the indicator function. Let $\norm{\cdot}_{\psi_2}$ denote the Orlicz norm. Let $O_p$ denote a probabilistic version of big-$O$ notation and $\tilde{\Omega}$ denote the variation on big-$\Omega$ notation that ignores logarithmic terms (see Appendix (ref) for precise definitions). For sets of indices $S_1 \subseteq [m_1]$ and $S_2 \subseteq [m_2]$ and a matrix $\Pi \in \bbR^{m_1 \times m_2}$, let $\Pi[S_1, S_2] \in \bbR^{|S_1| \times |S_2|}$ denote the submatrix corresponding to the rows indexed by $S_1$ and columns index by $S_2$. We use “$\col$” as a shorthand for all indices such that $\Pi[\col, S_2] \in \bbR^{m_1 \times |S_2|}$ and $\Pi[S_1, \col] \in \bbR^{|S_1| \times m_2}$. Let $\cX^*$ denote the $*$-product space, where its length is not pre-determined. Let $\Pi^+$ denote the pseudo-inverse of $\Pi$.
Consider $N \geq 1$ units, $D \geq 1$ treatments, and $T \geq 1$ measurements of interest. We denote the potential outcome for a given unit $n$ and measurement $t$ by the real-valued random variable $Y^{(\bintv)}_{t,n}$, where $\bintv \in [D]^{N}$ denotes the vector of treatments over all $N$ units. This definition allows for spillover effects because the potential outcome for a given unit is a function of the treatment assignment of all units. To model spillover across units, we use a network graph. Let $\cG = ([N] , \cE)$ denote a graph over the $N$ units, where $\cE \subseteq [N] \times [N]$ denotes the edges of the graph. Throughout, we assume that $\cG$ is {\em fixed} and {\em known}. Let $\cN(n)$ denote the neighbors of unit $n \in [N]$ with respect to $\cG$ such that $j \in \cN(n) \iff (j, n) \in \cE$. For simplicity of notation, let self-edges be included, i.e., $(n, n) \in \cE$ for all $n \in [N]$. We assume that the network graph $\cG$ captures spillover effects in the following way.
See Figure (ref) for an example of spillover and its network representation. Several prior works on network interference also assume SNTVA, e.g., as the Neighborhood Interference Assumption (NIA) sussman2017elements. It can be viewed as a particular instantiation of exposure mappings, as defined by aronow2017estimating, and effective treatment functions (e.g., under the constant treatment response assumption) Manski13.
In this section, we introduce the model that we use to develop our estimator and formal results.
We make several remarks. First, we note that Assumption (ref) automatically satisfies Assumption (ref). Second, the latent factor $\bu_{j, n}$ captures the effect in the potential outcome $Y^{(\bintv_{\cN(n)})}_{t, n} $ due to the interaction between node $n$ and its neighbour $j$; analogously $\bw_{t , a_j}$ captures the effect due to the treatment that neighbor $j$ receives (i.e., $a_j$) for measurement $t$. Specifically, their effect is captured through the inner product $\left< \bu_{j, n} , \bw_{t , a_j} \right>$. In this sense, the spillover effect of different neighbors in (ref) is additive. Lastly, (ref) can be equivalently written as
where \ifarxiv
\else $ \tilde{\bu}_{n, \cN(n)} \defeq [ \bu_{\cN_1(n), n}^\top \, , \, \hdots \, , \, \bu_{\cN_{|\cN(n)|}(n), i}^\top ]^\top $ and $ \tilde{\bw}_{t , \ba_{\cN(n)}} \defeq [ \bw_{t, a_{\cN_1(n)}}^\top \, , \, \hdots \, , \, \bw_{t, a_{\cN_{|\cN(n)|}(n)}}^\top ]^\top$. \fi Here, $\cN_i(n)$ refers to $i$-th neighbor of $n$. (ref) is reminiscent of classical interactive fixed effects models studied in the literature. Indeed, we can think of $\tilde{\bu}_{n, \cN(n)} \in \mathbb{R}^{r |\cN(n)|}$ and $\tilde{\bw}_{t, a_{\cN(n)}} \in \mathbb{R}^{r |\cN(n)|}$ as the network-adjusted latent factors and $r |\cN(n)| \in \bbN_{> 0}$ as denoting the network-adjusted “rank” (note that $r|\cN(n)|$ is actually an upper bound on the model's rank, but we will refer to it as the network-adjusted rank for convenience).
We discuss how examples of latent factor models previously studied in the literature are captured by the model we propose in Assumption (ref). Further, we discuss how additive non-linear latent factor models can be approximated by the linear additive model we propose.
Recall that we consider the panel data setting in which we observe $T$ measurements (e.g., a time series) for every unit. Let $a^t_n \in [D]$ denote the treatment assignment for unit $n$ at measurement $t$; $\ba^t \in [D]^N$ denote the vector of treatment assignments for all $N$ units at $t$; and $A \defeq [ \ba^{1}, \ba^{2}, \hdots, \ba^{T} ] \in [D]^{N \times T}$ denote the sequence of treatment assignments across all $N$ units and $T$ measurements. Note that the treatment assignments $A$ are observed, and the potential outcome $Y_{t,n}^{(\ba^t)}$ is observed for every unit $n \in [N]$ and measurement $t \in [T]$. We denote the observed outcomes for unit $n$ at measurement $t$ by $Z_{t,n} = Y^{(\ba^{t})}_{t,n}$ for all $t \in [T]$ and the matrix of observations by $Z \in \bbR^{T \times N}$.
To define the target causal estimand of interest, let $\cT_{\post} \subseteq [T]$ refer to a subset of measurements for which we would like to make counterfactual predictions; let $T_{\post} = |\cT_{\post}| \leq T$. To simplify notation, we assume without loss of generality that the treatment assignments are fixed across the measurements in $\cT_{\post}$, i.e., $A[:, t] = \ba^\post \in [D]^N$ for all $t \in \cT_\post$ (see Remark (ref)).
For any given unit $n$ and target treatment assignment $\tilde{\ba} \in [D]^N$, our goal is to estimate the individual potential outcome averaged over the prediction period:
using observations $Z$, where we condition on the latent factors, $\LF$. Under Assumption (ref), the outcome of unit $n$ depends only on the treatments applied to $\cN(n)$, i.e., on $\tilde{\ba}_{\cN(n)}$ rather than the entire $\tilde{\ba}$. See Figure (ref) for an illustration of our target causal estimand and observation pattern.
Note that our results are given for any $T_\post \geq 1$. That is, we show identifiability and finite-sample consistency even for point estimates (i.e., for $T_\post = 1$).
We now describe our estimator for the estimand of interest (ref), which we term {\em Network Synthetic Intervention} (NSI). It can be seen as a natural extension of the Synthetic Interventions (SI) estimator agarwal2020synthetic, which is itself a generalization of Synthetic Controls (SC) abadie2021using estimator, to settings in which there is network interference. For the remainder of this work, we fix the unit $n$ and counterfactual treatment assignment $\cfA$ of interest.
To define the NSI estimator, we introduce some necessary concepts. First, let $\cT_{\pre} \subset [T]$ denote a subset of the measurements known as {\em training} measurements. Without loss of generality, let $\cT_{\pre} \coloneqq \{1, 2, \hdots, T_{\pre}\}$, $\cT_{\post} \coloneqq \{T_{\pre} + 1, \dots, T\}$, $T_{\pre} \defeq |\cT_{\pre}|$, and $T_{\post} \defeq |\cT_{\post}|$. We note that $\cT_{\pre}$ does not need to be $[T] \backslash \cT_{\post}$ but we keep it as such to simplify the exposition. Recall that $A \in [D]^{N \times T}$ denotes the treatment assignments to the various units over time. Let $A^\pre \defeq A[: , \cT_{\pre}]$ and $A^\pre_n \defeq A[\cN(n) , \cT_{\pre}]$. Next, we introduce the notion of a “donor set."
For the remainder of this work, we fix the unit $n$ and counterfactual treatments $\cfAn$ of interest and let $\cI^{(n)} \subset [N] \setminus \{ n \}$ denote the corresponding set of donors. One can think of the donor set $\cI^{(n)}$ as units whose observed outcomes can be used to estimate the unobserved potential outcome of unit $n$ under the counterfactual treatments of interest.
Recall that $Z \in \bbR^{T \times N}$ denotes the matrix of observations. We define $\bz_{\pre, n} \defeq Z[\cT_{\pre}, n] \in \bbR^{T_{\pre}}$, $Z_{\pre, \cI^{(n)}} \defeq Z[\cT_{\pre}, \cI^{(n)}] \in \bbR^{T_{\pre} \times | \cI^{(n)} |} $, and $Z_{\post, \cI^{(n)}} \defeq Z[\cT_{\post}, \cI^{(n)}]\in \bbR^{T_{\post} \times | \cI^{(n)} |}$. NSI takes in one hyperparameter $\kappa \in [\min(T_{\pre}, |\cI^{(n)}|)]$ and proceeds in two steps, as follows. \ifarxiv \\ \else \fi
1. Point estimate. Let $\{(\hat{s}_\ell, \hat{\boldsymbol{\mu}}_\ell, \hat{\boldsymbol{\nu}}_\ell)\}_{\ell = 1}^{\min(T_\pre, |\cI^{(n)}|)}$ denote the set of singular values, left singular vectors, and right singular vectors for the observed matrix $Z_{\pre, \cI^{(n)}}$, where $\hat{s}_1 \geq \hat{s}_2 \geq \hdots \geq \hat{s}_{\min(T_\pre, |\cI^{(n)}|)} \geq 0$. The NSI estimator produced a point estimate as follows:
For the given hyperparameter $\kappa$, $ \hat{\bbE} [ Z_{\pre, \cI^{(n)}} | \LF , \cO]^+ = \sum_{\ell = 1}^\kappa \frac{1}{\hat{s}_\ell} \hat{\boldsymbol{\nu}}_\ell \hat{\boldsymbol{\mu}}_\ell^\top, $ can be viewed as an estimate of $\bbE [ Z_{\pre, \cI^{(n)}} | \LF , \cO]$ that is obtained via hard singular value thresholding, where only the top $\kappa$ components are preserved. This estimator, which begins with singular value thresholding, has been shown to be equivalent to principal component regression PCR_1, PCR_2. \ifarxiv \\ \else \fi
2. Confidence interval. Let $\hat{\boldsymbol{\alpha}} = \hat{\bbE} [ Z_{\pre, \cI^{(n)}} | \LF , \cO]^+ \bz_{\pre, n}$. Then, the $\textsc{CI}$-percent confidence interval can be constructed as:
where $\Phi$ denotes the cumulative distribution function (CDF) of the standard normal distribution, $\Phi^{-1}$ is the inverse CDF, and \ifarxiv $$\hat{\sigma}^2 = \frac{1}{T_{\pre}} \norm{ \bz_{\pre, n} - Z_{\pre, \cI^{(n)}} \hat{\boldsymbol{\alpha}} }_2^2 ,$$ which can be interpreted as the in-sample prediction error of the NSI estimator. \else $\hat{\sigma}^2 = \frac{1}{T_{\pre}} \norm{ \bz_{\pre, n} - Z_{\pre, \cI^{(n)}} \hat{\boldsymbol{\alpha}} }_2^2$, which can be interpreted as the in-sample prediction error of the NSI estimator. \fi
We briefly provide intuition for the NSI estimator, then compare it to the traditional Synthetic Control abadie2021using and Synthetic Interventions agarwal2020synthetic estimators. \ifarxiv \\ \else \fi
{\bf NSI linearly combines donor outcomes.} NSI begins by finding units, called “donors,” whose outcomes can be used to estimate the potential outcomes of unit $n$. In Section (ref), under suitable assumptions, we establish that the expected potential outcome of unit $n$ can be expressed as a linear combination of the expected outcomes of the donor units, i.e.,
where $\alpha_j \in \bbR$ (note that $\alpha_j$ can be negative). NSI can be viewed as a method for estimating the coefficients $\{ \alpha_j \}$. More precisely, recall that $\hat{\boldsymbol{\alpha}} = \hat{\bbE} [ Z_{\pre, \cI^{(n)}} | \LF , \cO]^+ \bz_{\pre, n}$. Then, the NSI estimator (ref) can be rewritten as \ifarxiv $$ \hIPO{n} = \frac{1}{T_{\post}} \sum_{t \in \cT_{\post}} \sum_{j \in \cI^{(n)}} \hat{\alpha}_j Z[t, j], $$ \else $ \hIPO{n} = \frac{1}{T_{\post}} \sum_{t \in \cT_{\post}} \sum_{j \in \cI^{(n)}} \hat{\alpha}_j Z[t, j], $ \fi which is precisely what would follow from expressing the target causal estimand (ref) using (ref). \ifarxiv \\ \else \fi
{\bf Comparing NSI to SI & SC: Choosing donors appropriately.} NSI is a generalization of SI and SC. The key difference between SC/SI and NSI is the choice of donor units. In SC/SI, a valid donor unit only needs to undergo the same training and prediction treatments as the training and target prediction treatments of $n$. In NSI, there are more stringent requirements on donors, as given by Definition (ref) and illustrated in Figure (ref). These more stringent requirements on how donors are chosen are how NSI removes the bias that SI and SC suffer from when there is spillover.
Under the appropriate choice of donors, the way that the linear model (ref) is learned can depend on modeling assumptions. NSI uses principal component regression (PCR), which is motivated by agarwal2020synthetic). However, other estimators, such as convex regression abadie2021using and variants thereof can also be used. In Section (ref), we detail the conditions under which PCR produces consistent and asymptotically normal estimates.
In this section, we provide formal results for the NSI estimator. We characterize conditions under which $\IPO{n}$ can be identified. We then establish that NSI provides a consistent estimate of $\IPO{n}$ and its estimation error is asymptotically normal, justifying the confidence interval given in Step 3 of Section (ref). As before, we restrict our attention to a specific unit $n$ and target counterfactual treatment assignment $\cfAn$. All proofs are given in Appendices (ref)-(ref).
We now discuss the key assumptions we make about the intervention assignments $A$. We begin with an assumption on the treatment assignment.
Given Assumption (ref), this conditional independence is equivalent to assuming that $\epsilon_{t, n}^{(\ba_{\cN(n)})} \perp \cO \, | \, \LF $. Similar conditions of “selection on latent factors” have been considered in the literature (see agarwal2020synthetic and discussion therein). In this work, we analogously require “selection on network-adjusted latent factors.” We make two additional assumptions, as follows.
We discuss Assumptions (ref)-(ref) in Sections (ref) and (ref). We show that NSI's confidence interval indicates the degree to which Assumption (ref) holds, and we provide a way to test for Assumption (ref) in Section (ref).
Before stating our identification result, we first recall identifiability for an arbitrary estimation problem of interest. In the definition below, $P_\theta \in \cP$ refers to data distribution under model parameters $\theta$, i.e., $P_\theta$ describes how the data behaves under model $\theta$.
Identifiability implies that if $f(\theta) \neq f(\theta')$, then $g(P_\theta) \neq g(P_{\theta'})$; otherwise, $f(\theta) = g(P_\theta) = g(P_{\theta'}) = f(\theta')$. In other words, the estimand is identifiable if we can compute it exactly when given access to the full data distribution, which is necessary for estimation from (noisy) data to be possible.
In our setting, $\theta = \LF$ denotes the latent factors and $f(\theta) = \IPO{n}$ is the target causal estimand, which we note can be written solely as a function of the latent factors. Recall that the quantities $(A, \cG, \cT_{\pre}, \cT_\post)$ are observed and known. Let $P_{\theta}$ denote the joint distribution over the matrices of observed outcomes $Z[\cT_\pre,:]$ and $Z[\cT_\post,:]$. Note that, by Assumption (ref), the distribution of $Z[\cT_\pre,:]$ and $Z[\cT_\post,:]$ is given by the latent factors $\LF$, the treatment assignment $A$, and the random variables $\epsilon_{t, n}^{(\ba_{\cN(n)})}$. We now show that $\IPO{n}$ is identifiable under (ref) and Assumptions (ref)-(ref).
Under Definition (ref), $g$ is given by (ref). Note only the first moments of $P_\theta = P_{\LF}$ are needed. NSI estimates $\IPO{n}$ by replacing the expectations in (ref) with the corresponding empirically observed quantities and smoothing out the pseudoinverse using hard singular value thresholding as given by (ref). For the purposes of the analysis, we denote
Next, we give conditions under which the NSI estimator achieves finite-sample consistency and asymptotic normality. Let $r_{\pre} \in [ r |\cN(n)| ]$ be the rank of $\bbE \big[ Z_{\pre, \cI^{(n)}} | \, \LF , \cO \big]$, $s_1 \geq \hdots \geq s_{r_{\pre}} \geq 0$ denote its singular values, and $R_{\pre} \in \bbR^{|\cI^{(n)}| \times r_{\pre}}$ denote its right singular vectors.
In Section (ref), we prove that in the setting of $d$-regular graphs and where the latent factors are sampled independently from a Gaussian distribution, the parameters $\xi'$ and $\xi''$ are inverse polynomials of $r$ and $d$.
The following results establish that NSI is consistent and asymptotically normal.
Theorem (ref) indicates that, under the stated conditions, NSI is consistent. Specifically, for fixed $r$, $d$, and $r_{\pre} \leq r (d+1)$, the estimation error of NSI approaches $0$ as the number of training measurements $T_\pre$ and number of donors $|\cI^{(n)}|$ grow if $T_\pre = \omega(| \cI^{(n)} |^{1/3})$. Importantly, the number of prediction measurements $T_\post$ need not grow in order for NSI's estimation error to decay to $0$.
Let $\Delta = \hat{\boldsymbol{\alpha}} - \boldsymbol{\alpha}$, the estimation error of learning the linear weights that represent the outcomes of the target units in terms of the donor units (see (ref)). The following result establishes a general result that as long as $\Delta$ is decaying sufficiently quickly for any linear estimator (it does not have to be via principal component regression as we do), the NSI estimator is asymptotically normal. While Theorem (ref) allows NSI to produce the point estimate in Step 1 of Section (ref), Theorem (ref) justifies the confidence interval provided in Step 2.
The key enabling conditions for Theorems (ref)-(ref) are Assumptions (ref), (ref), and (ref). In this section, we discuss these assumptions further. \ifarxiv \\ \else \fi
{\bf Assumption (ref).} Recall from Section (ref) that NSI can be interpreted as linearly combining the potential outcomes of donors under appropriately chosen coefficients, denoted by $\hat{\boldsymbol{\alpha}}$. The key enabling condition that makes linearly combining donor outcomes valid under our model (ref) is Assumption (ref). The extent to which Assumption (ref) holds can be examined in two ways.
First, recall that $\hat{\sigma}^2 = \frac{1}{T_{\pre}} \norm{ \bz_{\pre, n} - Z_{\pre, \cI^{(n)}} \hat{\boldsymbol{\alpha}} }_2^2$ is a measure of how well NSI's linear fit explains the training data. Recall further that the coefficients $\hat{\boldsymbol{\alpha}}$ are estimates of the coefficients $\boldsymbol{\alpha}$ that are used to combine donor outcomes. As such, $\hat{\sigma}^2$ can be viewed as a statistic for Assumption (ref), where a large $\hat{\sigma}^2$ suggests Assumption (ref) does not hold. Since NSI's confidence interval scales with $\hat{\sigma}$ (see Step 2 of (ref)), how well Assumption (ref) holds is captured by NSI's confidence interval.
Second, since Assumption (ref) requires that unit $n$'s $\tilde{\bu}$-latent factor is contained in the span of the donors' $\tilde{\bu}$-latent factors and $\tilde{\bu}_{n, \cN(n)} \in \bbR^{r |\cN(n)|}$, at least $r |\cN(n)|$ donors are needed. Suppose, for example, that all units' $\tilde{\bu}$-latent factors are drawn i.i.d. from a multivariate Gaussian. Then, Assumption (ref) holds almost surely if and only if there are at least $r |\cN(n)|$ donors (cf. Lemma (ref)). Given an estimate $\bar{r}$ of $r$, one can therefore perform a simple sanity check that $|\cI^{(n)}| \geq \bar{r}|\cN(n)|$. \ifarxiv \\ \else \fi
{\bf Assumption (ref).} This condition ensures that the linear coefficients that NSI learns from the training observations generalize to the prediction task. In Section (ref), we provide two validity tests to verify whether Assumption (ref) holds, i.e., whether the observations are sufficiently rich such that a generalizable model can be learned. To motivate the need for such tests, below we provide a simple example where Assumption (ref) does not hold. Suppose that $D = 2$ (the treatments are binary). Let
Intuitively, $B_{\pre, n}$ is an indicator matrix that tracks the training treatment assignment over $\cN(n)$.
Proposition (ref) shows that the diversity of treatment assignments (as captured by $B^{\pre, n} $) affects the feasibility of Assumption (ref). We unpack this relationship in detail in the next section. Before doing so, we present a negative example in which Assumption (ref) does not hold.
{ {\bf Assumption (ref).} This condition requires that the number of components used by NSI, given by $\kappa$ matches the rank $r_\pre$ of $\bbE[ Z_{\pre, \cI^{(n)}} | \LF, A ]$. Since $\bbE[ Z_{\pre, \cI^{(n)}} | \LF, A ]$ is unknown in practice, one must estimate $r_\pre$, which can be done by applying an elbow point (or knee point) method to the spectrum of the observed matrix $Z_{\pre, \cI^{(n)}}$ zack1977automatic, satopaa2011finding. There are other heuristics for setting $\kappa$, such as the universal thresholding method given in chatterjee2015matrix. Alternatively, suppose we have an estimate $\bar{r}$ of the model “rank” $r$, defined in Section (ref). By our model (ref), $r_\pre$ is upper bounded by $r | \cN(n) |$, which suggests that one can use the heuristic $\kappa = \bar{r} |\cN(n)|$. It also suggests that one should always set $\kappa$ to be at least $|\cN(n)|$, assuming that $r \geq 1$. }
We present two validity tests for Assumption (ref), one of the key enabling assumptions of Theorems (ref) and (ref). The first test can be performed before any data is collected. It tests for whether the treatment assignment in the training period is diverse enough relative to the target treatment assignment in the prediction period. The second test can be performed only after the data is collected and, as such, is a relatively stronger test. Proofs for this section can be found in Appendix (ref).
The first test can be run before the prediction samples are collected. \ifarxiv \\ \else \fi
TrainingTreatmentTest. This test takes in one hyperparameter $\bar{r} \in \bbN_{> 0}$, which is an estimate of the model “rank” $r$ (see Section (ref)). If one does not have a good estimate, $\bar{r}$ can also be an upper bound on $r$. In order to run this test, we first define several “masking” matrices. For a given treatment $a \in [D]$, let $B^{\pre}(a) \in \{0, 1\}^{N \times T_{\pre}}$ and $\bb^{\post}(a) \in \{0, 1\}^{N}$ be defined such that their $(i, t)$-th elements are $ B_{i t}^{\pre}(a) = \Ind(A^{\pre}_{i t} = a)$ and $ \tilde{b}_{i}^{\post}(a) = \Ind(\tilde{a}_i = a) . $ That is, the $(i, t)$-th entry of $B^{\pre}(a)$ is $1$ if and only if unit $i$ at measurement $t$ receives treatment $a$ under the training treatments $A^{\pre}$. Similarly, the $i$-th entry of $\tilde{\bb}^{\post}(a)$ is $1$ if and only if unit $i$ is assigned the target counterfactual treatment $a$ under $\tilde{\ba}$. Let $B^{\pre}$ and $\tilde{B}^{\post}$ be the concatenated matrices across different treatments:
For the hyperparameter $\bar{r}$, the NSI estimator passes the TrainingTreatmentTest if
otherwise, it fails. \ifarxiv \\ \else \fi
Connecting the TrainingTreatmentTest to Assumption (ref). The following result formalizes the relationship between the test and Assumption (ref) under a natural data generating process.
As such, TrainingTreatmentTest tests whether Assumption (ref) can hold under the training treatments for the given target treatment of interest. Intuitively, it requires that the training treatments are sufficiently diverse relative to $\tilde{\ba}_{\cN(n)}$. In Section (ref), we provide an experiment design that guarantees TrainingTreatmentTest is passed for any $n$ and $\tilde{\ba}_{\cN(n)}$. Note that the i.i.d. condition in Proposition (ref) can be relaxed, as long as the latent factors are always drawn from non-degenerate distributions. The condition on latent factors ensures that there is enough variation across units and time such that we can isolate the role that training treatment assignments plays in Assumption (ref) from the role that latent factors play.
We now furnish a data-driven check for Assumption (ref) that we call the SubspaceInclusionTest. This test can be run only after the training and prediction samples are collected as opposed to the TrainingTreatmentTest test, which can be run beforehand. \ifarxiv \\ \else \fi
SubspaceInclusionTest. The test takes in three hyperparameters: $\kappa$, $\kappa'$, and $\gamma$. Note that we overload $\kappa$ (which also appears in Section (ref)) because both instances refer to an estimate of the rank of $\bbE[Z_{\pre, \cI^{(n)}}]$. Similarly, let $\kappa'$ denote the estimated rank of $\bbE[Z_{\post, \cI^{(n)}}]$, respectively. (Refer to Appendix (ref) for various approaches to selecting parameters $\kappa$ and $\kappa'$.) The third $\gamma \in (0, 1)$ is a tunable parameter, where a smaller $\gamma$ results in a stricter test.
Let $\hat{R}_{\pre} \in \bbR^{|\cI^{(n)}| \times \kappa}$ and $\hat{R}_{\post} \in \bbR^{|\cI^{(n)}| \times \kappa'}$ denote the matrices constructed from the top $\kappa$ right singular vectors of $Z_{\pre, \cI^{(n)}}$ and the top $\kappa'$ right singular vectors of $Z_{\post, \cI^{(n)}}$, respectively. Let
Then, the NSI estimator passes the SubspaceInclusionTest if $\hat{\beta} \leq (1 - \gamma) \kappa'$; otherwise, it fails. \ifarxiv \\ \else \fi
SubspaceInclusionTest is a data-driven check for Assumption (ref). Let ${R}_{\post}$ and ${R}_{\pre}$ denote the matrices constructed from the right singular vectors of $\bbE \big[ Z_{\pre, \cI^{(n)} } \big| \, \LF, \cO \big]$ and $\bbE \big[ Z_{\post, \cI^{(n)} } \big| \, \LF, \cO \big]$, respectively. Then, Assumption (ref) can equivalently be stated as requiring that $\text{columnspace} (R_{\post}) \subseteq \text{columnspace}(R_{\pre})$. Although one cannot directly test for Assumption (ref) since both $\bbE \big[ Z_{\pre, \cI^{(n)} } \big| \, \LF, \cO \big]$ and $\bbE \big[ Z_{\post, \cI^{(n)} } \big| \, \LF, \cO \big]$ are not observable due to noise, we now show that SubspaceInclusionTest is a sample-based test for Assumption (ref) using $\hat{R}_\post$ and $\hat{R}_\pre$. Recall that SubspaceInclusionTest fails if $ \hat{\beta} = \norm{ (\mathbb{I} - \hat{R}_{\pre} \hat{R}_{\pre}^\top ) \hat{R}_{\post} }_F^2 \geq (1 - \gamma) \kappa',$ where $\hat{R}_{\pre}$ and $\hat{R}_{\post}$ contain the top $\kappa$ and $\kappa'$ right singular vectors of $Z_{\pre, \cI^{(n)}}$ and $Z_{\post, \cI^{(n)}}$, respectively. This can be viewed as a test for Assumption (ref) since smaller values of $\hat{\beta}$ indicate the extent to which $\text{columnspace} (\hat{R}_{\post}) \subseteq \text{columnspace}(\hat{R}_{\pre})$. Indeed, suppose that $R_{\pre} = \hat{R}_{\pre}$, $R_{\post} = \hat{R}_{\post}$, and Assumption (ref) holds; then, $\hat{\beta} = 0$. As the span of $\hat{R}_{\post}$ moves outside of the span of $\hat{R}_{\pre}$, $\hat{\beta}$ increases. Since $\hat{\beta} = \norm{ (\mathbb{I} - {R}_{\pre} {R}_{\pre}^\top ) {R}_{\post} }_F^2$ is always upper bounded by $r_{\post}$, which is estimated by $\kappa'$, we use the threshold $(1 - \gamma) \kappa'$ such that the test fails if $\hat{\beta} \geq (1 - \gamma) \kappa'$. A formal analysis of this test remains important future work.
In this section, we propose an experiment design based on graph coloring, under which we can precisely answer the question of how should $T, N$ scale to enable the estimation of $\IPO{n}$ within $\varepsilon \in (0,1)$? Proofs for this section can be found in Appendix (ref).
We begin by describing the experiment design. The design assumes access to a subroutine $\textsc{TwoHopColoring}(\cG)$ that outputs $\textsc{NumColors}, \textsc{Coloring}$ and proceeds in two steps: (1) Construct the graph $\cG' = ([N], \cE')$ such that $(i, j) \in \cE \implies (i, j) \in \cE'$ and $(i, j) , (j, k) \in \cE \implies (i, k) \in \cE'$. That is, $\cG'$ is constructed by taking $\cG$ and adding edges between every node and its two-hop neighbors. (2) Perform a coloring on $\cG'$ by labeling the vertices in a graph such that no two adjacent vertices receive the same color, greedily adding colors whenever an existing color cannot be used. (Under a color, vertices of the same color form an independent set of $\cG'$.) Let $\textsc{NumColors}$ denote the number of colors required to color $\cG'$. Let $\textsc{Coloring} \in [\textsc{NumColors}]^N$ denote the colors assigned to each node (or unit). As before, let $\bar{r} \in \bbN_{> 0}$ denote an estimate (or, alternatively, an upper bound) of the model “rank” $r$. Then, the experiment design procedure proceeds as follows. \ifarxiv \\ \else \fi
Step 1. Let $\textsc{NumColors}, \textsc{Coloring} = \textsc{TwoHopColoring}(\cG)$. \ifarxiv \\ \else \fi
Step 2. Divide the colors into $T' = \lceil \frac{\textsc{NumColors}}{D - 1} \rceil$ disjoint sets $\{ \textsc{Colors}_\ell : \ell = 0, 1, \hdots, T' - 1 \}$ such that $\textsc{Colors}_1$ contains the first $D - 1$ colors, $\textsc{Colors}_2$ contains the next $D - 1$ colors, and so on.
Step 3. Then, for $\ell = 0, 1, \hdots, T' - 1$, let $\bc^\ell \in [D]^N$ denote a treatment vector such that
The intuition behind $\bc^\ell$ is as follows. Since $\textsc{Colors}_\ell$ contains at most $D - 1$ colors, each color in $\textsc{Colors}_\ell$ can be associated with a different treatment in $\{ 2, 3, \hdots, D\}$. Units with one of those colors receive the corresponding treatment. That is, any unit $i$ for which $\textsc{Coloring}_i \in \textsc{Colors}_\ell$ is assigned the corresponding treatment in $\{ 2, 3, \hdots, D\}$. All other units receive treatment $1$. \ifarxiv \\ \else \fi
Step 4. Let $\cT_{\pre}$ be divided into disjoint sets $\{ \cT^\ell_{\pre} : \ell = 0, 1, \hdots, T' - 1 \}$, each of length $\bar{T} \geq \bar{r} D$ such that $T_{\pre} = \bar{T} T'$. Then, let $A^{\pre} \in [D]^{N \times T_{\pre}}$ be defined such that, $ A^{\pre}[:, t] = \bc^{\ell} \quad \forall t \in \cT^\ell_{\pre} $ for all $\ell = 0, 1, \hdots, T' - 1$. For the remainder of this work, we assume $\bar{T} = \bar{r} D$ unless otherwise stated. \ifarxiv \\ \else \fi
Step 5. Let the prediction treatment of each unit be assigned i.i.d. uniformly at random from the $D$ possible treatments, i.e., $a^\post_i \iid \text{Unif}(D)$ for all $i \in [N]$. \ifarxiv \\ \else \fi
{\bf Discussion of experiment design.} The experiment design described above uses the graph coloring over the two-hop version of $\cG$ to assign treatments. The training measurements are divided into $T'$ disjoint sets---that we will call “periods”---denoted by $\cT^\ell_\pre$ for $\ell = 0, 1, \hdots, T' - 1$. The treatment assignment during each period remains constant, i.e., for any $\ell$, $\ba^t = \bc^\ell$ for all $t \in \cT^\ell_\pre$.
The experiment design ensures several important properties hold. First, all nodes that receive the same color also receive the same treatment at any $t \in \cT_\pre$. Second, nodes that receive the same color have to be at least three hops away from one another, which ensures that, for any neighborhood $\cN(n)$ of an arbitrary node $n$, no two nodes receive the same non-control treatment at any given $t \in \cT_\pre$. Third, each node receives the control treatment $1$ for every period, except for one period during which it receives a non-control treatment. The non-control treatment that a node receives is given by $ \textsc{Coloring}_i \, \text{mod} (D - 1) + 2$, where $\textsc{Coloring}_i$ denotes the color assigned to unit $i$. Lastly, each period has length $\bar{T} \geq \bar{r} D$. We show that these properties are important to proving Lemma (ref) in the next section. See jensen2011graph for further information on graph colorings.
We present two results. The first establishes that the graph-theoretic experiment design described in Section (ref) guarantees that TrainingTreatmentTest is passed.
Importantly, Lemma (ref) gives a guarantee for any unit $n$ and counterfactual treatments $\cfAn$ of interest, i.e., for all $N D^d$ possible estimands of interest. That is, the experiment design in Section (ref) can be used to ensure that TrainingTreatmentTest is passed for any target estimand of interest. {Moreover, the experiment design can be applied to any graph of interest (and any valid two-hop coloring, as described in Section (ref)).} In Appendix (ref), we discuss how one can adapt our experiment design to be less stringent if one is interested in a specific choice of $n$ and $\cfAn$.
The next result shows that the number of training measurements required under the graph-theoretic experiment design is $O(d^2)$, where $d$ denotes the maximum degree of $\cG$.
By Lemma (ref), the experiment design requires $T_\pre = O(d^2)$ training measurements. Since $r|\cN(n)| = r d$ donors are needed for a given $n$ and $\tilde{\ba}_{\cN(n)}$ of interest (as discussed in Section (ref)), this result shows that one needs $O(d^3)$ training samples under the experiment design to guarantee that TrainingTreatmentTest passes for a given unit and target treatment of interest. That is, with $O(d^3)$ training samples, there is enough variation in the training data such that it is possible to generalize to the training data to a given target counterfactual treatment of interest.
In this section, we examine how much data is needed for the NSI estimator to get within $\varepsilon$ accuracy, using regular graphs as an illustrative setting.
For a $d$-regular graph, Proposition (ref) suggests that to achieve error of order $\varepsilon$ with high probability, NSI needs $T_\pre = \tilde{\Omega} \left( \frac{ r^{24} d^{40} }{\varepsilon^4} \right)$. To understand whether the sample complexity is reasonable, consider a naive alternative. For a given unit $n$, there are $D^{|\cN(n)|}$ possible counterfactual treatments that could be applied to the neighborhood $\cN(n)$. Suppose that we do not impose any structure on the potential outcomes, i.e., we do not assume (ref). Then, to learn how unit $n$ behaves under every possible neighborhood treatment, one would naively need at least one observation per $D^{|\cN(n)|} \leq D^{d+1}$ possible treatments. This naive approach would require $\Omega(D^d)$ samples in order to estimate the potential outcome for a given $n$ and any $\cfAn$ of interest. Moreover, to achieve an error of $\varepsilon$, at least $\frac{1}{\varepsilon^2}$ samples are needed per treatment, which implies $T_\pre = \Omega\left(\frac{D^d}{\varepsilon^2}\right)$ under a naive approach.
In this section, we present simulation results illustrating the behavior of the NSI estimator and compare it to two related estimators. Let $\cG$ be a regular graph with degree $d$, and let the treatments be binary, i.e., $D = 2$. In each of the experiments below, we indicate the graph degree. Let the training treatments be assigned according to the experiment design in Section (ref). Further experiments and details are given in Appendix (ref). \ifarxiv \\ \else \fi
Predictions. Note that the NSI estimator (ref) can be adapted to produce pointwise estimates $$\widehat{\bbE} \Big[ Y_{t, n}^{(\tilde{\ba}_{\cN(n)})} \Big] = Z[t, \cI^{(n)}] \hat{\bbE}[Z_{\pre, \cI^{(n)}} | \LF , A ]^+ \bz_{\pre, n} ,$$ such that $\hIPO{n} = \frac{1}{T_\post} \sum_{t \in \cT_\post} \widehat{\bbE} \big[ Y_{t, n}^{(\tilde{\ba}_{\cN(n)})} \big]$. Under this observation, Figure (ref)(a) shows an example of the pointwise estimates given by NSI for an example unit $n$. Consider the bottom plot. The solid line gives the ground truth potential outcomes for unit $n$ across measurements $t \in [200]$. The pointwise estimates produced by NSI are marked by asterisks $*$, with the 95 percent confidence interval in gray. The measurements to the left of the vertical line (i.e., in blue and green) correspond to the training set $\cT_{\pre}$ while those to the right (i.e., in red and orange) correspond to the prediction set $\cT_{\post}$. The top plot gives the spectrum $\{\hat{s}_\ell\}_{\ell=1}^{q}$ produced in Step 1 of Section (ref), where the vertical line marks the hard singular value threshold $\kappa$ that is used in Step 1. In Figure (ref)(a), $\cG$ is a ring graph ($d = 2$) with $N = 1000$ units, $\epsilon^{(\bc_{\cN(i)})}_{\tau, i} \sim \cN(0, 0.1)$, and $r = 2$.
As shown in the bottom plot of Figure (ref)(a), the predictions closely match the ground-truth values. As shown on top, $6$ components are used to construct the estimates. Since the network-adjusted rank is $6 $ ($r = 2$ and $|\cN(n)| = 3$), the fact that NSI uses $6$ components explains why its estimates are fairly accurate. We provide similar plots for other units and target treatments in Appendix (ref). \ifarxiv \\ \else \fi
Consistency and asymptotic normality. Figure (ref)(b) verifies that the NSI estimates are consistent and asymptotically normal. Specifically, we let $\cG$ be a ring graph (i.e., $d = 2$) with $N = 1000$ units, $\epsilon^{(\bc_{\cN(i)})}_{\tau, i} \sim \cN(0, 0.1)$, and $r = 2$. For each simulation, we randomly generate the potential outcomes of all units under (ref) (see Appendix (ref) for details). We run 500 simulations, then compute the NSI residuals $(\hIPO{n} - \IPO{n})$ for all units $n \in [N]$ and across all possible counterfactual treatments for each $n$. That is, we use NSI to estimate $\IPO{n}$ for $\tilde{\ba}_{\cN(n)} = (1, 0, 0)$, $\tilde{\ba}_{\cN(n)} = (0, 1, 0)$, $\tilde{\ba}_{\cN(n)} = (1, 1, 0)$, and so on. Figure (ref)(b) gives a histogram of the NSI residuals. A Gaussian distribution is fit to the residuals and given by the red line. \ifarxiv \\ \else \fi
MSE trends. Figure (ref)(c) summarizes the performance of NSI across different parameters. The performance is given by the mean-squared error (MSE) across the prediction measurements $\cT_{\post}$, averaged across $50$ units. Each group of bars gives the MSE for regular graphs of degree $2$, $4$, and $6$, as indicated on the $x$-axis. Within each group of bars, the left (blue) bars are for $N = 1000$, $T_{\pre} = 100$, and $T_{\post} = 50$; the middle (red) bars for $N = 1000$ and $T_{\pre} = T_{\post} = 50$; and the right (yellow) bars for $N = 500$ and $T_{\pre} = T_{\post} = 50$. Each bar is the average of $200$ simulations with $\epsilon^{(\bc_{\cN(i)})}_{\tau, i} \sim \cN(0, 0.1)$, and $r = 2$. As expected, the MSE increases with degree ({because, holding $N$ fixed, a higher degree leads to fewer valid donors}), fewer nodes ({which also leads to fewer valid donors}), and fewer training measurements. \ifarxiv \\ \else \fi
Comparing to other estimators. We also compare the NSI estimator to two others: the SI estimator agarwal2020synthetic and a baseline estimator. The SI estimator is similar to NSI, but SI assumes that there is no spillover and therefore does not account for network interference. The baseline estimator finds donor units that satisfy Definition (ref), then averages the donor units' observed outcomes. We compare the estimators for a ring graph (details given in Appendix (ref)). We compare the estimators for a ring graph under the same parameters as those used in Figure (ref)(b) averaging across $200$ simulations, $50$ units, and all possible counterfactual treatments.
The MSEs and R-squared values for the NSI estimator, SI estimator, and baseline estimators are, respectively, (0.1174, 0.8735), (0.2310, 0.8149), and (3.398, -2.957). Both the NSI and baseline estimators use donor sets that contain, on average, 41 units. The SI estimator uses donor sets with, on average, 166 units. As such, even though the SI estimator has more donors, the performance of NSI is better than that of SI, which is better than that of the baseline estimator.
There is rising interest in estimating unit-specific potential outcomes. In this work, we consider the estimation of unit-specific potential outcomes in the presence of spillover, i.e., the treatment assigned to one unit affects the outcome of another unit. We focus on the panel data setting and model spillover as network interference.
As our main contribution, we provide an estimator that we call Network Synthetic Interventions (NSI). In addition to producing point estimates, NSI provides confidence intervals. We show that, under a low-rank latent-factor model and suitable conditions, the NSI estimates are consistent and asymptotically normal. We provide two validity tests that determine whether key conditions hold. We find that obtaining good estimates under spillover requires that the data is rich enough. To this end, we provide an experiment design.
There are many paths for future work. Although the method that we provide comes with strong performance guarantees, it has strict data requirements, as discussed in Section (ref). One path for future work would be to explore whether these requirements can be relaxed. Second, we explore unit-specific potential outcome estimation. If such fine-grained estimates are not needed, one could examine whether NSI and its data requirements can be improved for coarser estimands of interest. Finally, one compelling path for future work would be to test NSI on real-world datasets.
The authors gratefully acknowledge funding from the MIT-IBM project on Causal Representation, the National Science Foundation (NSF) grant CNS-1955997, and the Air Force Research Laboratory (AFOSR) grant FA9550-23-1-0301.