Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
56,366 characters · 17 sections · 38 citation commands
Synthetic Potential Outcomes and Causal Mixture Identifiability
\twocolumn
Causality encompasses both counterfactual (what could have been) and hypothetical (what could be) statements. If a patient is given treatment and cured, saying that treatment caused recovery is equivalent to saying that the patient would not have recovered without treatment (a counterfactual). Similarly, recommending a treatment carries with it the implication that the patient is better off with treatment than without.
\paragraph{Latency of Counterfactuals} When hoping to recover causal effects, we are limited to data from a single world, revealing only one “potential outcome” (e.g. the patient recovers under treatment). As such, the counterfactual remains unobserved --- for all we know, the treatment had no effect and the patient would have recovered either way. If a treated patient's recovery is $Y^{(1)}$ and the untreated counterfactual is $Y^{(0)}$, then the individual treatment effect (ITE) is the difference between these “potential outcomes”: $Y^{(1)} - Y^{(0)}$. The inherent latency of counterfactual and hypothetical outcomes is referred to as the “fundamental problem of causal inference” imbens2015causal.
\paragraph{Exchangeability} In order to estimate causal effects, we must “pair” examples deemed “exchangeable.” For example, two twins given different treatments may be considered to be approximate counterfactuals of each other. When twins are not available, a common approach is to instead pair populations to identify an “average treatment effect” (ATE) $\mathbb{E}(Y^{(1)} - Y^{(0)})$. This can be achieved with a randomized control trial (RCT), which randomly assigns “treatment” and “control” groups to ensure exchangeability in expectation.
RCTs are not always feasible --- it is unethical to withhold potentially lifesaving medicine from sick patients, and many economic policies must be implemented without a preliminary test. In such settings, methods for causal inference have developed across a number of fields pearl2009causality, imbens2015causal, peters2017elements.
\paragraph{Refining Exchangeability} Approaches to causal inference all rely on the insight that two individuals (or groups) need not be identical in every way to provide access to a counterfactual. Epidemiological methods often make use of assumptions like “unconfoundedness” (also called strong ignorability), which is formalized as $Y^{(1)},Y^{(0)} \perp \!\!\! \perp T \mid \mathbf{X}$ for treatment $T$ and covariates $\mathbf{X}$ rosenbaum1983central. This assumption implies that no unobserved $U$ confounds both $T$ and $Y$, implicitly guaranteeing that two data entries with identical $\mathbf{X} = \mathbf{x}$ and different $T=t$ are sufficiently exchangeable to be considered counterfactuals.
Relaxing unconfoundedness, graphical methods can be used to model the relationships between covariates lauritzen1996graphical, koller2009probabilistic. Such models define “adjustment sets” of covariates that are sufficient for exchangeability pearl2009causality. These approaches allow for exchangeability only with respect to a subset of $\mathbf{X}$ --- something that is needed if $\mathbf{X}$ contains a common effect of $T, Y$ (e.g. $T \rightarrow X \leftarrow Y$). However, they still cannot handle unobserved confounding due to $U$ unless a valid adjustment set can fully resolve its effect on the ATE.
\paragraph{Identifiability} Tools for causal inference are often employed when there is no ground truth validation for success. Therefore it is essential to guarantee identifiability of the desired quantity, i.e. that infinite data from a distribution will uniquely correspond to a single answer. The work of Judea Pearl has heavily explored the identification of ATEs using adjustment sets that are given with respect to graphically modeled unobserved confounding pearl2009causality. This approach links causal identifiability to the existence of an observable metric of exchangeability.
The principal culprit for entanglement between causation and correlation is latent heterogeneity. In the absence of an observable adjustment set, it is impossible to pair exchangeable points or distributions. For example, if groups with differing rates of recovery are also given treatment at different rates, then the apparent treatment effect will be biased. Treatment is often given to the most severe cases of a disease, resulting in treated patients being associated with poorer outcomes despite a positive treatment response. This is known as Simpson's Paradox simpson1951interpretation.
\paragraph{Formal Setting} To study this setting rigorously, we consider a $k$-mixture to be given by:
We will generally assume that $U$ has nontrivial causal arrows to all four $\{T, Y, \mathbf{X}, \mathbf{Z}\}$. These “nontrivial arrows” imply an “overlap” or “positivity” assumption of $0 < \Pr(T \mid U) < 1$. We restrict ourselves to an “unconfounded” setting where $Y^{t} \perp \!\!\! \perp T \mid U$ for simplicity while noting that extending to adjustment sets of the form $\mathbf{C} \cup \{U\}$ for observable $\mathbf{C}$ is straightforward. In addition, we will assume $\mathbf{X}, \mathbf{Z}$ can be transformed into feature maps $\mathbf{\Phi}(\mathbf X), \mathbf{\Theta}(\mathbf Z) \in \mathbb{R}^k$ with distinct expectations within each $U$. That is $\mathbb{E}[\Phi \mid u] \neq \mathbb{E}[\Phi \mid u']$ whenever $u \neq u'$ for all $\Phi \in \mathbf{\Phi}(\mathbf X)$ (and similarly for $\mathbf{\Theta}$). To simplify the presentation of our algorithm, we will assume $\mathbf{X}, \mathbf{Z} \in \mathbb{R}^k$ instead of directly referring to their feature maps. The requirement of “distinctness” manifests as the invertibility of a matrix used in our algorithm, which is empirically verifiable.
We define the “mixture of treatment effects” (MTEs) problem to be the task of uniquely identifying conditional average treatment effects on a latent class $U$, e.g. $\mathbb{E}(Y^{(1)} - Y^{(0)} \mid u)$, in addition to the probability of each class $\Pr(u)$.
To illustrate the importance of identifying causal heterogeneity, consider two possible populations of a bacterial infection. One consists of a sub-population that is resistant to an antibiotic, as well as a non-resistant sub-population. Another bacterial population is homogeneously partially resistant to the antibiotic, requiring a higher dosage to be killed. In the first setting, use of the antibiotic is ill-advised, as it will allow the resistant subpopulation to dominate. In the second setting, a larger dose is appropriate.
A similar problem emerges within virus vaccination. A vaccine may lose average effectiveness over time due to a loss in antibodies (a homogeneous loss in potency) or the emergence of a new resistant variant (a heterogeneous response). In this setting, the homogeneous response should be combated with a booster, while the heterogeneous response requires a new vaccine.
A final motivating task is that of evaluating the success of an intervention's implementation. For example, imperfect participation or a faulty shipment may create a heterogeneous response to a treatment, while general ineffectiveness may be homogeneous.
There are multiple notions of identifiability within causally heterogeneous settings. The first is that of the heterogeneous treatment effects (HTEs) with respect to a subset $\mathbf{A}'$ of adjustment set $\mathbf{A}$, which is only identifiable if $\mathbf{A}'$ is observable. In principal, this means that each entry in a dataset can grouped according to its conditional average treatment effect (CATE) indexed by a set of values for $\mathbf{A}'$. In our setup, $\mathbf{A}' = \mathbf{A} = \{U\}$, and $U \in \mathbf{A}$ regardless of the causal structure on $(\mathbf Z, T, Y, \mathbf X)$.
In contrast to the endogenous and directly observed heterogeneity of HTEs, the causal effects within MTEs are exogenously and latently heterogeneous. The primary computational challenge when recovering MTEs is the latency of $U$, which disallows the direct observation of CATEs with respect to $U$. Mixture models are considered identifiable when observed statistics map uniquely to the parameters of the model, i.e. $\Pr(U), \Pr(T, Y, \mathbf X, \mathbf Z \mid u)$, up to the $k!$ symmetric permutations of $U$. If a mixture model is identified, MTEs can be recovered with respect to adjustment sets including unobserved $U$, even though such treatment effects cannot be assigned to individual entries of data.
At an even coarser resolution than MTEs, ATEs are considered identifiable when observed statistics map uniquely to the average causal effects, e.g. $\mathbb{E}(Y^{(1)} - Y^{(0)})$, which marginalizes out the effect of $U$.
These notions of identifiability are hierarchical. Clearly, datapoints that have been grouped by HTE can be separately analyzed to recover the CATEs needed for MTEs. Furthermore, MTEs can be marginalized into an ATE using the law of total probability. Meanwhile, negative controls miao2018confounding identify ATEs without giving access to MTEs or HTEs, and mixture models can identify MTEs without giving access to point-specific mixture membership. It is essential to develop methods at the level of the scientific goal --- using a method that identifies a “more coarse” level can lead to uncertainty in results and using methods that are “more fine” have more restrictive identifiability requirements.
Particular interest has been given to mixture model methods as tools for causal inference because they give access to the joint probability distribution between $U$ and $T,Y$, allowing the identification of both ATE and MTEs. However, two mixture models can easily generate identical ATEs. While less obvious, two mixture models can also generate identical MTEs, motivating an additional level of identifiability.
Mixture model identifiability does not precisely correspond to MTE identifiability. That is, two different distributions for $\Pr(\mathbf Z, T, Y, X \mid u)$ could map to the same MTEs. In particular, correctly identifying the distribution on the proxies $X, Z$ is inconsequential for recovering the conditional average treatment effects with respect to $U$. For this reason, the conditions for MTE identifiability are milder than those for mixture identification. We present a straightforward differentiation between these conditions as a hierarchy on four levels:
The four notions of identifiability require decreasingly restrictive assumptions, which can be interpreted graphically as adding possible edges to the $5$-vertex graph shown in Figure (ref). The full requirements for identifiability at levels 3 and 4 are formally given in Theorems (ref) and (ref) after introducing some notation \footnote{The independence conditions needed for identification can be relaxed into a notion of exchangeability in expectation, since all that is required is for expectations to factorize. This will be discussed in more detail in Section (ref).}.
Identification at levels 2 and 4 are addressed for categorical (discrete) $X, Z$ in previous works discussed in Section (ref), which we expand to continous $X, Z$ as a secondary result. Our primary contribution is to give the first identifiability result for level 3, including an algorithm called “synthetic potential outcomes” (SPOs) that we develop in Section (ref). Section (ref) then gives counter counterexamples that show distinctness for each level illustrated in Figure (ref).
Whenever $U$ is not fully determined by $\mathbf{Z}, \mathbf{X}$\footnote{Formally this is described as positivity, i.e. $\Pr(u \mid \mathbf{z}, \mathbf{x}>0$}, the observed supports of the latent classes overlap. A consequence of this overlap is that identical entries can belong to two different classes. Hence, covariates are insufficient to assign individuals to MTE groups with perfect accuracy. This introduces class-misspecification biases loh2022evaluating in addition to invalidating nearest-neighbor approaches suk2021hybridizing. To handle this challenge, we develop SPOs as a method of moments.
\paragraph{Ladder of Causality} pearl2009causality also gives a causal hierarchy: counterfactuals, interventions, associations. We further refine the space in between counterfactuals and interventions: HTEs correspond to counterfactuals that are indexed with respect to a known SCM, and ATEs correspond to interventions.
\paragraph{HTEs} Recent work has studied HTEs for non-overlapping (i.e. qualitatively different) classes xie2012estimating, wendling2018comparing, kunzel2019causaltoolbox, imai2011estimation. Popular approaches include causal forests wager2018estimation and two-step algorithms generalized by nie2021quasi.
\paragraph{MTEs} MTEs are latent classes defined by heterogeneous treatment effects that overlap in their observed covariates. Early attempts involve learning distinct parameters per latent class lyu2023estimating, kim2015mixture, kim2015multilevel, suk2021hybridizing. All of these approaches make use of the unconfoundedness assumption, meaning they do not simultaneously deconfound the treatment effects that they recover (i.e., the latent class may affect outcome, but not treatment). Our approach combines the task of recovering and deconfounding heterogeneous treatment effects.
\paragraph{Proximal Causal Inference} Methods for proximal causal inference deconfound ATEs (but not MTEs) using “negative controls” that may depend on the unobserved confounder, but do not influence (at least one of) treatment or outcome tchetgen2020introduction. The majority of work has focused on a continuous unobserved confounder miao2018confounding,tchetgen2020introduction,mastouri2021proximal.
Proximal causal inference and negative controls have demonstrated effective application of the DAG setup from Figure (ref) level 4, where $Z$ is a negative control effect and $X$ is a negative control outcome. See shi2020selective for a review as well as negativecontrolcovid for a recent application to COVID-19 effectiveness. Our method of SPOs can be thought of as a generalization of miao2018identifying to joint probabilities instead of conditionals, thereby allowing for continuous covariates. By providing insight into how negative controls synthetically copy potential outcomes, we also give a natural extension to MTEs.
\paragraph{Mixtures} Latent variable methods have been employed to uncover the joint probability distribution between the latent class $U$, treatment $T$, and outcome $Y$. This approach was popularized by wang2019blessings, though criticized by ogburn2019comment, ogburn2020counterexamples for not guaranteeing identifiability.
Identifiability of discrete mixture models has been studied extensively. In contrast to clustering approaches, this line of works uses the method of moments to handle overlapping probability densities. The primary focus has been on “mixtures of products,” in which the observable variables are conditionally independent from each other when the latent $U$ is held constant. This problem can be thought of as a decomposition into rank 1 tensors, giving rise to a number of tensor-methods spurred by the seminal work of allman2009identifiability and later followed up in anandkumar2014tensor. Outside of tensor methods, the current best\footnote{in sample and time complexity} algorithm with provable guarantees is developed in gordon2023identification. gordon2023causal recently generalized this setting to arbitrary DAGs.
\paragraph{Tensor and Matrix Completion} SPOs are similar in concept to synthetic controls abadie2010synthetic,abadie2021using, synthetic interventions agarwal2020synthetic,squires2022causal, and synthetic nearest-neighbors agarwal2023causal. However, these approaches use linear combinations of data entries, whereas we use linear combinations of features. We also use higher-order moments and do not attempt to recover any unit-specific information.
\paragraph{Random Variables versus Assignments} We will use the capital Roman alphabet to denote random variables and boldface to denote sets of these random variables. Corresponding lowercase denotes assignment to those random variables. To shorten notation, $X = x$ is sometimes just written as $x$, e.g. $\Pr(y \mid x)$.
\paragraph{Vectors of Probabilities and Expectations} Whenever a quantity is left ambiguous in an expectation or probability, i.e. a boldface $\mathbf{X}, \mathbf{Z}$ or a capital $U$, we will first expand it into a column vector and then a matrix if a second ambiguity is given. That is,
$\mathbf{E}[\mathbf{X}Y]$ is a column vector of second-order multilinear moments, while $\mathbf{E}[ \mathbf{X}, \mathbf{Z}] \in \mathbb{R}^{\abs{\mathbf{X}} \times \abs{\mathbf{Z}}}$ expands both sets of variables as a matrix:
We will also give vectors of conditional expectations,
$\mathbf{E}[\mathbf{X} \mid U]$ is a matrix with columns of $\mathbf{E}[\mathbf{X} \mid u]$, given by
\paragraph{Observable Moments} Not all of our $\mathbf{E}[\cdot]$ vectors and matrices can be estimated directly by observable statistics. More specifically, we cannot estimate any vector or matrix of probabilities that involves $U$. To emphasize which matrices are observable, we will replace $\mathbf{E}$ with $\mathbf{M}$ for “observable moments.” We will similarly replace scalar $\mathbb{E}(\cdot)$ with $M(\cdot)$.
$\mathbf{M}[\cdot]$ vectors can be decomposed as inner products with $\mathbf{E}[\cdot \mid U]$ vectors and $\mathbf{P}[U]$. That is,
When $\mathbf{X} \perp \!\!\! \perp \mathbf{Z} \mid U$, we have a similar decomposition for second-order moment matrices:
Algorithmic computations will be restricted to using $\mathbf{M}[\cdot]$s. To prove correctness and analyze stability, we will decompose into $\mathbf{E}[\cdot]$ and $\mathbf{P}[\cdot]$, which exist hypothetically but cannot be accessed directly from observed statistics.
\paragraph{Potential Outcomes} To understand the difference between intervening and conditioning on $T$, compare the following quantities:
The crucial difference is that intervening on $T=t$ does change the right column vector. $\mathbb{E}(Y \mid t)$ is an observable moment that is estimated by counting $Y=1$ when conditioning on $T=t$. $\mathbb{E}(Y^{(t)})$, in contrast, is not given by an observable moment because we cannot sample from the marginal $U$ while simultaneously drawing from the conditional $Y \mid U, t$. This is due to confounding, which shifts the distribution of $U$ between the treatment ($T=1$) and control ($T=0$) groups.
For ATEs, our goal is to recover $\mathbb{E}(Y^{(1)} - Y^{(0)})$. We will simplify this using $R \coloneq Y^{(1)} - Y^{(0)}$ for the “response.” For MTEs, we must recover $k$ CATEs, i.e. $\mathbf{E}[R \mid U]$, as well as $\mathbf{P}[U]$. We present SPOs as a method for identifying ATEs and MTEs.
For both theorems, conditions (i) and (ii) correspond to the graphs given in Figure (ref). Notice that $\mathbf{Z} \perp \!\!\! \perp \mathbf{X} \mid U$ is implied in both cases if no other variables are present (i.e. no other active paths).
If we could sample from $Y^{(t)}$ without conditioning on $T$, we could access the expected value as an observable moment. Notice that if $\mathbf{X} \perp \!\!\! \perp T \mid U$, then $\mathbb{E}[\mathbf{X} \mid U, t] = \mathbb{E}[\mathbf{X} \mid U]$. Our approach will rely on the observation that $\mathbf{E}[Y^{(t)} \mid U]$ is a $k$-dimensional vector that can be expressed as a linear combination of the rows of $\mathbf{E}[\mathbf{X} \mid U, t]$ provided that the matrix is full-rank, which will be guaranteed by $\mathbf{M}[\mathbf{Z}, \mathbf{X} \mid t]$ being full rank due to its decomposition in Equation (ref). Let this linear combination be given by coefficients $\boldsymbol{\alpha}$,
If $\mathbf{X} \perp \!\!\! \perp T \mid U$, the same $\boldsymbol{\alpha}$ can be applied to the unconditioned moments of $\mathbf{X}$ to access a “synthetic potential outcome”:
In the next section, we show how to find $\boldsymbol{\alpha}$.
We want to find $\boldsymbol{\alpha} = (\alpha_1, \ldots, \alpha_k)^\top$ such that, using unconfoundedness,
We begin by recalling that,
Notice that the left row vector in this inner product is precisely what we want to “copy”. To compute $\boldsymbol{\alpha}$, we can set up a system of linear equations that matches second-order moments with $\mathbf{Z}$:
To see how this moment-matching approach gives rise to a system of linear equations, apply the expansion given by Equation (ref) to $\mathbf{M}[\mathbf Z, \mathbf X]$ on both sides (requires $\mathbf{Z} \perp \!\!\! \perp \mathbf{X} \mid U)$, and expanding $\mathbf{M}[\mathbf{Z}Y \mid t]$ similarly using condition (i) from both theorems.
The average treatment effect is given by the difference between the moments of the potential outcomes,
Both moments are accessible using SPO coefficients
The superscript $1$ is used to signify that the coefficients are used to compute a first order-moment, as we will be generalizing this procedure shortly. We use $\boldsymbol{\gamma}^{(1)} = \boldsymbol{\alpha}^{(1)} - \boldsymbol{\beta}^{(1)}$ to find the ATE,
We will denote the $\ell$th order moment as
with $\odot \ell$ indicating an element-wise exponent. This quantity is different from $\mathbb{E}(R^{\ell})$, instead being interpreted as the result of choosing $U=u$ and then multiplying $\ell$ i.i.d. samples of $R \mid u$.
Information about mixed treatment effects is contained in these higher-order moments of $R$. For example, a homogeneous treatment effect of $0$ will give $\mathbb{E}(R^{\odot 2}) = \mathbb{E}(R) = 0$, whereas a mixed treatment effect of half $+\nicefrac{1}{2}$ and half $-\nicefrac{1}{2}$ will give $\mathbb{E}(R^{\odot 2}) = \nicefrac{1}{4}$, despite an identical $\mathbb{E}(R) = 0$.
Using $\odot$ to denote element-wise multiplication, we slightly abuse notation to define
We can then expand the $\ell$th order moment as
Hence, our goal is now to estimate $\mathbb{E}(Y^{(t)} R^{\odot \ell - 1})$ the same way we estimated $\mathbb{E}(Y^{(t)})$.
An additional requirement of $Y \perp \!\!\! \perp \mathbf{X} \mid U$ means that
We will compute $\boldsymbol{\gamma}^{(\ell)}$ when $\boldsymbol{\gamma}^{(\ell-1)}$ has already been computed (with the base case $\boldsymbol{\gamma}^{(1)}$ already covered for ATE recovery). Hence, we can assume that $\mathbf{X}$ can already synthetically “copy” $R^{\odot \ell-1}$. We now rewrite Equation (ref) using $\boldsymbol{\gamma}^{(\ell-1)}$ and second-order moments between $X_i$ and $Y$:
We again moment-match with $\mathbf{Z}$ to get
Once we have computed $\boldsymbol{\gamma}^{(\ell)} = \boldsymbol{\alpha}^{(\ell)} - \boldsymbol{\beta}^{(\ell)}$, we can estimate the $\ell$th moment of the treatment effect,
Using these higher-order moments to recover a multiplicity of treatment effects reduces to the well-studied “sparse Hausdorff moment problem,” which seeks to recover $\mathbb{E}[R \mid U]$ and $\mathbf{P}[U]$. Notice that this new mixture model on $R$ only contains the parameters of interest and is therefore no longer beholden to the correct identification of $\Pr(\mathbf{X}, \mathbf{Z} \mid U)$.
Identifiability of a mixture of $k$ treatment effects requires $2k-1$ moments and can be calculated using Prony's method de1795essai, as illustrated and analyzed in gordon2020sparse, or the matrix pencil method hua1990matrix, which is outlined in Section 2.2 of kim2019many. The full algorithm pseudocode is given in Appendix (ref). Runtime and sample complexity analysis are detailed in Appendix (ref)
We now give an in-depth discussion on the identifiability of each level of our hierarchy and why these levels are distinct. Having just discussed the identifiability of levels 3 and 4, we will briefly explain the identifiability of levels 1 and 2 with slight modifications of previous results. By nature of the hierarchy, a solution for a “harder” level can be used to obtain a solution for an “easier” level. We will also show the distinctness of each of these levels by giving two different models that resolve as two different “answers” for harder levels, but the same “answer” for easier levels.
To simplify things, all of our models will use Bernoulli $U, Z, X, T$, and $Y$, with $\mathbf P[U] = [\nicefrac{1}{2}, \nicefrac{1}{2}]$, and
The features $\mathbf{X}$ are therefore $X_1 = \mathbbm{1}(X=0)$ and $X_2 = \mathbbm{1}(X=1)$ and likewise for $\mathbf{Z}$. The resulting matrix $\mathbf M[\mathbf Z, \mathbf X]$ is full rank as required by Theorem (ref).
\paragraph{Level 1: HTEs} Identifiability of a CATE with respect to an observed adjustment set is perhaps the most well-studied of the levels. In our setup, we have
This allows us to compute the CATE as $\mathbb{E}(Y^{(1)} \mid u, \mathbf x, \mathbf z) - \mathbb{E}(Y^{(0)} \mid u, \mathbf x, \mathbf z)$, which can be assigned to all data points with $U=u, \mathbf X = \mathbf x, \mathbf Z = \mathbf z$ when $U$ is observed.
\paragraph{Level 2: Mixtures of Products} When $U$ is not observed, CATEs cannot be assigned to individual points. For example, if $(\mathbf{Z}, T, Y, \mathbf{X})$ are all discrete, then a strictly positive distribution will give nonzero probabilities for $\Pr(\mathbf z, t, y, \mathbf{x} \mid U=1)$ and $\Pr(\mathbf z, t, y, \mathbf{x} \mid U=0)$. If the $U=0$ and $U=1$ components exhibit different causal effects, then we can arbitrarily assign all $n$ points with covariates $(\mathbf z, \mathbf x)$ to classes $U=0$ and $U=1$ so long as we obey the correct ratio of $\Pr(U=0) : \Pr(U=1)$ in our labeling. Hence, any mixture component overlap makes HTE identifiability at the individual level impossible.
Despite this, under some circumstances, the conditional probability distributions $\Pr(\mathbf{Z}, T, Y, \mathbf{X} \mid U)$ and $\Pr(U)$ are still identifiable. One such circumstance is when $\mathbf Z$, $\mathbf S$, and $\mathbf X$ are conditionally independent given $U$, where $\mathbf S = (T, Y)$; allman2009identifiability showed this case to be identifiable, up to symmetries in permuting the labels of $U$, under certain algebraic assumptions on the conditional distribution matrices $\mathbf P[\mathbf Z \mid U]$, $\mathbf P[\mathbf S \mid U]$, and $\mathbf P[\mathbf X \mid U]$. As we show in Appendix (ref), this result can be extended to include the continuous setting by using moments instead of probabilities. After recovering the within-component conditional probabilities, $\Pr(\mathbf{Z}, T, Y, \mathbf{X} \mid U)$, we can also recover $\mathbb{E}(Y^{(t)} \mid u) = \mathbb{E}(Y \mid t, u)$.
\paragraph{Level 3: MTEs} SPOs identify MTEs under the conditions given in Theorem (ref). We give two models with invertible moments that identify the same MTE, but represent different mixture models that are not distinguishable at level 2. In particular, we define two parameterized families:
First, for ${\textcolor{blue}{\mu_{zt}}} \in [0, 1]$, we define
Second, for ${\textcolor{red}{\mu_{xy}}} \in [0, 1]$, we define
Thus, for $R = Y^{(1)} - Y^{(0)}$,
The value of ${\textcolor{blue}{\mu_{zt}}}$ controls how treatment assignment depends on $\mathbf Z$ and $U$, whereas the value of ${\textcolor{red}{\mu_{xy}}}$ controls how treatment effect depends on $\mathbf X$ and $U$.
We define the following two models:
In both models, since ${\textcolor{red}{\mu_{xy}}} = 0$, we have $Y \perp \!\!\! \perp X \mid U$, and $\mathbf E[R \mid U] = [-\nicefrac{3}{4}, \nicefrac{3}{4}]$, i.e., the two models have the same mixture of treatment effects. However, for Model A, since ${\textcolor{blue}{\mu_{zt}}} = 0$, we have $T \perp \!\!\! \perp Z \mid U$. Meanwhile, in Model B, since ${\textcolor{blue}{\mu_{zt}}} = 1$, we have $T \perp \!\!\! \perp U \mid Z$. Thus, these two models trade off whether heterogeneity in treatment assignment is driven by $U$ or by {\color{blue} $Z \to T$}, and represent two different mixture models.
\paragraph{Level 4: ATEs} SPOs identify ATEs under the conditions given in Theorem (ref). We define a third model:
In this model, $Y \perp \!\!\! \perp U \mid X$, with $\mathbf E [R \mid X] = [-\nicefrac{3}{4}, \nicefrac{3}{4}]$. A quick calculation shows that $\mathbf E[R \mid U] = [-\nicefrac{3}{8}, \nicefrac{3}{8}]$. Hence, Model C\ has a different mixture of treatment effects than Model B, but both models have the same ATE, $\mathbb{E}(R) = 0$. Thus, these two models trade off whether heterogeneity in treatment effect is driven completely by $U$, or completely by {\color{red} $Y \leftarrow X$}.
The previous section introduces a model in which the strength of ${\color{blue}Z \leftrightarrow T}$ and ${\color{red} X \rightarrow Y}$ are controlled by ${\textcolor{blue}{\mu_{zt}}}$ and ${\textcolor{red}{\mu_{xy}}}$. Recall that full mixture recovery (level 2) is identifiable when neither ${\color{blue}Z \leftrightarrow T}$ nor ${\color{red} X \rightarrow Y}$ exist, MTEs (level 3) are identifiable with ${\color{blue}Z \leftrightarrow T}$ but no ${\color{red} X \rightarrow Y}$, and ATEs are identifiable with both ${\color{blue}Z \leftrightarrow T}$ and ${\color{red} X \rightarrow Y}$.
Our empirical results explore the transition between identifiability with respect to ${\textcolor{blue}{\mu_{zt}}}$ and ${\textcolor{red}{\mu_{xy}}}$, which gradually shift us between the regimes of identifiability for these levels. Each level is computed and evaluated using an algorithm and loss function tailored to the task:
In all experiments, we report results averaged over 100 runs for each parameter value. Each single run consists of taking 1,000 samples and running each method method on the empirical moments. An anonymized repository containing a Python implementation of our SPO method and code for replicating our experiments can be found \href{https://anonymous.4open.science/r/synthetic-potential-outcomes-submission-E900/README.md}{\textcolor{magenta}{here}}. For mixture recovery, we use the PARAFAC algorithm in tensorly tensorly to decompose the empirical probability distribution into factors, then normalize the factors into probability distributions $\widehat{\Pr}(U)$, $\widehat{\Pr}(Z \mid U)$, $\widehat{\Pr}(X \mid U)$ and $\widehat{\Pr}(Y, T \mid U)$.
\paragraph{Level 2 vs. Level 3} First, we demonstrate the separation between Level 2 and Level 3 by varying the parameter ${\textcolor{blue}{\mu_{zt}}}$ from 0 to 1 while keeping ${\textcolor{red}{\mu_{xy}}} = 0$, i.e., we interpolate from Model A\ to Model B. As shown in Fig. (ref), our method of SPOs accurately recovers the mixture of treatment effects for all values of ${\textcolor{blue}{\mu_{zt}}}$. In contrast, when we perform tensor decomposition, the performance of mixture recovery becomes significantly worse as ${\textcolor{blue}{\mu_{zt}}}$ becomes larger.
\paragraph{Level 3 vs. Level 4}
Next, we demonstrate the separation between Level 3 and Level 4 by varying the parameter ${\textcolor{red}{\mu_{xy}}}$ from 0 to 1 while keeping ${\textcolor{blue}{\mu_{zt}}} = 1$, i.e., we interpolate from Model B\ to Model C. As shown in Fig. (ref), our method accurately recovers the average treatment effect for all values of ${\textcolor{red}{\mu_{xy}}}$; however, as expected, the performance of MTE recovery becomes significantly worse as ${\textcolor{blue}{\mu_{zt}}}$ becomes larger.
\paragraph{Relaxing Assumptions for Identifiability} The independence conditions given in Theorem (ref) and Theorem (ref) have a nice graphical correspondence in Figure (ref), but can be relaxed to exchangeability with respect to expectations\footnote{Thank you to the anonymous reviewer who pointed this out to us.}. To understand this, notice that the decomposition given in Equation (ref) only demands $\mathbb{E}[XZ]=\mathbb{E}[X \mid U]\mathbb{E}[Z \mid U]$ for all $Z \in \mathbf{Z}, X \in \mathbf{X}$. This is a milder condition than $\mathbf{X} \perp \!\!\! \perp \mathbf{Z} \mid U$. As a result, Theorem (ref) only requires
for ATE identifiability with SPOs. Similarly, Theorem (ref) only requires
for MTE identifiability using SPOs.
\paragraph{The Importance of Identifiability} In the absence of validation sets, identifiability is essential. However, it is important to study identifiability at the correct granularity --- too coarse and we limit the information we recover, too fine and we limit the application of the approach. Empirical results show that the identifiability transitions are “soft,” meaning that loosely violated requirements can still give accurate results. In Appendix (ref), we give sample complexity results for SPOs relative to the condition number of $\mathbf{M}[\mathbf{X}, \mathbf{Z}]$. Future studies of sample complexity may further resolve stability under mildly violated assumptions.
\paragraph{Mechanism-based Classes} In this paper, we argue that MTEs represent a fundamental granularity worth studying and showed how their identifiability fits within the more well-studied frameworks and problems. MTEs are classes that are differentiated by their mechanisms, which are ensemble-level properties. Covariate values are often insufficient to differentiate relationships between observables, leading to distribution overlap that many clustering approaches cannot appropriately address. It is therefore essential to consider parametric approaches and methods of moments when developing algorithms. These approaches are not only advantageous because of their increased power of identification (as demonstrated by the gap between level 1 and level 2), but also more private, as individuals are never assigned class memberships.
\paragraph{Covariate Richness} Finding a “sufficiently-rich” set of features $\mathbf{\Phi}(\mathbf X), \mathbf{\Theta}(\mathbf Z)$ is left as an open problem. Many results in proximal causal inference assume that the negative controls are rich categorical variables, e.g. alphabets with cardinality $k$. This can be interpreted with respect to algebraic geometry. While additional parameters usually increase the difficulty of identification, more parameters within $\mathbf X, \mathbf Z$ give more degrees of freedom along which these random variables can differ between different classes. Each of these degrees of freedom can be made into a feature, which gives us access to more moments. Because we look at second-order moments between $\mathbf{Z}, \mathbf{X}$, a linear increase in parameters corresponds to a quadratic increase in the moments used to recover those parameters.
\paragraph{Heterogeneity Detection} Our approach is limited by the inherent instability of mixture models. As such, future work should develop statistical tests for the presence of (significantly different) MTEs, as in pearl2022detecting, rather than guaranteeing full mixture recovery. This problem likely makes up an additional layer of identifiability. Such tests could be used for quality control (e.g. detecting an ineffective batch of medication), quantification of uncertainty in treatment response, and the detection of vaccine-resistant viral variants.
\onecolumn