Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
73,750 characters · 26 sections · 127 citation commands
Asymptotically Unbiased Synthetic Control Methods by Moment Matching
Synthetic control methods AbadieGardeazabal2003,AbadieDiamondHainmueller2010 have emerged as a crucial tool for program evaluation in comparative case studies across disciplines such as economics, statistics, and machine learning. In SCMs, there are multiple units, where one unit receives a policy intervention (treated unit) at a certain time while the remaining unitts remain untreated. The objective is to estimate the treatment effect for the treated unit, defined as the difference between its observed factual outcome and its unobserved counterfactual outcome. Untreated units help account for unobserved trends in the outcome over time that are unrelated to the policy intervention. The key idea is that an optimally weighted combination of the untreated units, referred to as the synthetic control (SC) unit, serves as an appropriate estimate of the counterfactual outcome. See Figure (ref) for an illustration.
SCMs have been widely applied in various empirical studies, such as evaluating the impact of terrorism on GDP AbadieGardeazabal2003 and the decriminalization of indoor prostitution Cunningham2017. For instance, AbadieDiamondHainmueller2010 applies SCM to study the treatment effect of a comprehensive tobacco control program implemented in California in 1988. While smoking rates in California declined following the program’s implementation, it remained unclear whether this decrease was attributable to the policy. For estimating the treatment effect, annual per-capita cigarette sales data from states that did not implement a similar tobacco control program were used as untreated units. SCM was then applied to construct California’s counterfactual outcome, representing the smoking rate in the absence of the tobacco control program.
A key challenge in SCMs is the assumption, made in most existing studies Abadie2002, that the expected outcome of the treated unit can be represented as a weighted sum of the outcomes of untreated units. Ferman2021 highlights that this assumption leads to a non-negligible bias in the estimator. According to Ferman2021, this bias arises from endogeneity in the linear regression model implied by the SCM assumption due to the correlation between the outcomes of untreated units and the error term. This issue is analogous to the endogeneity caused by measurement error Greene2003Econometric. We refer to this problem as implicit endogeneity.
To address this issue, we propose an alternative approach based on a mixture model assumption rather than a linear relationship among expected outcomes. Specifically, we assume that the distribution of the treated unit’s outcome can be approximated by a weighted combination of the distributions of the untreated units. This assumption further implies that each moment of the treated unit’s outcome can be represented as a weighted sum of the moments of the untreated units. Leveraging this insight, we estimate SC weights by matching the moments of the treated unit’s outcome to those of the untreated units. We refer to our method as the Moment Matching SCM (MMSCM). Unlike existing methods, our proposed estimator is asymptotically unbiased if the mixture model assumption is correct. We validate the effectiveness of our approach through both simulation studies and empirical analysis.
The mixture model assumption has appeared in previous and concurrent studies, including Wan2018, Gunsilius2020, and Shi2022. Our proposed method can be viewed as a simplification of the distributional SCMs introduced by Gunsilius2020. While Gunsilius2020 does not explicitly address the implicit endogeneity problem, they note that their results diverge from those obtained under standard SCMs. This discrepancy arises because, under the mixture model assumption, Gunsilius2020 provides a consistent estimator for the treatment effect, whereas existing SCM methods do not. We establish a connection between distributional SCMs and the asymptotic bias problem and propose a simplified estimation method based on moment matching.
In summary, our contributions are as follows:
We formulate the problem in Section (ref). Next, in Section (ref), we review the basic idea of SCMs. In Section (ref), we highlight the problem of implicit endogeneity. In Section (ref), we then propose our method under the mixture model assumption. We investigate the asymptotic properties of this method in Section (ref) and present a statistical inference method in Section (ref). In Section (ref), we discuss related topics and existing studies. Finally, in Section (ref), we examine the performance of our estimators through extensive simulations.
Related work. Following the pioneering work by AbadieGardeazabal2003, numerous studies have advanced the use of SCMs in comparative studies AbadieDiamondHainmueller2010,AbadieDiamondHainmueller2015. These SCMs, including their variants such as those by Doudchenko2016 and Ben2019, are based on minimizing the pre-treatment sum of squared errors between the outcome of the treated unit and the weighted sum of those of the untreated units, with some regularization and constraints regarding the weights. We refer to such SCMs as Least-Squares SCMs (LS-SCMs).
The bias of the LS-SCMs strongly depends on whether there exist weights that can predict the outcome of the treated unit without error. In other words, a random outcome of the treated unit must be replicable by the sum of outcomes of untreated units. Since such weights yield a perfect fit in the estimation process using observations from the pre-treatment period, this situation is called pretreatment-fit.
AbadieDiamondHainmueller2010 demonstrates that under a pre-treatment fit, the bias of the LS-SCM estimator converges to zero asymptotically as the length of the pre-treatment periods $T_0$ increases. However, Ferman2021 finds that if the pre-treatment fit does not hold, then the LS-SCM estimator has a bias and is not consistent. The cause of this bias is the correlation between the outcomes of the untreated units and the error term in the posited linear model, which we refer to as implicit endogeneity.
Several methods have been proposed to address this bias issue. Ferman2021OntheProp demonstrates that we can obtain an asymptotically unbiased estimator under the regime where both the number of untreated units ($J \to \infty$) and the length of the pre-intervention periods ($T_0$) grow large. Concurrently with us, fry2024method also discusses the endogeneity problem in SCMs as a cause of the asymptotic bias reported by Ferman2021.\footnote{Note that his study became public on arXiv after our study became public on arXiv and at the ICML workshop, and he and we conducted our studies completely independently.} While our formulation and his regarding endogeneity are similar, our approach differs significantly. We address the problem through moment matching, similar to the strategy in Gunsilius2020, whereas fry2024method employs an instrumental variable approach using outcomes of untreated units not used in SCMs as instruments.
In this study, we focus on mixture models among outcome distributions, an assumption also employed in Wan2018, Gunsilius2020, Shi2022, and nazaret2023misspecification. Although these studies introduce mixture models with motivations other than the bias problem, we highlight that such models can effectively circumvent bias under imperfect pre-treatment fit.
Here, we briefly review existing work using mixture models. Wan2018 points out that SCMs essentially rely on mixture models, distinguishing them from traditional panel data analysis. Gunsilius2020 proposes distribution SCMs (DiSCo), which estimate SC weights by matching the outcome distributions of the treated unit and the weighted sum of the distributions of the untreated units. Shi2022 introduces fine-grained models to elucidate hidden assumptions in SCMs and explores connections to linear factor models. nazaret2023misspecification extends these insights by analyzing model misspecification under a mixture model formulation.
This study also addresses a question raised by Gunsilius2020. While Gunsilius2020 suggests that standard least squares SCMs try to match the first moment of the outcome distribution (and therefore differ from distributional approaches), we stress that least squares approaches are additionally vulnerable to implicit endogeneity, which can generate bias. Our moment-matching approach avoids this problem by matching a broader set of distributional features.
Suppose there are $J+1$ units, indexed by $j \in \mathcal{J} \coloneqq \{0, 1, 2, \dots, J\}$, and a time series indexed by $t \in \mathcal{T} = \{1, 2, \dots, T\}$, where $3 \leq T \in \mathbb{N}$. Without loss of generality, assume that the first unit ($j = 0$) is the treated unit, i.e., the unit affected by the policy intervention of interest. The set of potential control units is $\mathcal{J}^U \coloneqq \mathcal{J} \backslash \{0\} = \{1, 2, \dots, J\}$, consisting of units not affected by the intervention. We also assume that our data span $T$ periods and that the intervention occurs at $t = T_0$, with $2 \leq T_0 < T$. Observations in period $t \in \mathcal{T}_0 \coloneqq \{1, 2, \dots, T_0\}$ are from before the intervention, and observations in period $t \in \mathcal{T}_1 \coloneqq \mathcal{T} \backslash \mathcal{T}_0$ are from after the intervention. Let $T_1 \coloneqq |\mathcal{T}_1| = T - T_0$.
Following the Neyman-Rubin causal model Neyman1923,Rubin1974, we define the potential outcomes of the policy intervention. For each unit $j \in \mathcal{J}$ and period $t \in \mathcal{T}$, there are two outcomes of interest, $\left(Y^I_{j,t}, Y^N_{j,t}\right) \in \mathbb{R}^2$, corresponding to scenarios with and without the intervention. For each $j \in \mathcal{J}$ and $t \in \mathcal{T}$, let $F^I_{j,t}(y)$ and $F^N_{j,t}(y)$ be the cumulative distribution functions (CDFs) for $Y^I_{j,t}$ and $Y^N_{j,t}$, respectively.
We next define our observations. For each unit $j \in \mathcal{J}$, only one of the two potential outcomes can be observed, corresponding to what actually happened. Specifically, for each $j \in \mathcal{J}$, we observe $Y_{j,t} \in \mathbb{R}$ such that
We aim to estimate the effect of the policy intervention by predicting $Y^I_{0,t}$ from $\left(Y^N_{j,t}\right)_{j \in \mathcal{J}^U}$ and comparing that prediction to the observed $Y^N_{0,t}$ for $t \in \mathcal{T}_1$. We define the treatment effect more formally in the next subsection.
In SCMs, there are several ways to define the treatment effect $\tau_{0,t} \in \mathbb{R}$ for each $t \in \mathcal{T}_1$ (i.e., after the intervention). First, for each $t \in \mathcal{T}_1$, we can define the treatment effect for the treated unit $j = 0$ as
Another definition is based on the following linear factor model:
Under the linear factor model, the treatment effect is given by $\tau_{0,t} = Y^I_{0,t} - Y^N_{0,t}$, a non-random variable by definition. Consequently, the expression $\tau_{0,t} \coloneqq \mathbb{E}[Y^I_{0,t}] - \mathbb{E}[Y^N_{0,t}]$ also holds under the factor model.
Comparative case studies aim to evaluate $\tau_{0,t}$ by predicting $Y^N_{0,t}$ for $t > T_0$, the counterfactual outcome that would have been observed for the treated unit had the intervention not occurred. Because the intervention takes place after $t = T_0$, $Y^N_{0,t}$ is a counterfactual outcome. In settings where data consist of only a few units (e.g., regions or countries), it can be difficult to find a single untreated unit that provides a suitable comparison. For this problem, SCMs enable us to construct a counterfactual outcome using a predictor based on the outcomes of the untreated units. Under suitable model assumptions that relate $Y^N_{0,t}$ to $\left(Y^N_{j,t}\right)_{j \in \mathcal{J}^U}$, we can construct the predictor using observations from the pre-intervention period and then predict the unobserved counterfactual outcome in the post-intervention period. Consequently, we estimate the treatment effect by comparing the predicted $Y^N_{0,t}$ with the observed $Y^I_{0,t}$.
Distributional Treatment Effects. In program evaluation, we are often interested not only in the scalar treatment effect but also in the distributions of outcomes Maier2011 or their functionals, such as quantile treatment effects (QTEs) and expected social welfare Abadie2002. By applying our method, we can also estimate the distribution of the counterfactual outcome, $p^N_{0,t}(y)$, for $t \in \mathcal{T}_1$. The treatment effect is then captured by the difference between $p^I_{0,t}(y)$ (the distribution of the actual outcome) and $p^N_{0,t}(y)$, or by functionals of these distributions, such as QTEs and expected social welfare. Depending on the application, we may also adjust the target density, for example, by averaging across all post-intervention periods: $\overline{p}^N_{0}(y) = \frac{1}{T_1}\sum_{t\in\mathcal{T}_1}p^N_{0,t}(y)$. We refer to the treatment effects related to these distributions as distributional treatment effects ParkShalit2021, KallusOprescu2022, chikahara2022feature. In the context of SCMs, Gunsilius2020 and Chen2020 have proposed methods for estimating such distributional treatment effects.
This section reviews the original SCM proposed by AbadieGardeazabal2003. To estimate the treatment effect $\tau_{0,t}$, we predict the counterfactual outcome $Y^N_{0,t}$ using a weighted sum of $\{Y^N_{j,t}\}_{j=1}^J$ for $t\in\mathcal{T}_1$. Specifically, SCMs predict $Y^N_{0,t}$ as \[ \widehat{Y}^N_{0,t} \coloneqq w_0 + \sum_{j\in\mathcal{J}^U} w_j Y^N_{j,t}, \] where $w_0 \in\mathbb{R}$ and $w_j \in\mathbb{R}$ $(j\in\mathcal{J}^U)$ are the SC weights.
Several methods exist for estimating the SC weights. In the SCM proposed by AbadieGardeazabal2003, these weights are determined by minimizing the squared error between $Y^N_{0,t}$ and the weighted sum of $Y^N_{1,t},\dots,Y^N_{J,t}$. Define \[ \Delta^J \coloneqq \left\{\bm{w} = (w_1\ w_2\ \dots\ w_J)^\top \in [0, 1]^J \mid \sum_{j\in\mathcal{J}^U}w_j = 1\right\}. \] Then, AbadieGardeazabal2003 estimate the SC weights as \[ \bm{w}^{\mathrm{Abadie}} = \operatorname*{arg\,min}_{\bm{w}\in\Delta^J} \sum_{t\in\mathcal{T}_0}\left(Y_{0, t} - \sum_{j\in\mathcal{J}^U} w_jY^N_{j, t}\right)^2. \] Recall that we referred to SCMs using this approach as the Least-Squares-SCMs (LS-SCMs). The counterfactual outcome is then predicted as \[ \widehat{Y}^{N, \mathrm{Abadie}}_{0, t} = \widehat{w}^{\mathrm{Abadie}}_0 + \sum_{j\in\mathcal{J}^U}\widehat{w}^{\mathrm{Abadie}}_jY^N_{j,t}. \] Using $\widehat{Y}^{N, \mathrm{Abadie}}_{0, t}$, we estimate the treatment effect as \[ \widehat{\tau}^{\mathrm{Abadie}}_{t} = Y^I_{0, t} - \widehat{Y}^{N, \mathrm{Abadie}}_{0, t}. \]
The literature on inference for SCMs has grown significantly, extending the original proposals by AbadieGardeazabal2003 and AbadieDiamondHainmueller2010. Notable contributions include works by Kathleen2020, Shaikh2021, CattaneoFeng2021, and Chernozhukov2021. Doudchenko2016 proposes estimating the SC weight as \[ \widetilde{\bm{w}}^{\mathrm{DI}} = \operatorname*{arg\,min}_{\widetilde{\bm{w}} = (w_0\ w_1\ \cdots\ w_J)^\top \in \mathbb{R}^{J+1}} \sum_{t\in\mathcal{T}_0}\left(Y_{0, t} - w_0 - \sum_{j\in\mathcal{J}^U} w_jY_{j, t}\right)^2 + \lambda_1 \|\widetilde{\bm{w}}\|_1 + \lambda_2\|\widetilde{\bm{w}}\|_2, \] where $\lambda_1 > 0$ and $\lambda_2 > 0$ are regularization parameters.
This section addresses the identification problem of treatment effects and points out the problem of implicit endogeneity in LS-SCMs.
We revisit the assumptions underlying SCMs. In many existing studies, a linear relationship is assumed between $Y^N_{0,t}$ and $Y^N_{1,t},\dots, Y^N_{J,t}$. One possibility is positing the existence of $\bm{w}^* \in \Delta^J$ AbadieDiamondHainmueller2010 such that \[ Y^N_{0,t} = \sum_{j\in[J]} w^*_j Y^N_{j, t}. \] This assumption, often called the perfect pre-treatment fit Ferman2021, implies that the stochastic process $\left(Y^N_{0,t}\right)_{t\in \mathcal{T}_0}$ can be fully replicated by $\sum_{j\in[J]} w^*_j Y^N_{j, t}$ for $t\in \mathcal{T}_0$. In linear factor models where $Y^N_{j, t} = c_j + \delta_t + \lambda_t \mu_j + \epsilon_{j, t}$, this assumption might imply $\epsilon_{0, t} = \sum_{j\in[J]} w^*_j \epsilon_{j, t}$, which is often considered unrealistic. Note that this is a specific case and not necessary for $Y^N_{0,t} = \sum_{j\in[J]} w^*_j Y^N_{j, t}$.
A more natural assumption is a linear model between $\mathbb{E}_{0, t}[Y^N_{0, t}]$ and $(\mathbb{E}_{j, t}[Y^N_{j, t}])_{j\in\mathcal{J}^U}$.
Compared to Assumption (ref), the assumption $Y_{0, t} = \sum_{j\in\mathcal{J}^U} w^*_j Y_{j, t}$ is stronger and less plausible, given that $Y_{j,t}$ is random.
We now consider the linear factor model introduced in Definition (ref), along with Assumption (ref). This is a specific case of (ref). Under the linear factor model, we adopt the following more specialized assumption:
Assumption (ref) is a specific case of Assumption (ref); if Assumption (ref) holds, then Assumption (ref) also holds under the linear factor model in Definition (ref).
Under these assumptions, Ferman2021 shows that the SC estimator $\widehat{\tau}^{\mathrm{Abadie}}_{0,t}$ suffers from bias and is not consistent for $\tau_{0, t}$ when $c_j$ is non-zero. We also adopt the following assumption regarding the linear factor model.
Under Assumptions (ref) and (ref), Ferman2021 concludes that the LS-SCM proposed by AbadieGardeazabal2003 does not yield a consistent estimator for $\tau_{0, t}$, as the following proposition.
To alleviate this bias problem, Ferman2021 proposes the demeaned SC estimator: \[ \widehat{\bm{w}}^{\mathrm{FP}} = \operatorname*{arg\,min}_{\widehat{\bm{w}}^{\mathrm{FP}}\in\Delta^J}\frac{1}{T_0}\sum_{t\in\mathcal{T}_0}\left\{\left(Y^N_{0, t} - \overline{Y}^N_0\right) - \sum_{j\in\mathcal{J}^U}w_j\left(Y^N_{j, t} - \overline{Y}^N_j\right)\right\}^2, \] where $\overline{Y}^N_j = \frac{1}{T_0}\sum_{t\in\mathcal{T}_0}Y_{j, t}$. They then estimate $\tau_{0, t}$ as \[ \widehat{\tau}^{\mathrm{FP}}_{0, t} = Y^I_{0, t} - \overline{Y}^N_0 - \sum_{j\in\mathcal{J}^U}\widehat{w}^{\mathrm{FP}}_j\left(Y^N_{j, t} - \overline{Y}^N_j\right). \] They show that this approach can reduce the bias seen in the classical LS-SCM estimator. However, there still remains bias in the estimator, which does not vanish even if $T_0 \to \infty$.
We now interpret the asymptotic bias found by Ferman2021 as an endogeneity problem. We consider (ref), that is, $\mathbb{E}_{0, t}[Y^N_{0, t}] = \sum_{j\in\mathcal{J}^U} w^*_j \mathbb{E}_{j, t}[Y^N_{j, t}]$, which is more general than the linear factor model.
To illustrate, we consider a simpler case with the least-squares estimator rather than $\bm{w}^{\mathrm{Abadie}}$: \[ \widehat{\bm{w}}^{\mathrm{LS}} = \operatorname*{arg\,min}_{\bm{w}\in\mathbb{R}^J} \frac{1}{T_0}\sum_{t\in\mathcal{T}_0}\left(Y_{0, t} - \sum_{j\in\mathcal{J}^U} w_j Y_{j, t}\right)^2. \] We show that $\widehat{\bm{w}}^{\mathrm{LS}}$ is asymptotically biased due to endogeneity. Define $\epsilon_{j, t} = Y_{j, t} - \mathbb{E}_{j, t}[Y_{j, t}]$. Under (ref), we obtain
Noting that $\mathbb{E}[\nu_{t} Y_{j, t}] \neq 0$, we see that there is endogeneity, as the regressor and the error term are correlated. Endogeneity is known to induce bias in least-squares estimation. Let $Q^*$ be a $(J \times J)$ matrix whose $(i, j)$-element is $\frac{1}{T_0}\sum_{t\in\mathcal{T}_0}\mathbb{E}_{i, t}[Y_{i, t}]\mathbb{E}_{j, t}[Y_{j, t}]$, and let $\Sigma$ be a $J \times J$ diagonal matrix whose $(j,j)$-element is $\frac{1}{T_0}\sum_{t\in\mathcal{T}_0} \sigma^2_{j, t}$.
This result follows from Section 5.6 of Greene2003Econometric and implies that, under (ref), there is an asymptotic bias in $\widehat{\bm{w}}^{\mathrm{LS}}$; in other words, least-squares-type estimators do not converge to the true $\bm{w}^*$.
Thus, the identification of treatment effects faces a challenge due to implicit endogeneity. While the estimator $\widehat{\bm{w}}^{\mathrm{Abadie}}_n$ remains standard in many applications, it inherits this endogeneity issue. Such concerns have been noted in prior works Chernozhukov2021,Ferman2021,Ferman2021OntheProp, and fry2024method independently frames the bias as an endogeneity problem, proposing an instrumental variable method to address it.
Bias is caused by a measurement error. As we showed above, the source of the implicit endogeneity is the measurement error. Since we assume $\mathbb{E}_{0, t}[Y^N_{0, t}] = \sum_{j\in\mathcal{J}^U} w^*_j \mathbb{E}_{j, t}[Y^N_{j, t}]$ as a regression model, the regressors are $(\mathbb{E}_{j, t}[Y^N_{j, t}])_{j \in \mathcal{J}^U}$. However, in practice, we can only use $(Y_{j, t})$ as the regressors. Because the regressors contain measurement errors $(\mathbb{E}_{j, t}[Y^N_{j, t}] - Y_{j, t})_{j \in \mathcal{J}^U}$, endogeneity arises, as is commonly discussed in elementary textbooks Greene2003Econometric.
\color{black}
As discussed in Section (ref), Assumption (ref) can be weak or incompatible for identifying $\bm{w}^*$ when using least squares. We, therefore, adopt a different assumption for identification. Specifically, we propose that the distribution of $Y^N_{0, t}$ can be replicated by a weighted combination of the distributions of $Y^N_{j, t}$, and we introduce an SCM based on moment matching.
Section (ref) and the results in Ferman2021 indicate that combining \[ \mathbb{E}_{0, t}\left[Y_{0, t}\right] = \sum_{j\in\mathcal{J}^U} w^*_j \mathbb{E}_{j, t}\left[Y_{j, t}\right] \] with least squares estimators do not yield consistent estimators of $\bm{w}^*$. We thus adopt mixture models as a more theoretically favorable but still realistic assumption.
Identification. We begin by positing that the distribution of the target unit's outcome $Y^N_{0, t}$ can be expressed as a linear combination of the distributions of $Y^N_{1, t}, \dots, Y^N_{J, t}$. For simplicity, in this subsection, we assume that $Y^N_{j,t}$ is a continuous random variable with a probability density function (PDF). Under this scenario, we suppose that the density of $Y^N_{0, t}$ can be replicated by a weighted sum of the densities of $Y^N_{j, t}$:
Estimation. We next discuss how to estimate the SC weights that satisfy (ref). One approach employs optimal transport to minimize the Wasserstein distance Gunsilius2020. In this study, we propose a simpler estimation procedure, as the approach of Gunsilius2020 is more complex than that of the original SCMs.
There are various ways to estimate $(w^*_j)_{j\in\mathcal{J}^U}$. We consider the following:
A straightforward approach to estimating $(w^*_j)_{j\in\mathcal{J}^U}$ is to minimize the divergence or distance between two distributions whose densities are given by $p^N_{0,t}(y)$ and $\sum_{j\in\mathcal{J}^U} w_j p^N_{j,t}(y)$, given $(w_j)_{j\in\mathcal{J}^U}$. For the divergence or distance measures, we may use, for example, the Kullback-Leibler (KL) divergence or the Wasserstein distance. The KL divergence can be estimated nonparametrically using density ratio techniques (assuming the densities exist) Sugiyama:2012:DRE:2181148,Kato2021bregman. Alternatively, we may estimate the weights by minimizing the maximum mean discrepancy.
The second approach is to match characteristic functions. Since the mixture model (ref) holds, we have $ \mathbb{E}_{0,t}\left[\exp\left(\sqrt{-1} \xi Y^N_{0,t}\right)\right] = \sum_{j\in\mathcal{J}^U} w^*_j \mathbb{E}_{j,t}\left[\exp\left(\sqrt{-1} \xi Y^N_{j,t}\right)\right] $ for all $t \in \mathcal{T}$ and $\xi\in\mathbb{R}$. where $\sqrt{-1}$ denotes the imaginary unit. The LHS is the characteristic function (CHF) of $p^N_{0,t}(y)$, and the RHS is that of $\sum_{j\in\mathcal{J}^U} w_j p^N_{j,t}(y)$, given $(w_j)_{j\in\mathcal{J}^U}$. Since CHFs uniquely determine the corresponding distributions, we can estimate $(w^*_j)_{j\in\mathcal{J}^U}$ by matching the CHFs. Such matching can be implemented using the method of moments, for instance, the approach proposed in Carrasco2017efficientestimation.
The last approach is to match the moments of the distributions $p^N_{0,t}(y)$ and $\sum_{j\in\mathcal{J}^U} w_j p^N_{j,t}(y)$. Intuitively, if we find $(w_j)_{j\in\mathcal{J}^U}$ such that all the moments of the distributions match, we can interpret it as an estimate of $(w^*_j)_{j\in\mathcal{J}^U}$. This idea can also be motivated from the perspective of CHF matching, as noted in Remark (ref). Note that to justify this approach, we must ensure that the moments uniquely determine the distributions. This is the classical moment problem Stchmudgen2020lecturesmomentproblem, which has been extensively studied. It is known that certain conditions are required to guarantee uniqueness.
Among the three approaches, we focus on the last one, moment matching. As we show later, it is sufficient to estimate $(w^*_j)_{j\in\mathcal{J}^U}$ by matching sample means of polynomials of $Y_{a,j}$. Compared to the other approaches, the implementation of moment matching is significantly simpler. Note that it cannot be applied to all general distributions without additional assumptions since the moments may not be uniquely determined for each distribution. However, if we assume boundedness of $Y_{a,j}$, we can ensure that the distributions are uniquely determined by their moments Hausdorff1921summationsmethodenund. In this study, for simplicity, we adopt this moment matching approach under the assumption of bounded random variables.
\color{black}
We now state our core assumption for identifying the SC weights.
Assumption (ref) ensures that $Y^N_{j, t}$ has a finite $G$-th moment for any $G\in\mathbb{N}$. If $Y^N_{j, t}$ is bounded but not in $[0, 1]$, we can normalize $Y^N_{j, t}$ to satisfy this assumption without loss of generality. Assumption (ref) has been employed in earlier work, including Wan2018, Gunsilius2020, and Shi2022. Wan2018 notes that SCMs effectively rely on Assumption (ref). Gunsilius2020 similarly assumes (ref) and estimates $\bm{w}^*$ by minimizing the Wasserstein distance, while Shi2022 introduces fine-grained models that, combined with Assumption (ref), can justify standard SCMs. We provide further discussion of these fine-grained models in Section (ref).
Although mixture models and linear factor models differ, they are closely related Shi2022,nazaret2023misspecification. In the fine-grained models of Shi2022, certain implicit assumptions underlie linear factor models, suggesting that mixture model assumptions can also be valid for these settings. Independently, nazaret2023misspecification arrive at a similar conclusion. We detail these connections in Section (ref).
Under the mixture model assumption, we propose estimating SC weights by matching moment functions. We refer to our approach as the Momemnt Matching SCM (MMSCM).
Let $v_\gamma \in (0, \infty)$ for $\gamma = 1, 2, \dots, G$ be a fixed weight sequence. We then estimate the SC weights as
We may use any weights $v_\gamma \in (0, \infty)$ for $\gamma = 1, 2, \dots, G$. For example, based on Remark (ref), we can choose $v_\gamma = \frac{2 h^{\gamma + 1}}{(\gamma + 1)!}$. In this case, we estimate $(w^*_j)_{j\in\mathcal{J}^U}$ as
Using $\widehat{\bm{w}}^{\mathrm{MMSCM}}_{T_0, G}$, we can then estimate both the treatment effect and the distributional treatment effect, as discussed in the following subsections.
\color{black}
Once $\widehat{\bm{w}}^{\mathrm{MMSCM}}_{T_0, G}$ is obtained, we estimate the treatment effect similarly to standard SCMs:
We call this estimator the Moment Matching SCM (MMSCM) estimator. Algorithm (ref) summarizes our proposed estimator.
\color{black}
We now provide consistency results for our proposed SC weight estimator. The following theorem presents these results, including consistency for the treatment effect $\tau_0$.
The proof is shown in Appendix (ref). Note that this result also applies to $\widehat{\tau}^{\mathrm{MMSCM}}_{0, t}$ as a special case.
Consequently, because $\mathbb{E}\left[\epsilon_{0, t} - \sum_{j\in\mathcal{J}^U}w^*_{j} \epsilon_{j, t}\right] = 0$, $\widehat{\tau}^{\mathrm{MMSCM}}_{0, t}$ is asymptotically unbiased.
Relationship to the classical SCMs. We next consider the relationship between our MMSCM and the classical SCMs proposed in AbadieGardeazabal2003. AbadieGardeazabal2003 assumes that $Y^N_{0,t} = \sum_{j\in[J]} w^*_j Y^N_{j, t}$, which corresponds to a specific case of our mixture model assumption, $F_{Y^N_{0,t}}(y) = \sum_{j\in\mathcal{J}^U}w^*_j F_{Y^N_{j,t}}(y)\quad \forall y \in \mathbb{R}$. Therefore, under the assumptions in AbadieGardeazabal2003, by using the MMSCMs, we can estimate the weights in an asymptotically unbiased way.
\color{black}
This section presents methods for statistical inference using our proposed MMSCM estimator. We consider statistical inference for the treatment effect using our proposed MMSCM and D2SCM, focusing on testing a sharp null hypothesis, $H_0 : \tau_{0, t} = \alpha_t$ for some $t \in \mathcal{T}_t$ and $\bm{\alpha} = (\alpha_{t})_{t\in\mathcal{T}_1}$. Specifically, we adopt the conformal inference approach of Chernozhukov2021, which has been used by, for example, Ben2021 and Ferman2021. \ifnum\Comments=1\textcolor{black}{Note that under our linear factor model, $\tau_{0,t} = Y^N_{0,t} - Y^I_{0,t} = \alpha_t$ holds, that is, the ATE equals the individual treatment effect. Therefore, our null is sharp. This formulation is also adopted in other works, such as Chernozhukov2021.}\fi
Our goal is to construct a confidence interval whose $p$-value $\widehat{p}$ satisfies
To achieve this, conformal inference proceeds as follows. First, for the sharp null hypothesis \[ H_0 : \tau_{0, t} = \alpha_t, \] we create adjusted post-treatment outcomes for the treated unit $Y^{N\sharp}_{0, t} = Y^I_{0, t} - \alpha_t$ and combine them with the original dataset as \[ \left(Y_{0, 1}, \dots, Y_{0, T_0}, Y^{N\sharp}_{0, (T_0 + 1)}, \dots, Y^{N\sharp}_{0, T}\right). \]
We then apply our proposed method to this augmented dataset, comprising $ \left(Y_{0, 1}, \dots, Y^{N\sharp}_{0, (T_0 + 1)}, \dots, Y^{N\sharp}_{0, T}\right) $ and $(Y_{j, 1})_{t\in\mathcal{T}, j\in\mathcal{J}^U}$, to obtain adjusted weights $\widehat{\bm{w}}(\alpha)$ and an adjusted predictor of the counterfactual outcome, denoted $\widehat{Y}^N_{0, t}(\bm{\alpha})$.
Finally, we compute the $p$-value by assessing whether the adjusted residual “conforms” with the pre-treatment residuals. Define $u_t(\bm{\alpha}) = Y_{0, t} - \widehat{Y}^N_{0, t}(\bm{\alpha})$ for $t\in\mathcal{T}_0$, and $u_t(\bm{\alpha}) = Y^I_{0, t} - \alpha_t - \widehat{Y}^N_{0, t}(\bm{\alpha})$ for $t\in\mathcal{T}_1$. For $j\in \mathcal{T}$, let $\pi_j$ be a permutation of indices defined by $\pi_j(t) = t + j$ if $t + j \leq T$, and $\pi_j(t) = t + j - T$ otherwise. Let $\Pi = \{\pi_{1}, \dots, \pi_{T - 1}\}$. Then the $p$-value is defined as $\widehat{p}(\bm{\alpha}) = 1 - \widehat{F}(\bm{\alpha})$, where \[ \widehat{F}(\bm{\alpha}) = \frac{1}{T}\sum_{j\in\mathcal{T}} \mathbbm{1}\left[ \frac{1}{T}\sum_{t\in\mathcal{T}}\left| u_t(\bm{\alpha}) \right| < \frac{1}{T}\sum_{\pi_{j}\in\mathcal{T}}\left| u_{\pi_{j}}(\bm{\alpha}) \right|\right]. \] Because $Y^N_{0, t}$ is random, inverting this test yields a confidence interval for $\tau_{0, t}$, which is equivalent to constructing a conformal prediction set Vovk2005 for $Y^N_{0, T}$. The $(1-\xi)$ confidence interval is \[ \widehat{C}^{\mathrm{conf}}_{1-\xi} = \left\{ \bm{\alpha}\in\mathcal{A} \bigm| \widehat{p}(\bm{\alpha}) > \xi\right\}, \] where $\mathcal{A}$ is a set of candidate values. Theorem 1 of Chernozhukov2021 guarantees (ref). We summarize the procedure in Algorithm (ref)
\color{black}
To estimate the distributional treatment effect, we approximate the counterfactual distribution $p_{0, t}(y)$ using a bootstrap-like simulation-based method. Specifically, we resample $L > 1$ observations $\{Y^{l, \dagger}_{0}\}_{l=1}^{L}$ from $\{Y_{0, t}, Y_{1, t}, \dots, Y_{J, t}\}_{t \in \mathcal{T}_1}$ as follows: for each $l \in \mathcal{L} \coloneqq \{1, 2, \dots, L\}$,
The empirical distribution of $\{Y^{l, \dagger}_{0}\}_{l=1}^{L}$ serves as an estimator of the distribution of $\frac{1}{T_1} \sum_{t \in \mathcal{T}_1} Y_{0, t}$, as shown in the following theorem. The proof is provided in Appendix (ref). Thus, when our primary focus is on $\frac{1}{T_1} \sum_{t \in \mathcal{T}_1} p_{0, t}(y)$, the above procedure enables us to achieve this goal.
Two-sample homogeneity test. To test distributional treatment effects, consider the null and alternative hypotheses $H_0: p^I_t = p^N_t$ versus $H_1: p^I_t \neq p^N_t$. One can perform a two-sample homogeneity test in combination with our distributional treatment effect estimator. For instance, one may employ maximum mean discrepancy (MMD) Gretton2012 to implement this hypothesis test.
\color{black}
We conclude by discussing some remaining issues.
To clarify assumptions in SCMs, Shi2022 proposes fine-grained models wherein each unit is composed of multiple components, and the outcomes of treated and untreated units are averages of the outcomes across these components. For example, in the empirical study of California’s tobacco control program, the outcome of interest (annual per capita cigarette consumption) at the state level is an average of individuals’ consumption in the state. Building on this framework, one can justify mixture model assumptions by relating them to linear factor models.
Assume there are $U_j$ components in unit $j \in \mathcal{J}$. For each $u\in \mathcal{U}_j \coloneqq \{1,2,\dots,U_j\}$, let $(Y^I_{j, t, u}, Y^N_{j, t, u})$ be the potential outcomes of component $u$ in unit $j$ at time $t$. Suppose there is a $d_W$-dimensional unobserved random variable $W_{j, t, u} \in \mathbb{Z}^{d_W}$ capturing individual characteristics (e.g., age or income). We assume $(Y^I_{j, t, u}, Y^N_{j, t, u}, W_{j,t,u})$ is i.i.d.\ across $u \in \mathcal{U}_j$.
The relationship between $W_{j, t, u}$ and potential outcomes $(Y^I_{j, t, u}, Y^N_{j, t, u})$ can be linear or nonlinear, and possibly time-varying. Let $p_{j,t}(w, y)$ be the PDF of $(W_{j,t}, Y^N_{j,t})$, and let $p_{j,t}(y\mid w)$ be the conditional PDF of $Y^N_{j,t}$ given $W_{j,t}$, while $p_{j, t}$ is the marginal PDF of $W_{j, t}$. We assume that the intervention is administered at a group level and that individuals within each group comply with their group-level treatment.
Under these conditions, Shi2022 shows that the following assumptions are required for the linearity of SCMs in (ref):
This result also implies that the mixture model (ref) holds nazaret2023misspecification, aligning it with linear factor models. Hence, we obtain the following corollary for linear factor models similar to Ferman2021:
Proof Sketch. From Proposition (ref), under Assumptions (ref)--(ref), (ref) holds. Therefore, applying Theorem (ref) yields the desired result under (ref) with $c_j = 0$ for each $j\in\mathcal{J}$.
We can also extend our method to incorporate auxiliary covariates if they are observable. For each unit $j\in \mathcal{J}$, let \[ X_{j,t}\coloneqq (X_{j, t, 1}, X_{j, t, 2}, \dots, X_{j, t, K})\in\mathbb{R}^{K} \] be $K$-dimensional covariates that are unaffected by treatment. Define \[ \widehat{\ell}_{k}(\bm{w}) = \frac{1}{T_0}\sum_{t\in\mathcal{T}_0}\left(X_{0, t, k} - \sum_{j\in\mathcal{J}^U} w_j X_{j, t, k}\right). \] When these covariates are available, we can extend the MMSCM estimator as \ifnum\Comments=1\textcolor{black}{ \[ \widehat{\bm{w}}^{\mathrm{MMSCM}}_{T_0} \coloneqq \operatorname*{arg\,min}_{\bm{w}\in\Delta^J} b_0\sum_{\gamma=1}^G v_\gamma \left|\frac{1}{T_0}\sum_{t\in\mathcal{T}_0} \left(\left(Y^N_{0,t}\right)^\gamma - \sum_{j\in\mathcal{J}^U} w_j \left(Y^N_{j,t}\right)^\gamma\right)\right| +\sum_{k=1}^K b_k \widehat{\ell}^2_{k}(\bm{w}), \] }\fi where $(b_k)_{k=0}^K$ are weights such that $b_k \geq 0$ and $\sum_{k=0}^K b_k = 1$.
Gunsilius2020 propose distributional SCMs, which assume \[ F^{-1}_{Y_{0, t}, N}(q) = \sum_{j\in\mathcal{J}^U}w^*_j F^{-1}_{Y_{j, t}, N}(q) \quad \forall q \in (0, 1), \] where $F^{-1}_{Y_{j, t}, N}$ is the quantile function defined by \[ F^{-1}_{Y_{j, t}, N}(q) \coloneqq \inf\left\{ y\in\mathbb{R}\colon F_{Y_{j, t}, N}(y) \geq q\right\}. \] To estimate $\bm{w}^*$, Gunsilius2020 minimize the Wasserstein distance, referring to their method as DiSCo. Concretely, the DiSCo estimator is
where $\widehat{F}^{-1}_{Y^N_{j, t}}(V_m)$ is the empirical quantile function of $Y^N_{j, t}$, and $\{V_m\}_{m=1}^M$ are i.i.d.\ draws from the uniform distribution on $[0, 1]$.
We conduct experiments using both simulated and real-world data. In all experiments, we set $v_\gamma = 1$ for the MMSCM.
We first perform simulation studies to compare our proposed SCMs with traditional SCMs. Set $J\in\{10, 30, 60\}$ and $G \in \{2, 3, 5, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100\}$, $T_0 = 30$, $T_1 = 100$, and $K=5$.
We consider SCMs for \[ p_{0, t}(\bm{z}) = \sum_{j\in\mathcal{J}^U}w^*_j p_{j, t}(\bm{z}), \] where $\bm{z}$ is a $(K+1)$-dimensional (i.e., $6$-dimensional) vector: the first element corresponds to $Y_{j, t}$ and the remaining elements correspond to $X_{j, t}$. For each $j\in \mathcal{J}$, let $p_{j, t}$ be the density of a multivariate normal distribution \[ \mathcal{N}\left(\bm{\mu}_{j, t}, \bm{\sigma}^2_{j, t}\right), \] where $\bm{\mu}_{j, t} = (\mu_{j, t, 1},\dots,\mu_{j, t, 6})$ and $\bm{\sigma}^2_{j, t}$ is a $(6\times 6)$ diagonal matrix whose $(k,k)$-element is $\sigma^2_{j, t, k}$. Each $\mu_{j, t, k}$ is drawn from a standard normal distribution, and each $\sigma^2_{j, t, k}$ is drawn from a uniform distribution on $[1, 20]$. For each $j\in\mathcal{J}$, we modify $\mu_{j, t, 1}$ and $\sigma^2_{j, t, 1}$ by adding noise: \[ \mu_{j, t, 1} \leftarrow \mu_{j, t, 1} + \epsilon_{j, t}, \quad \sigma^2_{j, t, 1} \leftarrow \sigma^2_{j, t, 1} + \widetilde{\epsilon}_{j, t}, \] where $\epsilon_{j, t}$ and $\widetilde{\epsilon}_{j, t}$ follow $\mathcal{N}(0, 10)$. If $\widetilde{\epsilon}_{j, t}$ is less than or equal to $0.1$, we set $\widetilde{\epsilon}_{j, t} = 0.1$ to avoid near-zero variance.
For the treated unit, we let $Y^N_{0, t} \sim p_{0, t}(\bm{z})$ for $t\in[t_0, T_1]$ and set $Y^I_{0, t} = 20 + Y^N_{0, t}$ for $t\in\mathcal{T}_1$, implying $\tau_{0, t} = 20$. We draw $\bm{w}^*$ from a uniform distribution over $[0,1]^J$ and then normalize it so that its entries sum to $1$.
We apply MMSCM, the method proposed in AbadieDiamondHainmueller2010 (referred to here as Abadie), and DiSCo in Gunsilius2020. We iterate trials $100$ times and evaluate the estimation error of the treatment effect. We plot the estimation error box plots in Figure (ref) \ifnum\Comments=1\textcolor{black}{with medians and means}\fi, indicate that our proposed method achieves smaller estimation errors. Moreover, the estimation error declines as $G$ increases.
Next, we consider another set of simulation studies under conditions similar to those above, but with the following changes: $J\in\{1, 5, 10, 15, 20, 25, 30\}$, $G \in \{2, 5, 10\}$, $T_0 = 30$, $T_1 = 1000$, and $K=5$. All other settings remain as in Section (ref).
We again apply MMSCM and Abadie $100$ times and compute the error of the treatment effect estimation (Figure (ref)). We focus on how the mean squared estimation error changes as $J$ grows. While the estimation error under the method of Abadie2002 grows with $J$, our proposed method does not exhibit such growth. Additionally, our method maintains lower estimation errors than the method of Abadie2002.
A similar pattern appears in distributional treatment effect estimation. We measure the discrepancy between the true and estimated distributions using the Maximum Mean Discrepancy Gretton2012, a probability metric based on kernel embeddings. In the top panel of Figure (ref), we show how MMD varies with $J\in\{1, 5, 10, 15, 20,\dots,35, 40, 45\}$. As before, our proposed methods outperform the method of Abadie2002 in this setting. In the bottom panel of Figure (ref), we plot MMD for $G \in \{2, 10, 50, 100\}$. As $G$ grows (i.e., as more moment conditions are imposed), the predictive performance improves. \ifnum\Comments=1\textcolor{black}{Note that in this experiment, we included the results of AbadieDiamondHainmueller2010 for reference only, since it is not a method for distributional SCM. Our focus is on the gains induced by choosing a larger $G$.}\fi
Our observations may connect with findings by spiess2023double, who examine high-dimensional SCMs. Exploring these connections will be part of our future research.
We now present empirical studies based on the datasets and settings in AbadieGardeazabal2003, AbadieDiamondHainmueller2010, and AbadieDiamondHainmueller2015. In this analysis, we also compare our method with the prediction intervals for SCMs proposed by CattaneoFeng2021, which provide a robust uncertainty evaluation for SCMs. Although this prediction interval cannot be applied to our method due to differences in the underlying models, it serves as a promising alternative to conformal prediction. Therefore, we include its results for reference. Note that while we used our own implementation for our method, Abadie’s method, and DiSCo, we used the pcsi library for the prediction intervals, as published by Cattaneo2022scpiuncertainty.
Terrorist conflict in the Basque Country. AbadieGardeazabal2003 investigate the effect of terrorist conflict in the Basque Country on gross domestic product (GDP). In this example, $J = 16$, $K = 14$, and $(t_0, T_0, T_1) = (1955, 1970, 1997)$. Basque Euskadi Ta Askatasuna (Basque Homeland and Liberty; ETA) was founded in 1959 as a Basque separatist organization in Spain, seeking an independent Basque state primarily through terrorist activities in the Basque Country region. AbadieGardeazabal2003 use annual Spanish regional-level panel data from 1955 to 1997, with a particular focus on the 1970s when ETA’s terrorist activity was significant. The dataset includes fifteen years of pre-intervention observations. The set of untreated units consists of $17$ other Spanish regions. Each region’s record contains the outcome of interest (GDP per capita) and additional covariates (investment rate, population density, five sectoral output measures, and four measures of human capital), all averaged from 1955 to 1968.
Tobacco control program in California. AbadieDiamondHainmueller2010 also investigate the causal effect of Proposition 99, a comprehensive tobacco control program adopted by California. They use annual state-level panel data from 1970 to 2000. The program was implemented in January 1989, providing eighteen years of pre-intervention observations. Four states that adopted large-scale tobacco control programs (Massachusetts, Arizona, Oregon, and Florida) and states that raised cigarette taxes by at least \$0.50 between 1989 and 2000 (Alaska, Hawaii, Maryland, Michigan, New Jersey, New York, and Washington) as well as the District of Columbia are excluded from the set of 38 control states. The outcome of interest is annual per capita cigarette sales in packs; covariates include the average retail price of cigarettes, log of state income per capita, the percentage of the population aged 15--24, and per capita beer consumption, all averaged between 1980 and 1988. Here, $J = 20$, $K = 5$, and $(t_0, T_0, T_1) = (1970, 1989, 2000)$.
Reunification of Germany. AbadieDiamondHainmueller2015 examine the impact of Germany’s reunification in 1990. Because no single country provided a suitable comparison, SCMs were used to create a composite unit. In this example, $J = 8$, $K = 31$, and $(t_0, T_0, T_1) = (1960, 1990, 2003)$.
Results. For each dataset, Figures (ref) and (ref) show the estimated treatment effects and the distributional treatment effects, respectively. In Appendix (ref), Tables (ref)--(ref) report the estimated treatment effects and 90% confidence bands obtained via conformal inference. We also plot the confidence bands in Figures (ref)--(ref).
Our proposed method fits the observed outcomes well in the pre-treatment period ($t \in \mathcal{T}_0$) and, accordingly, produces plausible counterfactual outcomes and distributions.
In the conformal inference analysis, when we set the sharp null hypothesis as $H_0: \tau_{0, (T_0 + 1)} = \cdots = \tau_{0, T_1} = 0$ for the treatment effect, none of the methods reject the null hypothesis, except for the “California tobacco control” dataset. Prediction intervals proposed in CattaneoFeng2021 shows tight intervals for every settings. Note that we cannot directly compare the performance of their method and ours due to differences in the underlying models.
This study investigated the asymptotically unbiased estimation of the treatment effect in the setting of SCMs. We first pointed out the problem of implicit endogeneity in the linear regression models commonly employed by existing SCMs, which can be interpreted as a form of measurement error bias. To address this issue, we focused on distributional SCMs, in which we assume mixture models for the outcome distributions and developed a moment-matching SCM. By matching the moments between the outcome of the treated unit and the weighted sum of the outcomes of the untreated units, our method consistently estimates SC weights and reinforces the theoretical foundation for using SCMs in comparative studies. We demonstrated the effectiveness of this approach through both simulation experiments and empirical applications.