Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
16,301 characters · 4 sections · 17 citation commands
Temporal Aggregation for the Synthetic Control Method
Empirical researchers often use the synthetic control method (SCM) to estimate the impact of a treatment on a single unit in panel data settings AbadieAlbertoDiamond2010. The “synthetic control” is a weighted average of control units that balances the treated unit’s pre-treatment outcomes as closely as possible.
Two challenges arise when using SCM with higher frequency data, such as when the outcome is measured every month versus every year. First, because there are more pre-treatment outcomes to balance, achieving excellent pre-treatment fit is typically more challenging. Second, even when excellent pre-treatment fit is possible, higher-frequency observations raise the possibility of bias due to overfitting to noise. A recent review by abadie2022synthetic explicitly cautions about such bias from using disaggregated outcomes in SCM. Instead, researchers can first aggregate the outcome series into lower-frequency (e.g., annual) observations, and then estimate SCM weights that minimize the imbalance in these aggregated pre-treatment outcomes. Doing so mechanically improves pre-treatment fit as well.
In this paper, we propose a framework for temporal aggregation for SCM. Adapting recent results from sun2023using, we first derive finite-sample bounds on the bias for SCM under a linear factor model when using temporally disaggregated versus aggregated outcome series. With these bounds, we then show that temporal aggregation only reduces bias to the extent that doing so reduces noise without also overly reducing signal for the underlying factors. The optimal trade-off between the two approaches depends on unknown parameters. We argue, however, that finding synthetic control weights that jointly balance both the disaggregated and aggregate series is a promising compromise approach for practice.
Our setup builds on an expansive literature on SCM, especially recent papers that modify SCM to mitigate bias both due to imperfect pre-treatment balance ferman2018revisiting, benmichael2021_ascm and bias due to overfitting to noise kellogg2021combining. Most directly relevant, sun2023using discuss using use multiple outcomes to mitigate both sources of bias.
For each unit $i=1,\dots,N$ and at each lower-frequency time interval $t=1,\dots,T$, we observe $K$ higher-frequency observations of the outcome denoted as $Y_{itk}$ where $k=1,\dots,K$. For instance, in a monthly series $t=1,\dots,T$ could represent years and $k=1,2,\dots,12$ the months within each year. The choice of time interval $t$ reflects how we compute temporal aggregates, such as aggregating monthly data into yearly averages. We maintain fixed values for $N$ and $K$ throughout the analysis.
We denote the exposure to a binary treatment by $W_{i}\in\{0,1\}$. We restrict our attention to the case where a single unit receives treatment, and follow the convention that this is the first one, $W_1 = 1$. The remaining $N_0 \equiv N-1$ units are possible controls, often referred to as “donor units.” To simplify notation, we limit to one post-treatment period, $T = T_0 + 1$, though our results easily extend to larger $T$.
We denote the potential outcome under treatment $w$ with $Y_{itk}(w)$. We are interested in the treatment effects for the treated unit during the $K$ higher frequency observations during post-treatment period $T$: $\tau_{Tk} = Y_{1Tk}(1) - Y_{1Tk}(0)$ for $k=1,\dots,K$. Since we directly observe $Y_{1Tk}(1) = Y_{1Tk}$ for the treated unit, we focus on imputing the missing counterfactual outcome under control, $Y_{1Tk}(0)$.
Throughout, we will focus on de-meaned or intercept-shifted weighting estimators Doudchenko2017, ferman2018revisiting. We denote $\bar Y_{i\cdot \cdot} \equiv \frac{1}{T_0K}\sum_{t=1}^{T_0}\sum_{j=1}^{K} Y_{itj}$ as the pre-treatment average for the outcome for unit $i$, and $\dot Y_{itk} = Y_{itk} - \bar{Y}_{i\cdot \cdot}$ as the corresponding de-meaned outcome. We consider estimators of the form: $\widehat{Y}_{1Tk}(0) \equiv \bar{Y}_{1\cdot \cdot} + \sum_{i=2}^N \gamma_i \dot{Y}_{iTk},$ where $\gamma \in \mathcal C \subset \ensuremath{\mathbb{R}}^{N-1}$ is a set of weights. Our paper centers on how to choose the weights $\gamma$ from a set $\mathcal{C}=\{\gamma\in\mathbb{R}^{N-1}\mid\|\gamma\|_1\leq C,\sum_{i}\gamma_{i}=1\}$ for a known $C$.
The first approach we consider is finding a synthetic control that has the best pre-treatment fit on the (de-meaned) disaggregated high-frequency outcomes $q^{\text{dis}}(\cdot)$. We refer to this set of weights as the disaggregated weights $\hat{\gamma}^{\text{dis}}$: \[ \underset{\gamma\in \mathcal{C}}{\min} \frac{1}{T_{0}}\frac{1}{K}\sum_{k=1}^{K}\sum_{t=1}^{T_{0}}\left(\dot Y_{1tk} -\sum_{W_{i}=0}\gamma_{i} \dot Y_{itk}\right)^{2}. \]
An alternative choice is to optimize the aggregated objective $q^{\text{agg}}(\cdot)$, the pre-treatment fit for the temporally aggregated outcomes via averaging. We refer to the set of weights that minimize this objective as the aggregated weights $\hat{\gamma}^{\text{agg}}$: \[ \underset{\gamma\in \mathcal{C}}{\min} \frac{1}{T_{0}}\sum_{t=1}^{T_{0}}\left(\frac{1}{K}\sum_{k=1}^{K}\dot Y_{1tk} -\sum_{W_{i}=0}\gamma_{i}\dot Y_{itk}\right)^{2}. \]
To derive finite sample bias bounds for $Y_{1Tk}(0) - \widehat{Y}_{1Tk}(0)$ for each $k=1,\dots,K$, we assume that the outcomes under control are generated as $Y_{itk}(0) = \alpha_{i} + \beta_{tk} + L_{itk} + \varepsilon_{itk}$, with $\sum_{t=1}^T \beta_{tk} = 0$ for all $k$; see, for example, athey_matrix_2021. After incorporating the additive two-way fixed effects, the model component retains a term $L_{itk}$ with $\sum_{i=1}^N L_{itk} = 0$ for all $t, k$ and $\sum_{t=1}^T L_{itk} = 0$ for all $i, k$. We assume the idiosyncratic errors $\varepsilon_{itk}$ are mean zero sub-Gaussian random variables with scale parameter $\sigma$, independent of the treatment status $W_{i}$. We also assume independence across units and time, which is plausible if the model components capture co-movement in the outcome.
Let the matrix $L\in\mathbb{R}^{N\times(TK)}$ contain $L_{itk}$ for the treated unit and the remaining rows correspond to control units. sun2023using show that a low rank condition $rank(L)<N-1$ is necessary for there to exist oracle weights $\gamma^\ast$ over donor units that yield an unbiased estimate for the control potential outcome: $ \ensuremath{\mathbb{E}}_{\varepsilon_{Tk}}\left[Y_{1Tk}(0) - \widehat{Y}_{1Tk}(0) \right] = L_{1Tk} - \sum_{i = 2}^N \gamma^\ast_i L_{iTk} = 0. $ Here the expectation is taken over the idiosyncratic errors in the respective post-treatment periods.
The deterministic model component can be written as a linear factor model, $L_{itk}=\boldsymbol{\phi}_{i}\cdot\boldsymbol{\mu}_{tk},$ with $r=rank(L)$, where $\boldsymbol{\mu}_{tk}\in\mathbb{R}^{r}$ are latent time factors and each unit has a vector of time invariant factor loadings $\boldsymbol{\phi}_{i}\in\mathbb{R}^{r}$. Two important quantities for our discussion are $\underbar{\ensuremath{\xi}}^{\text{dis}}$ and $\underbar{\ensuremath{\xi}}^{\text{agg}}$, the smallest singular values of the variance-covariance matrix of, respectively, the time factors $\mu_{tk}$ and the averaged time factors $\bar{\mu}_{t}=\frac{1}{K}\sum_{k=1}^{K}\mu_{tk}$. Following previous literature we assume that $\underbar{\ensuremath{\xi}}^{\text{dis}} > 0$, which avoids issues of weak identification AbadieAlbertoDiamond2010.
For estimated weights $\hat{\gamma}$ based on pre-treatment fit, the bias for the effect in period $Tk$ is due to inadequate balance in the model components: $ Bias(\hat\gamma)=L_{1Tk}- \sum_{i=2}^N \hat{\gamma}_i L_{iTk} . $ We can decompose the bias from estimated weights $Bias(\hat\gamma)$ into two terms using the linear factor model:
where $\omega_{tj}$ are transformations of the factor values that depend on the estimator.
The first term is bias due to imperfect pre-treatment fit (or imbalance) in the pre-treatment outcomes, $\dot{Y}_{itj}$. The second term is bias due to overfitting to noise, also known as the approximation error. This term arises because the optimization problems minimize imbalance in observed pre-treatment outcomes --- noisy realizations of latent factors --- rather than minimizing imbalance in the latent factors themselves.
Theorem 1 in the appendix formally states high-probability bounds on the bias terms, which we obtain using results from sun2023using. These bounds hold in finite samples and account for imperfect pre-treatment fit. The leading terms in the bias due to imbalance and overfitting are:
where $\underbar{\ensuremath{\xi}}^{\text{dis}}$ and $\underbar{\ensuremath{\xi}}^{\text{agg}}$ are the relevant smallest singular values for the disaggregated and aggregated series.
The first term, bias due to imbalance, does not vanish for either approach. Consistent with ferman2018revisiting, bias remains because the SCM objective does not converge to the objective minimized by the oracle weights due to noise in the outcomes. Aggregation, however, reduces the level of noise in the objective and therefore reduces the bias by a factor of $1/\sqrt{K}$.
The second term in the bias is the contribution of overfitting to noise. This decreases in the total number of pre-treatment observations $T_0 K$ for both the disaggregated and aggregated estimators. For the disaggregated weights this follows prior results AbadieAlbertoDiamond2010. Aggregation reduces the noise in the objective, but this is counteracted by a commensurate reduction in the number of pre-treatment observations, netting out to the same risk of bias due to overfitting as with the disaggregated weights.
In practice, the most important consideration in temporal aggregation is whether doing so eliminates the signal for the underlying factors. In particular, if (ignoring constant terms) $\sqrt{K} \underline{\xi}^{\text{agg}} > \underline{\xi}^{\text{dis}}$, aggregation will lead to tighter bias bounds. There are many scenarios where we expect long-run variation to persist, especially after seasonally adjusting the outcomes, and therefore believe this condition might hold. However, if there is not substantial long-run variation and $\underbar{\ensuremath{\xi}}^{\text{agg}}$ is very small, aggregating can leave little behind to learn about the latent factor loadings, and possibly inflate the bias bound. Similar challenges arise in time series model estimation, where aggregation can lead to biased estimates for the true time series models marcet2019temporal.
Rather than choose between these two extremes, we propose finding SCM weights that control a linear combination of the two objectives, with weight $\nu$ on the aggregated objective and weight $(1-\nu)$ on the disaggregated objective. This creates an imbalance “frontier", similar to the approach in benmichael2022_stag, where $\nu = 0$ corresponds to the disaggregated objective and $\nu = 1$ to the aggregated objective. As formalized in Lemma 1 in the appendix, if the SCM weights yield excellent pre-treatment fit on both the disaggregated and aggregated outcomes, these weights will also achieve the minimum of the two bounds. In general, finding the optimal $\nu^\ast$ involves model-derived parameters and is infeasible. As a heuristic, we propose equal weight to the disaggregated and aggregated fits, $\nu = 0.5$, and then assessing sensitivity to this choice.
There are many directions for future work that incorporate recent innovations in panel data methods, including first de-noising amjad_robust_2018 or seasonally adjusting the disaggregated outcome series. We could also explore choosing an optimal level of temporal aggregation for a single SCM objective. Finally, questions about temporal aggregation also arise in event study and other panel data models, suggesting further avenues for fruitful research.
We revisit the bell_texas_2023 study on the SB8, a 2021 Texas law restricting abortion. Using monthly state-level live birth counts from 2016 to 2022, the authors construct a synthetic Texas to estimate SB8's impact on monthly births.
The original analysis uses SCM on monthly data ($\nu = 0$). We explore aggregating to yearly averages ($\nu=1$) and intermediate values of $\nu$ as illustrated in Figure (ref). Combining yearly and monthly births equally ($\nu=0.5$, labeled “Yearly + Monthly" in Figure (ref)) achieves substantial balance on both fronts, reducing potential bias. Appendix Figure 1 indicates a slightly larger estimated effect for post-treatment monthly births using the combined approach ($\nu = 0.5$) than the original analysis. Appendix Figure 2 demonstrates stable estimates across a wide range of $\nu$.