Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
20,420 characters · 5 sections · 11 citation commands
Synthetic Difference in Differences for Repeated Cross-Sectional Data
JEL Classification: C15, C29
Keywords: causal inference, difference-in-differences, synthetic difference-in-differences, latent factor models, repeated cross-sections
The synthetic difference-in-differences (SDiD) method (arkhangelsky_synthetic_2021) combines features from difference-in-differences and synthetic control methods. As in synthetic control methods, it computes weights that match pretreatment outcomes between the treated and the control groups. This reduces the need for the parallel trend assumption, often hard to verify in practice (roth_pretest_2022). As in difference-in-differences methods, it controls for unit-specific shifts and can include covariates.\\
This method is based on panel data, yet, repeated cross-sectional data, where each unit is observed only once, is often the only available data. For example, the evaluation of rent control policies often relies on online listing portals data that are repeated cross-sections where cities are different groups and dwellings are the individuals.\\
In this paper, I adapt the SDiD method to repeated cross-sectional data. I focus on the case where each unit belongs to the same group across time and where the treatment is applied to one or more group. Once data are aggregated, unit and time weights are computed for each group using the SDiD method. Then, a third type of weight is computed to account for the different number of observations in each group-period. Using a simulation study, I show that this estimator (RC-SDiD) performs better in terms of bias, standard deviation and RMSE that the SDiD one when the number of observations differs in each group-period.\\
In section (ref), I present the synthetic difference-in-differences estimator for repeated cross-sectional data. Then, I demonstrate the properties of the RC-SDiD estimator in a simulation study in section (ref). Section (ref) concludes.
Consider a dataset with $T$ independent cross-sections, $t=1,...,T$. There are $K$ groups: the first $K^{\rm co}$ are the control groups and the last $K^{\rm tr} = K - K^{\rm co}$ the treated groups. Each group is observed for every period in the sample, but all individuals are not observed for each $t$. Thus, the individual index depends on the cross-section, such as $i(k,t)$ is an individual observed at time $t$ in the group $k$, and its outcome is $Y_{i(k,t), t}$. $N_{k,t}$ is the number of observations in group $k$ at time $t$ and $N$ the total number of individuals. The treatment is assigned to groups $K^{\rm tr}$ at time $t \geq T_{\rm post}$.\\
We make two assumptions that differ from arkhangelsky_synthetic_2021.
Assumption (ref) imposes that we observe iid draws from the underlying population. However, we do not restrict the total number of observations in each cross-section to be random. In the Monte Carlo study, we test for different levels of correlation between unobservable factors and the number of observations in a cross-section.\\
where $Y_{i(k,t), t} (0)$ is the outcome of untreated observations. $\Lambda_k$ is a vector of group factor loadings and $f_t$ a vector of time factors. Both are of size $r$ such as: $\Lambda_k = (\Lambda_1, \Lambda_2, ...,\Lambda_r)^{'}$ and $f_t = (f_1, f_2, ..., f_r)^{'}$. $\alpha_k$ is a group level fixed effect and $\beta_t$ is a time fixed effect.\\ Assumption (ref) means that the factors are at the group level, as the weights we compute.\\
I apply the SDiD method on aggregated data. $Y_{i(k,t), t}$ is aggregated as:
where $\bar{Y}_{k,t}$ is the average of the outcome in group $k$ at time $t$.\\ Those aggregated data are used to compute the weights $\hat{\omega}^{sdid}$ and $\hat{\lambda}^{\rm sdid}$ presented in arkhangelsky_synthetic_2021 and the weights $\nu_{k,t}^{RC}$ specific for repeated cross-sectional data we introduce in this paper. The first weights to estimate are similar to those used in synthetic control methods (abadie_synthetic_2010). They match the outcome of the treated group with a combination of the outcome in the control groups in the pre-treatment period. They also include a group-specific shift that is accounted for in the last step in a weighted DiD regression (equation (ref)). Thus, those weights only need to make the outcomes from the treated and control groups parallel and not an exact match. Those weights $\hat{\omega}^{sdid}$ are computed as:
The regularization parameter $\zeta$ is:
$\zeta$ allows increasing the dispersion of the weights and ensuring their uniqueness.\\
The time weights, $\hat{\lambda}^{\rm sdid}$, match pretreatment periods with post-treatment periods for the control group. They give more weight to pre-treatment periods that are similar to post-treatment periods. They are computed as:
Because the number of observations in each group may be different for each group-period, $\hat{\omega}_k^{\rm sdid}$ do not ensure a similar trend between the treated and control groups. I compute cross-sectional weights that accounts for the different number of observations in each group-period. The cross-sectional weights $\nu_{k,t}^{RC}$ are computed as:
As demonstrated in appendix (ref), using the cross-sectional weights $\nu_{k,t}^{RC}$ allow the total weights of each group-period to be equal to $\hat{\omega}_k^{\rm sdid} \times \hat{\lambda}_t^{\rm sdid}$ as in the SDiD method.\\
Once the weights $\nu_{k,t}^{RC}$ are obtained, the treatment effect is estimated in a weighted regression as:
where $\mu$ is an intercept, $\alpha_k$ a group fixed effect and $\beta_t$ a time fixed effect. $W_{k,t}$ is a binary variable representing treatment exposure, and $\tau$ is the treatment effect.\\
The RC-SDiD method can be adapted to include covariates (arkhangelsky_synthetic_2021, kranz_synthetic_2022) and to staggered treatment timing (arkhangelsky_synthetic_2021, porreca_synthetic_2022).
I use Monte Carlo simulations to study the properties of the RC-SDiD estimator compared to the SDiD and DiD estimators\footnote{The code can be found here: https://github.com/yoannmorin/RC-SDID}. I generate the outcome using a data generating process (DGP) similar to xu_generalized_2017 but without covariates. The unobservable dimension is generated using interactive fixed effects (bai_panel_2009) at the group level. The outcome for observation $i$ in group $k$ at time $t$ is simulated as:
Only the first group is considered to be treated to keep a reasonable number of observations. The error term $\varepsilon_{i(k,t), t}$, time fixed effects $\beta_t$, and $f_t$ are i.i.d. $N(0,1)$. The values of group fixed effects $\alpha_k$ and factor loadings $\Lambda_k$ are drawn from uniform distributions. For the control groups, I use $U[-\sqrt{3}, \sqrt{3}]$ and $U[\sqrt{3} - 2w\sqrt{3}, 3\sqrt{3} - 2w\sqrt{3}] $ for the treated group, where $w \in [0,1]$. As highlighted in xu_generalized_2017, it allows treatment status and group-specific effects to be correlated and to have a variance of 1 for the random variables (when $0\leq w <1$).\\
The varying number of observations in each group-period is computed as:
where $S_k$ is a scale parameter for each group $k$. It allows the number of observations to differ between each group. It is drawn from a discrete uniform distribution and I test different range for this parameter. The draw of $S_k$ is allowed to be correlated with group fixed effects for different correlation levels $\rho$. $Base_{RC}$ is the baseline number of observations in each cross-section. To simulate the evolution of the number of observations, the variable $E_{k,t}$ is defined as:
On average, the number of observations in each cross-section will slightly increase, while keeping a reasonable number of observations.\\
I draw new independent values of $\varepsilon_{i(k,t), t}$ for each of the 1000 repetitions. The other parameters are fixed, with $Base_{RC}=100$, $S_k \in [1,10]$ is correlated with group fixed effects for $\rho=0.2$, and $w=0.2$. I use 30 control groups and 30 periods where half are pre-treatment periods. The treatment effect is set to $\tau=0.3$ and the number of factors and loadinds is $r=1$. I compare the performance of the RC-SDiD estimator with DiD and SDiD by computing the mean bias, the standard deviation and the RMSE.\\
Table (ref) displays the effect of the scale parameter on the performance of the model. When the value of this parameter increases, the number of observations in the dataset also increases. Thus, an efficient estimate should perform better when $S_k$ grows. The DiD estimate is the most biased estimate because of interactive fixed effects, while the RC-SDiD is the least biased one. Its bias is reduced when $S_k$ grows. The SD of all models gets lower as $S_k$ grows because of the higher number of observations. However, the RMSE of the DiD and SDiD estimators are only slightly affected by the increase of $S_k$, while the RMSE of RC-SDiD is greatly improved. Thus, adding the third type of weight presented in section (ref) significantly improves the performance of the estimator when the number of observations differs in each cross-section.\\
Table (ref) focuses on the effects of the number of factors and loadings on the performance of the estimators. When $r=0$, the DGP only includes group and time fixed effect, and the DiD estimator performs the best across all measures. However, when $r$ increases, it becomes the estimator that performs the worst because it cannot account for interactive fixed effects. The RC-SDiD estimator has the best performance across all values of $r>0$. The bias, SD and RMSE of the SDiD estimator are all highly increasing with $r$, while they stay moderate for the RC-SDiD one.\\
In appendix (ref), (ref) and (ref) the results are reported for respectively: different values of $w$, different values of $\rho$ and for different sample sizes. In all cases, RC-SDiD performs better than DiD and SDiD, confirming our previous results.
In this paper, I adapted the SDiD estimator (arkhangelsky_synthetic_2021) for repeated cross-sectional data, with a simple implementation that consists in adding a third type of weight that depends on the number of observations in each pair of group-period. Using Monte Carlo simulations, I demonstrated that the RC-SDiD estimator significantly improves the performance of the SDiD estimator when using repeated cross-sectional data.
I am grateful to the comments and suggestions from Marie Breuillé, Capucine Chapel, Julie Le Gallo and Morgan Ubeda. Financial support from the Atelier Parisien d'Urbanisme is gratefully acknowledged.