Difference-in-Differences with Multiple Events

Difference-in-Differences with Multiple Events

In 1992, New Jersey not only increased its minimum wage from \$4.25 to \$5.05 per hour, but also reduced its sales tax from 7% to 6% King1992-mv. So when Card1993-ej measured a differential employment trend between New Jersey and Pennsylvania, was it from the minimum wage hike or the sales tax cut? When these confounding events correlates with the event of interest, its treatment effect confounds the trend of the treated group and the control group unevenly. This threatens the parallel trend assumption and creates an omitted event bias.

This paper introduces a two-stage Difference-in-Differences estimator (Double DiD) that controls for confounding events while allowing for rich treatment effect heterogeneity. I introduce a second event to the group-time average treatment effect framework Callaway2021-vo. I show that the omitted event bias comes from the problem of confounded control group and of confounded treated group. Th two problems are addressed with two DiD designs: The first DiD identify the combined effect of the two treatments with a control group not yet treated nor confounded; The second DiD isolates the treatment effect of the target event with a control group simultaneously treated but not yet confounded. I also provide aggregation, estimation, and inference procedures based on CS' results. The estimator is implemented in an open-source R package, fastdid.\footnote{Available at CRAN and \url{https://github.com/TsaiLintung/fastdid}}

I apply Double DiD to revisit the effect of states' staggered raise of the minimum wage on teen employment Cengiz2019-vr, Callaway2021-vo. From 2010 to 2020, states' minimum wage hikes are positively correlated with the Medicaid Expansion under the ACA, causing the unconditional CS DiD estimator to have a bias the size of \DTLfetch{vartable}{name}{bias_avg}{value} % of the effect from Medicaid Expansion in the short term. This bias creates a spurious negative estimate: comparing the standard DiD with the Double DiD estimator, the estimated effect in the year after the minimum wage increase weakens from \DTLfetch{vartable}{name}{short_naive}{value} percentage points (p.p.) to \DTLfetch{vartable}{name}{short_double}{value} p.p. and loses statistical significance.

I contribute to the literature on confounding events by allowing for more treatment effect heterogeneity. To address the omitted event bias, 18% of the papers in American Economic Review using Two-way-fixed-effects (TWFE) control for confounding events De_Chaisemartin2020-ke. However, after the negative weight problem became known Goodman-Bacon2021-ze, TWFE is now largely replaced by new estimators \footnote{See Roth2022-cb for a review of this sprawling literature.}. While the replacements are more robust to treatment effect heterogeneity, they are also less flexible in specification and cannot directly address confounding events. To the best of my knowledge, two existing methods in the modern DiD literature can account for confounding treatments: Caetano2024-dn considers time-varying covariates and de-Chaisemartin2023-jb extends De_Chaisemartin2020-ke to incorporate multiple treatments. Double DiD complements these methods by being the most flexible with the treatment effect heterogeneity of both treatments. This is achieved through being the most selective with the control group, thus I trade some statistical power for a more credible parallel trend assumption.

Additionally, Fricke2017-jo and Roller2023-zy both study a two-treatment DiD in a two-period setup, Hull2018-mv study a two-period setup with multiple treatment arms. Finally, Matthay2022-tx and Griffin2023-nu document the correlation among state policies and their potential confoundedness.

Setup

My setup extends the model of Callaway2021-vo by adding a second event. Consider a panel with periods $t \in \{1,...,T\}$ and units $i \in \{1,...,N\} = \mathbb{I} $. $D^d_{it} \in \{0,1\}$ denotes whether unit $i$ is treated by treatment $d\in \{1,2\}$ in period $t$. Without loss of generality, I focus on identifying the effect from treatment 1 and call treatment 1 the target treatment and treatment 2 the confounding treatment. This two-treatment setup can also address multiple confounding treatments by setting $D^2_{it} = \max(D^2_{it},..., D^R_{it})$.

Once a unit is treated, it will continue to be treated (treatment is an absorbing state):

With irreversible treatments, the switch to being treated can be called an event. I call the period in which a unit experiences an event as its cohort, defined as the period when it was first treated by event $d$, $G^d_i = \underset{D^d_{it} = 1}{\min} t$. If a unit is never treated by event $d$, I set $G^d_i = \infty$. I also define an auxiliary indicator for whether the unit is in $g$ cohort, $G^d_{ig} = \mathbbm{1}\{G^d_i = g\}$. I denote the cohorts with some later-treated groups as $\mathbb{G}^d = \text{supp}(G^d)\setminus\{\bar{g}^d\}$ and the latest-treated among them as $\bar{g}^d = \max{G^d}$.

Let's set up the dynamic potential outcome Murphy2001-ha. Let $Y_{it}(0,0) \in \mathbb{R}$ denote the unit $i$'s potential outcome if it is never treated by either treatment. Then, for $g^1, g^2 = 2,..., T$, $Y_{it}(g^1,0) \in \mathbb{R}$ is its potential outcome if first treated by event 1 in $g^1$ and never treated by event 2. $Y_{it}(0, g^2) \in \mathbb{R}$ is its potential outcome if first treated by event 2 in $g^2$ and never treated by event 1. Finally, $Y_{it}(g^1, g^2) \in \mathbb{R}$ is its potential outcome if first treated by event 1 in $g^1$ and first treated by event 2 in $G^1_i = g^1$. The realized potential outcome $Y_{it}$ is determined by which cohort the unit is actually in:

I assume that the observed outcome $Y_{it}$, treatments $D^1_{it}, D^2_{it}$, and covariates $X_i \in \mathbb{R}^{K}$ is randomly sampled across units:

This imposes independent sampling across units but allows for arbitrary time-series autocorrelation. In the following parts, I suppress unit index $i$.

My first substantial assumption is no anticipation of both treatments:

No anticipation says that the events can not have a treatment effect before the event occurs. This not only holds for the periods before either event ($t<g^1, t<g^2$) but also in periods after one event but before the other, e.g., $g^1\leq t < g^2$. I discuss the case of no anticipation only instead of allowing limited anticipation since it adds little insight and complicates the notation greatly.

Next, the two-treatment version of the parallel trend assumption says that the potential outcome untreated by both events evolves parallelly between the treated units and units not treated by either treatment:

Throughout this paper, I focus on the case of using not yet treated controls. Assumption (ref) is weaker than the one-treatment parallel trend assumption since it not only requires that the unit is not yet treated by the target treatment, but is also not yet treated by the confounding treatment.

Finally, I impose the overlap condition:

This ensures that there is always some treated group and control group available and rules out irregular identification discussed in Khan2010-pu.

\FloatBarrier

Identification

The Omitted Event Bias

To understand how the DiD estimator is biased under confounding treatments, let's focus on the unconditional moments using not-yet-treated controls in CS (their equation 2.9):

$$ATT^{CS}(g^1, t) \equiv \mathbb{E}[Y_t - Y_{g^1-1}|G^1_{g^1} = 1] - \mathbb{E}[Y_t - Y_{g^1-1}|D_t = 0],$$

which intends to identify the group-time average treatment effect of the target treatment:

$$ATT^1(g^1, t) \equiv \mathbb{E}[Y_t(g^1, 0) - Y_t(0,0)|G^1_{g^1} = 1].$$

With confounding events, $ATT^{CS}$ does not identify $ATT^1$. For exposition clarity, I shut down all treatment effect heterogeneity and denote the base period as $b = g^1-1$:

All proofs are collected in Appendix (ref). Theorem (ref) shows two sources of bias. First, the omitted variable bias occurs because under confounding events, the treatment group and control group have non-parallel exposure to the treatment effect of the confounding treatment. Additionally, interaction bias occurs if the target treatment interacts with the confounding treatment ($ ATT^1 + ATT^2 \neq ATT$). This paper will focus on the omitted event bias and leave interaction bias for future research.

Analogous to omitted variable bias, the size of the omitted event bias is determined by the size of the confounding treatment effect ($ATT^{2}$) and the correlation between two treatments:

which can be interpreted as the proportion such that $ATT^2$ contaminates the DiD estimates. Furthermore, the omitted event bias can be decomposed into two parts (see Figure (ref) for an example):

This shows two sources of bias. The control units ($D^2_t=0$) are confounded if the confounding event occurred between the base period and the target period ($D_t^2 = 1, D^2_b = 0 \iff t \geq G^2 > b$). Similarly, the treated units ($G^1_{g^1}=1$) can also be confounded. These two problems will dealt with using two DiD designs.

\FloatBarrier

DiD with Not Yet Treated Nor Confounded Controls

The first stage DiD addresses the problem of confounded control units by using only units not yet treated by either treatment (not-yet-treated-nor-confounded) as controls. Let's first introduce the building block of the analysis, the double-group-time average treatment effect on the treated (DGTATT), defined as the average treatment effect from both treatments among the units in the same event 1 cohort $G_{g^1}^1 = 1$, event 2 cohort $G_{g^2}^2 = 1$, and in period $t$:

I also define the target treatment effect, the parameter of interest of this paper:

for the confounding treatment:

This first stage focuses on identifying the combined effect of both treatments $ATT(g^1, g^2, t)$. The discrepancy between $ATT$ and $ATT^1$ comes from confounded treated units and will be addressed in the second stage.

The parallel trend assumption (Assumption (ref)) provides a straightforward choice for the control group: units neither-yet-treated by both events $D^1_t = 0, D^2_t = 0$. For example, with an unconditional parallel trend, we have

where $b = \min(g^1, g^2)-1$.

My DGTATT connects closely with the group-time average treatment effect in CS since using units not-yet-treated-nor-confounded is synonymous to using units not-yet-treated by an treatment defined as treated-or-confounded. Specifically, given a target cohort $g^1, g^2$, define a subsample of the data $\mathbb{I}(g^1, g^2)$ by removing units that are first treated by either event in $g = \min(g^1, g^2)$ but not in the target cohort ($g^1, g^2$). Formally, let $\mathbb{I}(g^1, g^2) = \{i\in \mathbb{I}: W(g^1, g^2) = 1\}$, where

In this subsample, my assumptions imply CS assumptions:

With Lemma (ref), I can now introduce other identifying moments of the double-group-time average treatment effect by defining them as CS' corresponding moment in $\mathbb{I}(g^1, g^2)$, let

with the base period $b=\min(g^1, g^2) - 1$, the two-event version of the propensity score $p_{g^1, g^2, t}(X) \equiv P(G^1_{g^1} = 1, G^2_{g^2} = 1|X, G^1_{g^1}G^2_{g^2}+(1-D_t)(1-G^1_{g^1}G^2_{g^2}) = 1)$, inverse probability weights $w^{comp}(g^1, g^2, t) \equiv \frac{p_{g^1, g^2, t}(X)(1-D_{t})(1-G^1_{g^1}G^2_{g^2})}{1-p_{g^1, g^2,t}(X)}, $ and outcome's control function $m^{ny}_{g^1, g^2, t} \equiv \mathbb{E}[Y_t - Y_{b}|X, D_t = 0, G^1_{g^1} = 0, G^2_{g^2} = 0]$. Therefore, their values are the same in either the full sample $\mathbb{I}$ or the subsample $\mathbb{I}(g^1, g^2)$.

By showing that these moments are equivalent to CS's corresponding moments in the subsample $\mathbb{I}(g^1, g^2)$, Theorem 1 in CS provides identification of the double-group-time average treatment effect:

Note that these moments use none of the units excluded from $\mathbb{I}(g^1, g^2)$, i.e., units first treated by either on or before $\min(g^1, g^2)$ but yet are not in cohort $(g^1, g^2)$. With the confounded controls addressed, the remaining problem is the confounded treated group.

Double DiD and Parallel Treatment Effects

The second stage DiD addresses the problem of confounded treated units. The combined treatment effect $ATT$ identified by the first DiD (Theorem (ref)) does not always equal the target $ATT^1$ because it switches depending on the order of the events. By Assumption (ref) (no anticipation), we have

In periods after event 1 and before event 2, $t\geq g^1, t<g^2$, we directly observe the target parameter $ATT^1$. However, if the unit is also confounded ($t\geq g^2$), we observe instead the combined treatment effect $ATT$. Therefore, our goal is to impute these missing treatment effects with a new parallel trend assumption. I again focus on the `not-yet` version:

This assumption sets units simultaneously treated but not yet confounded as the control group. By only using the units in the same own-cohort, I'm allowing the target treatment effect ($ATT^1$) to vary freely across the treatment cohort ($g^1$), and assuming it to be parallel across the confounding cohort ($g^2$). For the confounding treatment effect its the converse: free across confounding cohort and parallel across treatment cohort.

Assumption (ref) plays a similar role to the common trend assumption (Assumption 4) in de-Chaisemartin2023-jb. Their assumption restricts the outcome trend to be parallel conditional on the treatment status from the last period ($D^1_{t-1}, D^2_{t-1}$). This means their control group is all units with the same treatment status in the last period whereas I restrict it to the units in the same own-cohort. Being more selective with control groups means that my estimator will be robust to treatment effect heterogeneity across cohorts. Caetano2024-dn focus on covariates in DiD and doesn't explicitly discuss confounding events. Still, their method can control for confounding events by viewing them as a time-varying covariate. In such case, since their framework extends to CS', full cohort and time heterogeneity for the target treatment is allowed. Yet how the estimator behaves under treatment effect heteroegenity of the confounding event is not discussed.

Finally, for additive imputation, I need to shut down the interaction between the treatment effects:

I have two strategies for identifying the target treatment effect depending on the order of the events. First, the DiD strategy (see Figure (ref) for example) is for units first treated then confounded ($g^1 > g^2$). I use the units in the same event 1 cohorts but not yet confounded by event 2 ($s^2>t$) as the control group and the period before event 2 ($b = g^2 -1$) as the base period. Then, a DiD on the treatment effects identified by Theorem (ref) can be used to identify the target treatment effect. Given a identifying moment $ATT_{id}$ such that $ATT = ATT_{id}$, let

The second is imputation strategy (see Figure (ref) for example) for units first confounded by event 2 and then treated by event 1 ($g^1 > g^2$). I use units in the same event 2 cohorts but not yet treated by event 1 ($s^1>t$) as the control group, and the period before event 1 ($b = g^1 -1$) as the base period. Then, the difference in the combined treatment effect trends between the control and treatment groups identifies the target treatment effect.

Combining these two strategies and the direct identification for periods before the confounding event, we can identify the target treatment effects $ATT^1$ for units with the discrepancy in treatment timings ($g^1 \neq g^2$) in periods with suitable control groups:

Similar to the pre-trend in a single-event setup, the treatment effect in periods before event 1 $t < g^1-1$ is overidentified due to the parallel trend assumption. Hence the estimator can be estimated at the pre-periods to diagnose potential violation of the parallel treatment effects assumption. One limitation is that for units treated and confounded in the same period ($g^1 = g^2$), we cannot identify the target treatment effect due to the lack of a base period such that we observe only one effect.

\FloatBarrier

Aggregation, Estimation, and Inference

Given the connection in identification, the aggregation, estimation, and inference results are also similar to CS. To aggregate the double-group-time target treatment effects from the second stage, each summary parameter $\theta$ is associated with a weighting scheme, $w(g^1, g^2, t)$ such that

For example, to obtain the group-time average treatment effect, $ATT^1(\tilde{g}^1, t)$, there is the weighting scheme

where $A(g^1, g^2, t) \equiv \mathbbm{1}\{g^1<g^2\}\mathbbm{1}\{g^1 \leq t < \bar t^2\}+\mathbbm{1}\{g^1 > g^2\}\mathbbm{1}\{g^1 \leq t < \bar t^1\}$ is the indicator for availability of suitable control group. With the group-time parameter, the other aggregated parameters can be done using the summary scheme in CS.

An additional aggregation scheme is the dynamic-staggered heterogeneity, i.e., how the treatment effect dynamics $e^1 = t-g^1$ differ by the timing difference of the two events $s^{12}$. For $\theta_{e^1, s^{12}} = \mathbb{E}[Y_t(g^1, 0)- Y_t(0,0)|t-g^1 = e^1, g^1 - g^2 = s^{12}]$, we can use the weighting scheme

For estimation, the first stage moments from Theorem (ref) can be estimated with sample plug-ins. Following CS, I focus on the doubly-robust version since it has the benefit of additional robustness to misspecification and the arguments apply analogously to other parameters. For a generic $C \in \{0,1\}, Z$, let $\mathbb{E}_{n}[Z|C]$ denote the conditional summation $(\sum_{i=1}^NC_i)^{-1}\sum_{i=1}^NC_iZ_i$. Let

where

and $\hat{p}(\cdot), \hat{m}(\cdot)$ is some estimators of ${p}(\cdot), (m)(\cdot)$.

For the second stage moments in Theorem (ref), the formulation in Equation (ref) and (ref) means that we can treat them as a summary parameter of the first stage estimates and aggregate them as weighted summations similar to Equation (ref).

Since the double-group-time average treatment effect estimators are equivalent to CS' estimators for group-time average treatment effects in $\mathbb{I}(g^1, g^2)$ (for example, Equation (ref) corresponds to Equation 4.2 in CS), and the double DiD estimators and other summary estimators are their aggregations CS' inference procedure in Section 4.1 applies. In summary, by assuming that the control function and propensity score can be estimated by smooth parametric models with $\sqrt{N}$ linear representation (CS Assumption 7) and some weak integrability conditions that ensure the second moments are finite (CS Assumption 8), CS Theorem 2 provides the influence function for calculating the standard error and its asymptotic normality. Then, CS Theorem 3 establishes the validity of the multiplier bootstrap for calculating the standard errors. The estimator is implemented in the R package fastdid. \FloatBarrier

Application: Minimum Wage and Teen Employment

To illustrate the empirical relevance of omitted event bias, I revisit the effect of minimum wage on teen employment with the DiD design using staggered timing of minimum wage increases across the states in the U.S. Neumark2006-re, Dube2016-pr, Cengiz2019-vr, Callaway2021-vo. My specification largely follows Callaway2021-vo. I define treatment cohorts by the period when a state first increased its minimum wage and use the other states that do not increase the minimum wage as control groups (never-treated). Our specifications have two differences. First, to include the confounding event, Medicaid Expansion under the ACA, I use data from 2010 to 2020 instead of from 2001 to 2007. Similar to 2001 to 2007, the federal minimum wage is fixed at \$7.25 per hour during this period. Second, CS clustered their standard errors at the county level, whereas I clustered them at the state level, which is the level of the treatment as recommended in Abadie2023-yl.

I construct a state-year level dataset by linking several data sources, including state minimum wage history compiled by Sorens2008-ek, the timing of Medicaid Expansion from KFF2024-cb, teen employment from Quarterly Workforce Indicators Abowd2009-ul, and other state characteristics from University-of-Kentucky-Center-for-Poverty-Research2024-su. I define yearly teen employment as the logarithm of the average count of employed teenagers (age 14-18) in the state across four quarters. Alaska, Mississippi, and Massachusetts are removed from the sample due to missing observations on teen employment from the QWI. Table (ref) reports the summary statistics of states by treatment. We can see that the states that increased minimum wage are also more likely to adopt Medicaid expansion by 2020, which may be explained by the different political leanings proxied by whether the governor is a Democrat.

Figure (ref) reports the correlation between minimum wage increase and Medicaid expansion using the diagnostic in Equation (ref). States that increased the minimum wage also tend to adopt the Medicaid Expansion. For example. in a staggered DiD design with unconditional estimators, the estimates for the first year after the increase have a bias with the size of \DTLfetch{vartable}{name}{bias_short}{value} % of the Medicaid Expansion effect. Overall, the average bias across all post-periods is \DTLfetch{vartable}{name}{bias_avg}{value} % the Medicaid Expansion effect. Alarmingly, the correlation was not significant before the minimum wage increase, suggesting that pre-trends would not have been a good diagnostic in this case.

Figure (ref) reports the short-term effects of minimum wage on teen employment, I focus on the year after the increase because treatment in the middle of the year attenuates the effect in the first year. With a staggered DiD with CS' unconditional estimator (“naive” specification), I find negative effects on teen employment, similar to CS and Dube2016-pr. However, when I use the Double DiD estimator to control for the expansion (“double” specification), the negative effect in the first year after the increase weakens from \DTLfetch{vartable}{name}{short_naive}{value} p.p. to \DTLfetch{vartable}{name}{short_double}{value} p.p. and turns statistically insignificant. Summarizing all post-increase periods, the overall effect changed from \DTLfetch{vartable}{name}{post_naive}{value} p.p. to \DTLfetch{vartable}{name}{post_double}{value} p.p., but both are insignificant. This suggests that the negative effect of the “naive” specification is spurious due to the omitted event bias from the Medicaid Expansion.

Figure (ref) shows that this negative bias is robust to alternative specifications. Allowing for 1 year of anticipation, and adding not-yet-treated units as control does not change the result qualitatively, albeit the statistical significance fluctuates due to the limited sample size at the state-year level. Interestingly, controlling for whether the Governor is a Democrat in 2010 does not change the result at all, suggesting that the confoundedness goes beyond the ruling party of the state. Finally, switching to using only ever-treated units substantially weakens both estimates, suggesting that the confoundedness is most severe among states that never increased minimum wages nor expanded Medicaid.

This spurious negative effect of minimum wage on teen employment does not necessarily contradict previous empirical findings. Aside from using different periods, the empirical literature is aware of the potential bias in an unconditional staggered Difference in Difference estimate Allegretto2017-am, and often controls for many factors in their specification. This is very likely to weaken the strength of the omitted event bias. Additionally, other unaccounted state policies can confound the Medicaid Expansion as well, creating more bias. Therefore, my takeaway from this empirical exercise is that in an important application of the DiD design, the omitted event bias matters.

Discussion

In this paper, I show confounding treatments with correlated timing create bias in the standard staggered DiD estimators and provide a solution that allows for rich treatment effect heterogeneity. To eliminate this omitted event bias, I address the problem of using confounded units as controls by using only neither-yet-treated units. For the problem of a confounded treated group, I propose a second DiD design that employs a parallel treatment effect assumption. I also show that for the effect of minimum wage on teen employment, the Medicaid Expansion creates a spurious negative effect. Work for future research includes incorporating other advancements in the new DiD literature, a conditional parallel trend assumption for the treatment effect.

\FloatBarrier