Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
73,172 characters · 14 sections · 68 citation commands
Causal Duration Analysis with Diff-in-Diff
Many important topics in applied economics involve durations. To name a few, the impact of unemployment insurance on the length of unemployment spells (Katz1990, Hunt1995, LALIVE2006, Lalive2007, Card2007, Chetty2008, Schmieder2012, Schmieder2016, Lichter2021, and others), the effect of divorce laws on marriage duration (Friedberg1998a, Gruber2004, Wolfers2006), the strength of residency rules on the rate at which refugees pass language tests (Arendt), and the consequences of criminal justice policies on the rate of recidivism or probation revocation (Schmidt1989, Bhuller2020, Rose2021). In settings like these, available data often consist of panels in which the outcome is a binary indicator that an individual has entered an absorbing state. For example, an indicator that an individual has exited unemployment by a particular date. Difference-in-differences is a popular tool for policy-evaluation with panel data, but when the data take the form just described, the parallel trends assumption generally fails.
To fix ideas, consider the case in which the outcome indicates exit from unemployment. Suppose that some individuals receive an increase in unemployment benefits at a particular point in time while others do not. The foundational assumption of diff-in-diff is that, absent the policy of interest, the difference in mean outcomes between the treated and untreated groups would remain fixed. If a sufficiently large share of individuals eventually exit unemployment, then mean outcomes will tend to converge over time, even absent any treatment effect.\footnote{Apart from in the special case in which mean outcomes are identical between the two groups.} This entails a failure of parallel trends and may result in severely biased and inconsistent estimates of treatment effects.
In response, we consider alternative approaches based on insights from duration analysis. Rather than assume parallel trends between mean potential outcomes, we suppose that the difference between group-specific counterfactual hazard rates is constant. This condition is consistent with the convergence of counterfactual mean outcomes over time and thus the convergence does not imply inconsistency of the corresponding causal estimates. The alternative assumption motivates a simple fix. Instead of performing diff-in-diff using mean outcomes, we apply the same procedure using a particular function of the mean outcomes and the time period. We sidestep any estimation of the hazard rates themselves, and our analysis entirely avoids the need to specify an explicit model for the hazard function.
To be precise, in place of the mean outcome $E[Y_{i,t}|G_i=k]$, where $t$ indexes time periods, $i$ individuals, and $G_i$ is the group to which individual $i$ belongs, we use the `time-average hazard', denoted $\bar{H}_{k,t}$, which can be written in terms of mean outcomes. This object is the average of the hazard rate over the continuous period between times $1$ and $t$. Under our assumptions there is a fixed difference between the time-average hazards of the different groups under the no-treatment counterfactual. Thus we can perform difference-in-differences using this alternative object. We may then invert the definition of the time-average hazard to recover counterfactual mean outcomes and average treatment effects.
Note that the counterfactual mean outcome, understood as a function of time since the start of a spell, is precisely the cumulative distribution function of counterfactual durations. Thus our approach can produce estimates of the whole distribution of counterfactual durations, rather than say, the mean duration under a no-policy counterfactual.
Diagnostics like placebo tests and the standard test for pre-treatment parallel trends can be adapted straight-forwardly to our setting. Where standard tests use the mean outcome, we again use the time-average hazard rate. Analogous with vanilla diff-in-diff, one can perform informal visual inspections to assess whether parallel trends in time-average hazards holds in the pre-treatment period.
Adjusting for covariates can be important for credibly identifying causal effects in difference-in-differences. If there is covariate imbalance between treatment and control groups in diff-in-diff, then parallel trends may fail, even if it holds within each covariate stratum (say, within each demographic subgroup). We adapt our hazard diff-in-diff approach to allow for such cases by employing a simple propensity score weighting when we calculate the time-average hazard. This avoids the difficulties of partial likelihood estimation of proportional hazard models with discretised time periods.
Our approach extends beyond the assumption of a fixed difference between hazard rates. Rather, we can accommodate any fixed linear relationship between the hazard rates of different groups. Thus our approach applies if there is a fixed difference in log hazard rates, which is equivalent to a particular proportional hazards specification. In the case of multiple untreated groups we can obtain a duration analogue of the triple differences estimator and of the synthetic control method. As with our duration diff-in-diff approach, estimation differs from standard approaches only in the use of transformed mean outcomes.
We apply our methods to the setting of LALIVE2006. The authors in that study evaluate the impact of a policy that increased the generosity of unemployment insurance benefits for unemployed Austrian workers. The authors identify causal effects by exploiting the presence of individuals who were ineligible for the benefit changes. They estimate a flexible parametric duration model and from their estimates they recover causal effects.
In contrast, we identify causal effects using a cross-cohort comparison. We employ our methods in order to estimate the impact of an extension to the potential benefit duration (PBD). We adjust for the calendar date at which an unemployment spell begins using our covariate re-weighting strategy. Thus our estimates are robust to differential trends in job-seeking between individuals who become unemployed in different parts of the year. We obtain similar results to the authors of the original study, however we do so while avoiding estimation and specification of the hazard function, and without the need for numerical optimization of a likelihood. Moreover, the transparency of our approach allows us to both visually and formally assess whether there are parallel trends over the pre-treatment period. We find a statistically significant positive impact of PBD on unemployment duration with strongly positive estimates shortly following treatment which then taper off. We are unable to reject parallel trends in hazard rates even at the $50\%$ level.
We evaluate the finite-sample performance of our methods in a simulation study. The results also demonstrate the potential for standard diff-in-diff to produce severely misleading estimates in duration settings. The simulation results are available in Appendix A.
In sum, the present paper suggests a simple means of adapting existing difference-in-differences and synthetic control methods to settings with duration data. Whereas standard procedures extrapolate the relationship between mean outcomes for different groups forward in time, we instead extrapolate the relationship between these groups' time-average hazard rates, which are known functions of mean outcomes and therefore easily estimated from the data. What is crucial here is that we retain the intuitive appeal of diff-in-diff. One of the key benefits of diff-in-diff and related methods is that one can visually assess whether the assumption of a stable linear relationship holds in the pre-treatment period, and assess the magnitude of any deviation from this condition. Because we simply shift the objects to which these methods are applied, our approach allows researchers to perform similar visual inspections, as required for effective causal event studies.
We are not the first to suggest an extension of differences-in-differences to duration settings. Proportional hazard models (Cox1972) with a diff-in-diff-type linear index are considered by Hunt1995, Wu2022, and LALIVE2006. Wu2022 show that parallel trends cannot hold when the data are generated by the proportional hazards model. Hunt1995 suggests an approximate partial likelihood estimation procedure for the coefficients of the linear index in a proportional hazard diff-in-diff specification, and LALIVE2006 employ maximum likelihood estimation. Wu2022 consider estimation in the two-period case. In both Wu2022 and Hunt1995, interest is in the estimation of the coefficients on the binary indicators in the linear index, rather than average treatment effects.
The proportional hazard diff-in-diff model is a special case of the more general linear restrictions that we consider in the present work. Our approach avoids maximum likelihood estimation in favor of simple and transparent imputation of time-average hazards that closely resembles classic diff-in-diff. The simplicity and transparency of our approach is important not only in that it facilitates practical application of our methods, but because it allows us to both visually and formally test for pre-treatment parallel trends, much as in standard diff-in-diff. Our preferred specification assumes a fixed level difference in the hazard rates rather than a fixed ratio. This has the advantage of allowing for a simple means of flexibly incorporating covariates via propensity score weighting.
Also related to our approach is the literature on non-linear difference-in-differences. These papers, usually motivated by a limited dependent variable, specify a generalized linear model (GLM) in which the outcome is a non-linear transformation of a linear index with a diff-in-diff form. A number of empirical papers estimate the coefficients in a GLM diff-in-diff model. For example, Gruber1994 and Feldstein1996. Puhani2011 considers the interpretation of the coefficients in these models.
Motivation for GLM diff-in-diff models is discussed in Blundell2004, Blundell2009, Lechner2011, and Wooldridge2023. The general strategy in Blundell2004 and Wooldridge2023 is to transform mean outcomes by inverting the link function in a GLM and to perform diff-in-diff on the transformed means. Re-applying the link function then recovers counterfactual mean outcomes and thus treatment effects. Their specifications are motivated by latent variable models for the discrete outcomes. Thus in those works, the link function is determined by an a priori assumption that an unobserved noise term follows a known parametric probability distribution. In our case, the form of link function follows from the duration structure of the data. Works in which the link function is estimated using sufficiently rich pre-treatment data include Ashenfelter2004 and Athey2006. The latter suggest a nonparametric approach that is valid under weaker conditions than the GLM diff-in-diff model.
In addition, we differ from Wooldridge2023 in the manner in which we adjust for covariates. Rather that incorporate covariates in the linear index inside of a link function, we instead suggest a propensity score weighting approach that is valid under the assumption of parallel trends in covariate stratum-specific hazard rates but allows trends to differ between strata.
By proposing a simple and transparent extension of diff-in-diff to duration settings, we extend the ever-growing literature on difference-in-differences. For recent surveys see for example Roth2023 and Chaisemartin2023.
In our empirical application we employ a cross-cohort comparison in which the time period for a given individual is relative to the start of their unemployment duration. This is the same approach taken in Berg2020. However in that work, the authors employ an identification strategy based on regression discontinuity design for nonparametrically estimated hazard rates.
There is a sizable literature on causal analysis using duration data (see e.g., Abbring2003 and Abbring2007). In the seminal paper of Abbring2003 and similar works, researchers achieve identification under assumptions on the treatment process, its relation to individual heterogeneity, and the separability of unobserved heterogeneity in individual-level hazard rates. Our approach differs from this in its foundation. Rather than begin with an individual-level duration model incorporating heterogeneity, our analysis is premised upon an assumption concerning group-level hazard rates. This is closer in spirit to standard diff-in-diff which, while sometimes written as an individual-level model with two-way fixed effects, can be understood as a method for imputing mean counterfactual outcomes under a group-level parallel trends assumption. An advantage of the group-level approach is that it can accommodate settings in which there may be substantial spill-overs and interaction effects among individuals in the same group, and in addition it applies to settings in which only group-level data is available.
In some cases it may be possible to apply difference-in-differences using the durations themselves as outcomes as in Lichter2021. Consider again the unemployment example and suppose some individuals are ineligible for the increase in benefits. One might perform diff-in-diff by taking the difference in the mean unemployment durations between those who became unemployed before and after the reform and seeing how this varies with eligibility for the benefits increase. This approach differs from ours both in the settings to which it is applicable and in the causal objects it identifies.
First, consider that this alternative strategy identifies causal effects using variation in the start date of unemployment spells, whereas our method is applicable even if all spells in the data begin on the same date. In addition, suppose that the policy change applies to those in ongoing unemployment spells, as is common in practice. Then the group of eligible individuals who became unemployed prior to the reform will contain some treated individuals, and dropping these from the sample would lead to selection bias. Our approach avoids this problem.
Using durations as outcomes identifies distinct causal objects compared with our approach. Suppose that the policy impacts not only time spent in unemployment, but also who becomes unemployed. This compositional change represents an additional channel through which the policy might impact unemployment durations. Because we follow the same fixed set of individuals over time, we isolate the behavioral response of individuals in ongoing spells. Finally, our approach recovers a counterfactual cumulative distribution of durations evaluated at certain points in its support, whereas the alternative identifies the counterfactual mean of durations.
We sample binary outcomes $Y_{i,t}$ for individuals $i=1,...,n$ at periods $t=1,..,T$. Each individual belongs to a group $G_i$ where group membership is constant over time. An outcome of $1$ indicates that an individual has entered an `absorbing state' and so all future outcomes for that individual are also equal to $1$. Individuals in group $1$ receive an intervention at some point between periods $t^*-1$ and $t^{*}$ and we wish to assess the impact of this intervention on the evolution of the outcomes of individuals in that group.
Note that the data need not contain the exact lengths of spells. Rather, we only need to know whether or not a spell has ended by a particular length of time. For example, if the time increments are weeks, then we need only know whether or not a given duration has ended by the $t$-th week for $t=1,...,T$, not the exact moment at which it ended. While we focus on the case of discrete time increments, our approach can be adapted straightforwardly to accommodate time increments of varying lengths.
In order to define causal effects of the treatment, we consider a counterfactual in which there is no intervention on group $1$. We denote by $Y_{i,t}^{(0)}$ the outcome under this counterfactual at time $t$ for individual $i$. We sometimes refer to $Y_{i,t}^{(0)}$ as an `untreated potential outcome', although this differs from the standard definition in that the counterfactual is defined in terms of an intervention on group $1$ rather than an individual-level treatment. We use the superscript `$(0)$' to indicate counterfactual values throughout this work.
Our primary object of interest is the time-$t$ average treatment effect for individuals in the treated group where $t^{*}\leq t$. This is the average difference between the outcome for a randomly sampled individual in the treated group $1$ at time $t$, and that individual's outcome in the counterfactual world in which there is no intervention. This is defined as follows: \[ \tau_{t}=E[Y_{i,t}-Y_{i,t}^{(0)}|G_i=1] \]
Note that $E[Y_{i,t}^{(0)}|G_i=1]$ is the counterfactual cumulative distribution function (CDF) of the durations of individuals in group $1$ evaluated at $t$. Therefore, if treatment effects are identified, then so too are the values of the counterfactual CDF of durations evaluated at each discrete time increment.
To illustrate, consider two schools $1$ and $2$. We sample $n$ students each of whom attends one of the two schools. If $G_i=k$ then individual $i$ attends school $k$. Data is available from periods $t=1,...,T$. The students in each school have the opportunity to sit and pass an English proficiency exam. If a student $i$ has passed the exam by time $t$ then $Y_{i,t}=1$ and otherwise $Y_{i,t}=0$. We suppose that at a time between $t^*-1$ and $t^{*}$, the students in school $1$ receive some educational intervention, where $t^*$ is known.
The binary indicator $Y_{i,t}^{(0)}$ is equal to $1$ if and only if student $i$ would have passed the test by time $t$ in a counterfactual world in which there is never any intervention on school $1$. In this setting, the time-$t$ average treatment effect $\tau_{t}$ is the difference between the share of the student population in school $1$ who have passed the test by time $t$ versus the proportion who would have passed under the counterfactual in which there is no intervention on school $1$.
Assumptions 1 and 2 formally impose some elementary properties of the factual and counterfactual outcomes implicit in the discussion above.
\theoremstyle{definition} \newtheorem*{A01}{Assumption 1 (Absorbing State)} \begin{A01} $Y_{i,t}$ is a binary random variable and $Y_{i,t}=1$ implies $Y_{i,s}=1$ for all $t\leq s$. The same holds for the potential outcomes $Y_{i,t}^{(0)}$. \end{A01}
\theoremstyle{definition} \newtheorem*{A02}{Assumption 2 (No Anticipation/Spill-Overs)} \begin{A02} i. $1<t^*$ and for all $t<t^{*}$, $Y_{i,t}=Y_{i,t}^{(0)}$, ii. For all $t\geq t^{*}$, if $G_i>1$ then $Y_{i,t}=Y_{i,t}^{(0)}$. \end{A02}
Assumption 1 states that having an outcome of $1$ is absorbing state. This means that if an individual has an outcome of $1$ at time $t$, then that individual's outcome is equal to $1$ in all future periods, and similarly for potential outcomes. In the schools setting this follows simply from the definition of the outcome: a student cannot un-pass the exam.
Assumption 2 imposes conditions on potential outcomes that are standard in difference-in-differences. We assume that individuals do not anticipate treatment, and so observed outcomes in periods strictly prior to $t^{*}$ are identical to those in the counterfactual world in which there is no intervention. A no-anticipation condition was introduced into the literature on causal duration analysis by Abbring2003. In addition, we assume there are no spill-overs between groups. That is, the treatment of students in school $1$ has no impact on students in other schools. More formally, the potential outcomes of students in other schools under the no-treatment counterfactual are equal to their realized outcomes. However, this does not rule out spill-overs between individuals in the same group.
Difference-in-differences identifies causal effects under an assumption that there is a fixed level difference between the mean outcomes of the treated and untreated groups. Formally, this parallel trends condition imposes that there is some constant $c$ so that the following equation holds in all periods.
In the schools example, the condition above states that the difference in mean potential outcomes for the two schools is constant over time. Under Assumptions 1 and 2, the condition identifies the average treatment effect $\tau_{t}$ and motivates a simple estimator. Let $\bar{Y}_{k,t}$ be the sample average outcome for individuals in group $k$ at time $t$. That is, for each $t$ and $k$: \[ \bar{Y}_{k,t}=\frac{1}{n_k}\sum_{i=1}^{n}1\{G_i=k\}Y_{i,t} \]
Under Assumptions 1, 2, and parallel trends, an unbiased estimate $\hat{c}$ of $c$ is given below, where $\{\alpha_{s}\}_{s=1}^{t^{*}-1}$ are some positive weights that sum to $1$. \[ \hat{c}=\sum_{s=1}^{t^{*}-1}\alpha_{s}(\bar{Y}_{1,s}-\bar{Y}_{2,s}) \]
An unbiased estimator $\hat{\tau}_{t}$ of the time-$t$ average treatment effect is as follows. \[ \hat{\tau}_{t}=(\bar{Y}_{1,t}-\bar{Y}_{2,t})-\hat{c} \]
The estimator above can be expressed equivalently in terms of ordinary least squares regression and can be adapted to include covariates. A number of inference methods have been proposed in the literature. For example, the block bootstrap method of Bertrand2004. With two periods of data (so that $t^{*}=2$), the estimator above reduces to the simple difference-in-differences below: \[ \hat{\tau}_{2}=(\bar{Y}_{1,2}-\bar{Y}_{2,2})-(\bar{Y}_{1,1}-\bar{Y}_{2,1}) \]
Diff-in-diff is one example of a general class of methods that identify causal effects by assuming that there exists a fixed linear relationship between the counterfactual mean outcomes in different groups. That is, there exist parameters $W_1$,$W_2$,...,$W_K$ so that the following holds in all periods:
Imposing additional conditions on the coefficients yields alternative identification approaches. With two groups, difference-in-differences specializes the above by fixing $W_2=1$, in which case ((ref)) holds with $c=W_1$. Another case that fits into this framework is triple differences, which corresponds to $K=4$, $W_2=W_3=1$ and $W_4=-1$. This approach is applicable when the four groups correspond to different combinations of two binary characteristics, and only those with one of the four combinations is treated. For example, suppose individuals are drawn from two regions $A$ and $B$ and only individuals that satisfy eligibility criteria and live in region $A$ are treated. Then group $1$ may consist of eligible individuals in region $A$, group $2$ of eligible individuals in region $B$, group $3$ of ineligible individuals in region $A$, and group $4$ of ineligible individuals in region $B$. In this case the condition reduces to the following for some $c$.
Under Assumptions 1, 2, and ((ref)), treatment effects are identified so long as there are unique coefficients $W_{1}$, $W_{2}$, ..., $W_{K}$ that satisfy ((ref)) in all pre-treatment periods, subject to any additional a priori constraints on these parameters. The coefficients can be estimated by weighted least squares: \[ \{\hat{W}_{k}\}_{k=1}^{K}=\underset{\{W_{k}\}_{k=1}^{K}\in\mathcal{W}}{\arg\min}\sum_{t=1}^{t^{*}-1}\alpha_{t}(\bar{Y}_{1,t}-W_{1}-\sum_{k=2}^{K}W_{k}\bar{Y}_{k,t})^{2} \]
Having estimated the parameters $\{W_{k}\}_{k=1}^{K}$, the average treatment effect for $t\geq t^{*}$ may be estimated by plugging-in the coefficient estimates and observed average outcomes into the linear model: \[ \hat{\tau}_{t}=\bar{Y}_{1,t}-\hat{W}_{1}-\sum_{k=2}^{K}\hat{W}_{k}\bar{Y}_{k,t} \]
The parameter space $\mathcal{W}$ may incorporate constraints. For example, if $K=2$ then constraining $W_2=1$ yields precisely the diff-in-diff estimator specified earlier in this section. A number of variations on the above are available in the literature. Abadie2003 suggest a more general method for estimating the parameters $\{W_{k}\}_{k=1}^{K}$. In place of the mean outcomes in the objective they use vectors of group-specific covariates which may include average outcomes. Abadie2003 constrain the optimization problem so that ${W}_{1}=0$ and the remaining coefficients are positive and sum to $1$. Doudchenko suggest adding an elastic net penalty to the synthetic control objective.
A number of methods exist for selecting the weights $\{\alpha_{t}\}_{t=1}^{t^{*}-1}$. For example, Abadie2003 propose that the weights be chosen to minimize the objective above subject to the constraint that they are weakly positive and sum to $1$ and Abadie2015 suggest choosing $\{\alpha_{t}\}_{t=1}^{t^{*}-1}$ by cross-validation.
In the settings that we consider in this paper, the parallel trends assumption ((ref)) can be highly problematic. We consider cases in which the outcome is a binary indicator that an individual has reached an absorbing state by a given period. Therefore, the group mean outcome is the share of individuals in the group who have reached the absorbing state. For reasons that we describe below, these shares tend to converge over time, regardless of treatment. That is, the magnitude of the difference in group-mean outcomes tends to decrease, even if treatment has no effect. Standard diff-in-diff would erroneously interpret such a decrease as evidence of a treatment effect, leading to biased and inconsistent causal estimates.
Perhaps the most striking failure of parallel trends occurs when the shares in the absorbing state grow too close to $1$. Recall that the shares must be bounded above by $1$ and are weakly increasing over time. Consider the schools example and suppose that at some time prior to $t^{*}$, $60\%$ of students in school $1$ have passed and $40\%$ in school $2$, and by some time after $t^*$, strictly more than $80\%$ of students in school $2$ have passed. Under the assumption of counterfactual trends in mean outcomes, we must immediately conclude that there is a negative treatment effect, even before observing the post-treatment outcomes in the treated school. This is because no more than $100\%$ of students in school $1$ can pass the test, and so the gap in mean outcomes must be strictly lower in the post-treatment period than the $20\%$ gap in the pre-treatment period.
The example above is the result of a ceiling parallel trends places on the shares of individuals who can reach the absorbing state. Given that the shares are bounded above by $1$, it follows from ((ref)) that the counterfactual share who reach the absorbing state can never exceed $1+c$ in school $1$, and $1-c$ in school 2. Similar restrictions hold under other linearity assumptions of the form in ((ref)).
Convergence of mean outcomes is likely to arise even in cases where the shares do not reach the ceiling $1\pm c$. Consider that as the share of the population who have reached the absorbing state increases, there are fewer individuals left to enter the absorbing state. This means that if the share who have reached the absorbing state is high, the increase in mean outcomes will be small relative to the rate at which individuals who have not yet reached the absorbing state arrive at that state.
To demonstrate, consider the schools example and suppose that in school $1$, $80\%$ have passed the test by day $t$ and $60\%$ in school $2$. Thus the share who have not passed by day $t$ is twice as high in school $1$ as in school $2$. As such, for the difference in shares to remain the same up to the next period, students in school $1$ who have not passed by day $t$ must be twice as likely to pass on that day compared to those in school $2$. This effect strengthens as the shares who have reached the absorbing state grow. Suppose differences remain fixed and at some later period $t'$, $90\%$ have passed in school $1$ and $70\%$ in school $2$. For differences to remain fixed, the probability that a student in school $1$ who has not yet passed by day $t'$ passes that day must now be three times higher than for school $2$. We formalize this point at the end of this section by showing that the parallel assumption implies divergence of the counterfactual `hazard rates', that is, the rates at which individuals in the two groups who have not yet reached the absorbing state reach that state.
For a graphical illustration, observe Figure 1.1(a) which shows population mean factual and counterfactual outcomes under a duration model specified in Appendix A. We use this same model for our simulation exercises. Factual mean outcomes, i.e., shares of the population that have reached the absorbing state, are plotted over time by solid lines, blue for group $1$ and red for group $2$. The mean outcome for group 1 under a counterfactual of no treatment, is plotted by the dashed blue line.
In the figure, we see that mean outcomes for group $1$ are greater than for group $2$ in the initial period. Due to the duration nature of the setting, the difference decreases over time under the counterfactual of no intervention. In this example, the convergence is of a sufficient magnitude that the average difference in the observable factual outcomes is smaller over the post-treatment period than in the pre-treatment period. Therefore, in expectation standard diff-in-diff will estimate a negative treatment effect. However, the true treatment effect is positive, as seen by the the positive gap between the solid and dashed blue lines. In fact, in this model the bias of standard diff-in-diff is roughly four times the value of the true treatment effect. For simulation evidence of the poor performance of standard diff-in-diff under this model (and good performance of our proposed alternatives) see Appendix A.
The discussion above suggests that whatever assumption we use to identify and estimate causal effects should be compatible with the type of convergence shown in Figure 1.1(a). In addition, we should avoid assumptions that imply divergence between the hazard rates. As an alternative, we suggest analogues of the diff-in-diff assumption that apply directly to the hazard rates. Instead of assuming parallel trends in the shares who have reached the absorbing state, we instead assume parallel trends (or some other fixed linear relationship) in the hazard rates. Parallel trends in the hazard rates is consistent with the convergence of counterfactual mean outcomes. Indeed, the assumption holds in Figure 1.1(a) and thus the convergence in the figure does not result in misleading inference under this assumption.
Finally, we provide a brief formal argument that shows the parallel trends assumption implies divergence of counterfactual hazard rates. First we must define these objects, for concreteness we do so in the context of the schools example. Let $\Delta>0$ and consider the probability that, under the counterfactual of no intervention, an individual in school $k$ passes the exam between times $t$ and $t+\Delta$, conditional on having not passed prior to $t$. If we scale this probability by $1/\Delta$, then the limit as the increment $\Delta$ shrinks to zero is the counterfactual hazard rate for school $k$. We denote the counterfactual hazard rate by $h^{(0)}_{k}(t)$ where the superscript indicates that this is the hazard rate for the outcomes under the no-intervention counterfactual. It is defined formally as follows: \[ h^{(0)}_{k}(t)=\lim_{\Delta\downarrow0}\frac{P(Y_{i,t+\Delta}^{(0)}=1|Y_{i,t}^{(0)}=0,G_i=k)}{\Delta} \] We can define the factual hazard rate $h_k(t)$ analogously by replacing counterfactual outcomes with factual outcomes in the definition. In the schools example, the counterfactual hazard rate measures the rate at which students in school $k$ who have not passed the exam, pass the exam.
Suppose that $c$ is strictly positive and that for all $t$, we have $h^{(0)}_{2}(t)>0$ and $E[Y _{i,t}^{(0)}|G_i=2]<1-c$. Then under the diff-in-diff assumption ((ref)) we have that for any $t$: \[ \frac{h^{(0)}_{1}(t)}{h^{(0)}_{2}(t)}=\frac{1-E[Y _{i,t}^{(0)}|G_i=2]}{1-E[Y _{i,t}^{(0)}|G_i=2]-c} \]
The right hand side grows with $t$ and it goes to infinity as the mean counterfactual outcome for group $2$ approaches the upper bound $1-c$. So as the share in group 2 who have reached the absorbing state approaches this upper bound, the ratio of the hazard rates grows to infinity.
In the context of the schools example, as $t$ increases, the probability that a student who has not passed in school $1$ passes within the next $\Delta$ increment of time, must grow infinitely large compared to the same quantity for a student in school $2$. If $c$ is negative and $E[Y _{i,t}^{(0)}|G_i=1]<1+c$, then the hazard ratio shrinks to zero as $E[Y _{i,t}^{(0)}|G_i=1]$ approaches $1+c$.
A related point is made by Wu2022 who show formally that the parameter estimated by diff-in-diff must evolve over time when data are generated by a hazard model (apart from in certain degenerate cases).
The analysis above is concerning because it suggests that the hazard rates exhibit aberrant behavior as the proportion of individuals who have entered into an absorbing state grows large. The hazard rate is a primitive building block of duration models, and it has a clear structural interpretation. If the difference-in-differences assumption implies counter-intuitive and surprising behavior of the hazard rates, then we should reject the assumption a priori.
In light of the discussion above, we replace the standard parallel trends assumption, or a more general restriction of a fixed linear relationship between mean outcomes, with an analogous condition that applies directly to the hazard rates. By restricting hazards rather than mean outcomes, we ensure our modeling restrictions are consistent with the convergence of mean outcomes between groups. As such, the convergence of mean outcomes need not bias causal estimates based on our assumptions, in contrast to estimates from standard diff-in-diff.
Throughout, we continue to take $k=1$ to be the unique treated group. Our ky identifying assumption is that there exists a fixed linear relationship between the counterfactual hazard rates of the treated and untreated groups. Formally, we consider assumptions of the following form:
Imposing a priori constraints on the coefficients $W_1,..,W_K$ yields a range of alternative modeling assumptions. For example, in the two-group case, setting $W_2=1$ we get a duration analogue of standard difference-in-differences which is our preferred specification. We refer to identification and estimation based on this condition as `hazard diff-in-diff'.
The condition ((ref)) states that the level difference in hazard functions between groups $1$ and $2$ is constant over time. The condition allows for common shocks that impact the counterfactual hazards of individuals in both the treated and untreated groups, so long as these shocks induce parallel movements in the hazard rates. Moreover, the condition allows for the possibility that the hazard rates evolve over time, increasing or decreasing with the time spent in the $Y_{i,t}=0$ state.
If we instead assume $W_1=0$ and leave $W_2$ unrestricted then we recover the proportional hazard restriction that characterizes models in Hunt1995 and Wu2022:
The above restriction admits shocks to the counterfactual hazard rates and allows for the possibility that the hazards evolve over time. In this case the restriction is that the shocks and time trends leave the difference in the log counterfactual hazard rates unchanged.
The general linear restriction also nests a hazard analogue of the triple differences specification in Section 1.1. \[ [h^{(0)}_{1}(t)-h^{(0)}_{2}(t)]- [h^{(0)}_{3}(t)-h^{(0)}_{4}(t)]=c \]
We show that linear restrictions on the hazard rates allow for simple and transparent identification and estimation methods that are close analogues of the corresponding diff-in-diff and related methods.
We provide results that apply for general linear restrictions of the form in ((ref)). However, our primary interest is in the hazard diff-in-diff restriction ((ref)) and so we begin with this case. We suppose there are two groups $k=1,2$ and assume the level differences in hazard rates is fixed. The condition ((ref)), combined with Assumptions 1 and 2, identify counterfactual mean outcomes and average treatment effects. What follows is a sketch of the identification argument.
The first step is to rewrite the condition ((ref)) in terms of the time-average counterfactual hazard functions. Time-average hazards can be written in terms of mean outcomes and are directly identified from the data. We denote the time-average counterfactual hazard function for group $k$ at time $t$ by $\bar{H}^{(0)}_{k,t}$, and the time-average factual hazard by $\bar{H}_{k,t}$. The time-average hazard is, as the name suggests, the average over time of the hazard rate. Formally, these objects are defined as follows:
From the condition ((ref)) we obtain the equality below:
Key to our analysis is the observation that the time-average of the counterfactual hazard rate can be written in terms of mean outcomes. This follows from a standard result in duration analysis that the `cumulative hazard function' is equal to minus the log of the `survivor function'. For $t>1$ we have:
Note that the time-average factual hazard rate depends only on factual mean outcomes, and therefore it can be directly and straight-forwardly estimated from the observed outcomes. Moreover, under Assumptions 1 and 2, it is equal to the time-average counterfactual hazard when $k\neq1$, or when $k=1$ and $t<t^*$.
In order to identify average treatment effects, we perform difference-in-differences on the time-average hazards. Thus we obtain the time-average counterfactual hazards for the treated group in the post-treatment period. We can then recover the counterfactual mean outcomes in the post-treatment period by inverting ((ref)), and thus we identify average treatment effects.
This approach is illustrated graphically in Figure 1.1. In Figure 1.1(a) we plot group-specific mean factual and counterfactual mean outcomes over time from an underlying duration model that satisfies parallel trends in the hazard rates ((ref)). As we discuss in Section 1.2, the solid blue curve represents the factual group $1$ mean outcome, the solid red line is the group $2$ mean outcome, and the dashed blue line, the counterfactual group $1$ mean outcome. Figure 1.1(b) shows the corresponding time-average hazards in the same colors and styles.
The curves in 1.1(b) are related to those in 1.1(a) by the formulas ((ref)) and ((ref)). Note that while parallel trends clearly fails in 1.1(a), the counterfactual time-average hazards are parallel. By transforming the solid curves in 1.1(a) we could obtain the factual time-average hazard in 1.1(b). By extending the parallel trends in time-average hazards forward from the treatment date, we could then impute counterfactual time-average hazards for group $1$ in the post-treatment period, much as in standard diff-in-diff. Then, having imputed post-treatment counterfactual hazards, we can invert the transformation in order to recover the counterfactual mean outcomes for group $1$ in Figure 1.1(a).
The more precise identification result is given in Proposition 1 below. This result is a corollary of the more general Theorem 1 in the next subsection.
\theoremstyle{plain} \newtheorem*{P1}{Proposition 1} \begin{P1} Suppose Assumptions 1 and 2 and ((ref)) hold and define $\bar{H}_{k,t}$ as in ((ref)). Then for every $1<t<t^{*}$:
And for any $t^{*}\leq t$ the counterfactual mean outcome is given by:
The average treatment effect is then identified by: \[\tau_{t}=E[Y_{i,t}|G_i=1]-E[Y^{(0)}_{i,t}|G_i=1]\] \end{P1}
The characterization in Proposition 1 motivates a simple plug-in estimator. One can obtain estimates of the time-average hazards $\bar{H}_{k,t}$ by replacing the group-specific mean outcomes in ((ref)) with sample averages, and thus estimate both $c$, the group-1 mean counterfactual outcome $E[Y_{i,t}^{(0)}|G_i=1]$, and average treatment effects. We discuss estimation and inference in Section 3.
The equation ((ref)) shows that pre-treatment time-average hazard rates exhibit parallel trends. Pre-treatment time-average hazards are estimable and so we can visually inspect whether they appear to exhibit pre-treatment parallel trends or carry out a formal test This is akin to the parallel trends test in standard diff-in-diff.
We now turn to more general restrictions. We can identify counterfactual mean outcomes and treatment effects under an assumption of the form in ((ref)). Identification follows from similar steps to those in Section 2.1. The general linear restriction implies the following condition on the time-average counterfactual hazard rates.
The quantities in the equation above are all directly identified in the pre-treatment period. Thus we obtain $t^*-1$ equations from which we may identify the parameters $\{W_k\}_{k=1}^K$. If there is a unique solution then the parameters are identified. We can then identify counterfactual time-average hazards for the treated group in the post-treatment period. Inverting the equation ((ref)) we recover counterfactual mean outcomes and thus identify average treatment effects.
\theoremstyle{plain} \newtheorem*{P5}{Theorem 1} \begin{P5} Suppose Assumptions 1 and 2 and ((ref)) hold and define $\bar{H}_{k,t}$ as in ((ref)). Then for every $1<t<t^{*}$:
And for any $t^{*}\leq t$ the counterfactual mean outcome is given by:
Thus if ((ref)) has a unique solution, then both $E[Y^{(0)}_{i,t}|G_i=1]$ and $\tau_t$ are identified for all $t$. \end{P5}
Theorem 1 identifies treatment effects when the equations ((ref)) have a unique solution subject to any additional a priori constraints on the coefficients. Note that we must have at least three periods of data. Under the fixed level difference restriction in Section 2.1, uniqueness always holds. For the proportional hazards constraint it suffices that the following holds for some $t<t^*$: \[E[Y_{i,t}|G_i=2]\neq E[Y_{i,1}|G_i=2]\]
If we do not constrain the coefficients $\{W_k\}_{k=1}^K$, then a necessary condition for uniqueness is that $t^*>K$ and uniqueness is generic whenever this holds.
Given a sufficient number of pre-treatment periods, the parameters in ((ref)) are over-identified. This suggests we can test the identifying restrictions using say placebo tests or, in the special case in Subsection 2.1, a test for parallel trends.
In difference-in-differences analysis it is common to adjust for covariates. One motivation is that parallel trends may hold within each stratum of the covariates but not in the aggregate (see e.g., Abadie2005). For example, suppose each individual belongs to one of two demographic strata A and B. Outcomes are decreasing over time in stratum A but increasing in stratum B and individuals in stratum A are more prevalent in the treated group that in the untreated group. Then average outcomes in the treated group will tend to decrease over time compared to the untreated group, even if trends are parallel within each stratum.
The analysis in the previous subsection can be extended to cases in which condition ((ref)) holds only conditional covariates. One may simply apply the analysis separately within each covariate stratum. This would yield identification of conditional average treatment effects which could be aggregated to obtain average treatment effects. However, to apply that analysis to data would require estimation of conditional mean outcomes for each group and period.
Fortunately, in the special case described below, the problem can be straight-forwardly resolved by a form of propensity score weighting. Note that this weighting approach only applies to the hazard diff-in-diff strategy and not to the proportional hazard specification nor to general linear restrictions. The simplicity of incorporating covariates in hazard diff-in-diff is a major advantage over other specifications.
We suppose that there are two groups $k=1,2$ and for each individual $i$ we observe individual-specific and time-invariant covariates $X_{i}$. We can define group-specific counterfactual hazard rates within each stratum of the covariates as follows. Let $h_{k}^{(0)}(t;x)$ denote the group $k$ counterfactual hazard rate at time $t$ for the sub-population for whom $X_{i}$ is equal to $x$. \[ h_{k}^{(0)}(t;x)=\lim_{\Delta\downarrow0}\frac{P(Y_{i,t+\Delta}^{(0)}=1|Y_{i,t}^{(0)}=0,G_i=k,X_{i}=x)}{\Delta} \]
Suppose the hazard diff-in-diff assumption holds within each stratum of the covariates and moreover, that the level difference does not depend on the stratum. That is, the following equation holds for each $x$ in the support of the covariates:
If there is imbalance in the distribution of the covariates between the groups, then the condition above does not guarantee parallel trends for the marginal hazards ((ref)). However, by applying a weighting scheme we can recover covariate balance and identify treatment effects.
To achieve this, we define a weight function $\omega$ on the support of the covariates. In the case of continuous covariates one can replace the conditional probabilities with conditional probability densities and obtain a weighting of the kind introduced in DiNardo1996.
By weighting individuals in the untreated group by $\omega(X_{i})$ we down-weight those with values of the covariates that are more prevalent among survivors in group $2$ than group $1$ in the initial period, and up-weight those whose values were less prevalent. If the covariates have the same distribution across groups in the initial period, then the weight function reduces to $\omega(x)=1$.
The weighting function can be written equivalently in terms of a type of propensity score (Rosenbaum1983). Let $p(x)$ be the probability that an individual is in the treated group given they have covariate values $x$ and they are not in the absorbing state in the initial period. Formally, $p(x)$ is defined as follows: \[p(x)=P(G_{i}=1|Y_{i,t}=0,X_{i}=x)\] Then applying Bayes' rule we obtain the following:
With the weighting function $\omega$ in hand we can define the weighted time-average hazard $\tilde{H}_{2,t}$ for the untreated group.
Theorem 2 shows that an analogous result to Proposition 1 holds under the covariate-specific parallel trends assumption ((ref)). The only difference is the use of the weighted time-average hazard for the untreated group in place of the unweighted time-average hazard.
\theoremstyle{plain} \newtheorem*{P10}{Theorem 2} \begin{P10} Suppose Assumptions 1 and 2 and ((ref)) hold. Define $\bar{H}_{k,t}$ as in ((ref)) and $\tilde{H}_{2,t}$ as in ((ref)). Then for every $1<t<t^{*}$:
And for any $t^{*}\leq t$ the counterfactual mean outcome is given by:
\end{P10}
The identification results in the previous section motivate plug-in estimates of counterfactual mean outcomes and treatment effects. We first define an estimate $\hat{H}_{k,t}$ of the time-average hazard $\bar{H}_{k,t}$ as follows:
With the time-average hazard estimates in hand, the first result in Theorem 1 motivates regression estimates of the parameters $\{W_{k}\}_{k=1}^{K}$. Using data from the pre-treatment periods we regress the estimate $\hat{H}_{1,t}$ on the EstimatedTime-AverageHazards of the other groups. A weighted, and possibly constrained, least-squares estimator is given below where the weights are $\alpha_{t}$ for $t=2,..,t^*-1$ sum to unity.
The constraint set $\mathcal{W}$ can incorporate restrictions like positivity of the weights, or that the intercept $W_1$ is equal to zero. One can amend the weighted least squares objective to include a penalty (see Doudchenko for discussion in the non-duration context).
Having obtained parameter estimates, the second result in Theorem 1 motivates the following estimate of the time-$t$ average treatment effect: \[ \hat{\tau}_{t}=\bar{Y}_{1,t}-1+\big(1-\bar{Y}_{1,1}\big)exp\big(-(t-1)(\hat{W}_{1}+\sum_{k=2}^{K}\hat{W}_{k}\hat{H}_{k,t})\big) \]
In our practical application we use equal weights (i.e., $\alpha_2,\alpha_3,...$ all take the same value). More generally they may be chosen either to a) place greater emphasis on those periods that are closer to the intervention, or b) minimize the asymptotic variance of the estimates.
In the hazard diff-in-diff case examined in Section 2.1, the estimator reduces to the formula below, where $\hat{c}$ is an estimate of $c$. This corresponds to the imputation approach in Wooldridge2023.
In the proportional hazard model, we constrain $W_1=0$ and obtain the estimate of $W_2$, which we again denote by $\hat{c}$, and a corresponding treatment effect estimate. Both are given below.
In order to estimate treatment effects using the weighting scheme introduced in Section 2.3, one must first estimate the weighting function $\omega$. We suggest two alternatives.
The first estimator is based on ((ref)) and is appropriate for discrete covariates. In this case we simply replace the conditional probabilities in the formula ((ref)) with empirical frequencies. The estimate $\hat{\omega}(x)$ is defined for all $x$ for which there exists some individual $i$ in group $2$ for whom $X_{i}=x$ and $Y_{i,1}=0$, and is given below:
If some covariates are continuous or discrete with many support points, then we suggest an approach based on the propensity formulation ((ref)). Let $\hat{p}(x)$ be an estimate of $P(G_{i}=1|Y_{i,t}=0,X_{i}=x)$. One could obtain such an estimate using say, logistic regression of $1\{G_i=1\}$ on $X_{i}$ using the sub-sample of individuals for whom $Y_{i,1}=0$. We can then estimate the weights as follows:
Having obtained an estimate of the weight function, we use the following estimate of the weighted time-average hazard function in ((ref)) and ((ref)). \[ \hat{H}_{2,t}=\frac{1}{t-1}ln\bigg(\frac{1-\bar{Y}_{2,1}}{\frac{1}{n_2}\sum_{i=1}^n 1\{G_i=2\}\hat{\omega}(X_{i})(1-{Y}_{i,t})}\bigg) \]
Bertrand2004 propose the block bootstrap (Efron) for conducting inference in vanilla diff-in-diff. The asymptotic validity of the procedure rests on an assumption that the outcome histories of different individuals are independent. However, the outcomes of a given individual may exhibit arbitrary dependence over time. We suggest the use of the block bootstrap for inference in the duration settings in this paper. Implementation follows the same steps as for standard diff-in-diff with the distinction that the bootstrap samples are used to construct our estimator of the treatment effect rather than the usual diff-in-diff estimator.
To carry out block bootstrap inference, one independently resamples individuals uniformly with replacement and forms a new sample using the complete series of outcomes and covariates for each individual that is resampled. For example, if individual $i$ is sampled in the $b$-the bootstrap iteration, then that bootstrap sample will contain an individual whose outcome history is $Y_{i,1},Y_{2,i},...,Y_{i,t}$.
Having obtained block bootstrap samples, one may then evaluate bootstrap standard errors as well as pointwise and uniform confidence bands in the usual way. In particular, for each bootstrap sample $b=1,...,B$, one computes the estimate $\hat{\tau}_{t}$ using the bootstrap sample in place of the original data, and thus obtains a bootstrap estimate $\hat{\tau}^*_{b,t}$.
The standard deviation $\hat{\sigma}_{t}$ of $\hat{\tau}^*_{b,t}$ over the bootstrap samples $b=1,...,B$ is taken as the standard error for $\hat{\tau}_{t}$. To form pointwise confidence intervals for ${\tau}_{t}$, let $\hat{q}_{1-\alpha,t}$ be the $1-\alpha$-quantile of $|\hat{\tau}^*_{b,t}-\hat{\tau}_{t}|/\hat{\sigma}_{t}$. Then a $1-\alpha$-level confidence interval has the form below: \[ CI_{1-\alpha,t}=[\hat{\tau}_t-\hat{q}_{1-\alpha}\hat{\sigma}_{t},\hat{\tau}_t+\hat{q}_{1-\alpha}\hat{\sigma}_{t}] \]
The intervals described above are only designed to achieve correct pointwise coverage. Suppose we form confidence intervals for $\tau_t$ for each of the periods $t=t^*, t^*+1,...,T$. Each of these intervals is specified so that it covers the corresponding period's average treatment effect with probability approximately $1-\alpha$. However, the probability that every one of these intervals contains its corresponding period's treatment effect may be much lower.
In order to obtain a desired joint coverage probability, in place of the critical value $\hat{q}_{1-\alpha,t}$ defined above, we instead take $\hat{q}_{1-\alpha,t}$ to be the $1-\alpha$ quantile over $b=1,...,B$ of $\max_{t^* \leq s\leq T}|\hat{\tau}^*_{b,s}-\hat{\tau}_{s}|/\hat{\sigma}_{s}$ and otherwise form the confidence bands as above.
A more formal description of the pointwise and uniform inference procedures is given in Algorithm $1$ in the appendix.
The condition ((ref)) has testable implications. If the assumption holds in all periods then the difference in pre-treatment time-average hazards must be constant. This motivates a test analogous to the test for parallel trends in standard difference in differences.
In both hazard and standard diff-in-diff, we recommend that researchers proceed with caution when interpreting a failure to reject pre-treatment parallel trends. Even if parallel trends hold over the pre-treatment period this is no guarantee that they would have remained parallel into the post-treatment period in the absence of treatment.
To motivate the test, note that from ((ref)) and Assumptions 1 and 2, in all periods $t=2,3,...,t^*-1$ we have: \[ \delta_t=(\bar{H}_{1,t}-\bar{H}_{2,t})-(\bar{H}_{1,t^*-1}-\bar{H}_{2,t^*-1})=0 \]
In order to test whether the hazard rates are parallel in the pre-treatment period, we construct uniform (over $t=2,..,t^*-1$) confidence bands for $\delta_t$. In order to estimate $\delta_t$ we replace each population time-average hazard $\bar{H}_{k,t}$ with the corresponding estimate $\hat{H}_{k,t}$ specified in Section 2.1. We thus obtain an estimate $\hat{\delta}_t$: \[ \hat{\delta}_t=(\hat{H}_{1,t}-\hat{H}_{2,t})-(\hat{H}_{1,t^*-1}-\hat{H}_{2,t^*-1}) \]
We then construct uniform confidence bands for $\delta_t$ using the block-bootstrap analogous to the construction in the previous subsection. The test rejects if the uniform bands do not contain zero for all $t=2,...,t^*-1$. Note that if covariate adjustment is used then the weights given in Section 3.1 must also be evaluated for each bootstrap sample. The procedure is detailed in Algorithm 2 in the appendix.
An advantage of this method is that one can plot the estimates $\hat{\delta}_t$ and the corresponding confidence bands. Thus the researcher can observe the precision of the estimate $\hat{\delta}_t$ and can compare this to the magnitudes of the time-average hazard estimates themselves. Thus a researcher can make an informal assessment as to whether a failure to reject parallel trends is a consequence of imprecise pre-treatment time-average hazard estimates.
All of the procedures we have described can be written as generalized method of moments estimators or sequential generalized method of moments estimators. Thus, as the sample size grows (with $K$ and $T$ fixed) standard regularity conditions ensure the consistency of our estimates and asymptotically correct coverage of the bootstrap confidence intervals. These results are well-known and so, following the example of Wooldridge2023, we omit a formal statement here.
Nonetheless, it is important to note three caveats. First, standard inferential results require identification of the nuisance parameters, which means that there must be unique coefficients $\{W_k\}_{k=1}^K$ that satisfy ((ref)) subject to any priori constraint that $\{W_k\}_{k=1}^K\in\mathcal{W}$. Second, if this constraint is an inequality constraint and it binds, then treatment effect estimates may not be differentiable functions of mean outcomes. In this case, asymptotic normality generally fails and the bootstrap does not have correct coverage. Such settings may call for alternative inference procedures of the form described in Fang2019. Finally, for the estimation with covariate adjustment, regular estimation generally requires that the weights $\omega(X_i)$ be bounded above, in which case the propensity scores must be bounded below away from zero (see Khan2010).
We apply our methods to examine the impact of a policy change in Austria on the 1st of August 1989 that increased the generosity of unemployment benefits of eligible individuals. This setting was previously examined by LALIVE2006 and we use the data accompanying their paper. LALIVE2006 use the data to estimate a piece-wise constant proportional hazards model with a linear index that interacts eligibility for various aspects of the policy with an indicator that the policy change has occurred. We instead employ a cross-cohort study using our methods.
In our view, the primary benefit of our analysis over the original study is its simplicity and transparency. Our approach allows us to avoid specifying a particular parametric form for the hazard rates and numerical maximization of the corresponding likelihood. It allows us to visually assess the presence of deviations over the pre-treatment period from our foundational assumption of parallel trends in the hazard rates, and to test the assumption formally. In addition, we control for the calendar date at which unemployment spells begin using the our weighting approach. This ensures that our estimates are robust to differential trends between sub-populations who became unemployed on different dates.
Following the policy change, individuals aged 40-49 who had has been employed for at least 312 weeks out of the previous ten years became eligible for 39 weeks of benefits rather than the previous 30 weeks. Some individuals also qualify for a (modest) increase in the replacement rate, which is the proportion of expected earnings given to individuals receiving benefits. To simplify our analysis we consider only the increase in the potential benefit duration (PBD) rather than the change in the level of benefits. As such, we exclude all individuals who qualify for the change in replacement rate from our sample.
To construct our treated group, we collect all individuals in the data who qualify for the extension of PBD from $30$ to $39$ weeks, excluding those who qualify for the increased replacement rate. Of these individuals we retain only those who became unemployed at or prior to the reform date, and less than $30$ weeks after the reform date. The untreated group contains those individuals who became unemployed at or prior to one year before the reform date, and less than $335$ weeks before the reform date. The treated group consists of $4,058$ individuals and the untreated group $4,207$.
For each individual, the time period $t$ is taken to be the number of days since the beginning of their unemployment spell. Thus for an individual $i$, the period-$t$ outcome $Y_{i,t}$ is an indicator of whether that individual had exited unemployment at or prior to $t$ days after becoming unemployed.
We consider the treatment date to be the $30$ week mark. This is the point at which benefits end for those in the untreated group. Benefits for treated individuals last an additional $9$ weeks. Note that this may be problematic for the no-anticipation assumption. Individuals in the treated group were likely aware that their benefits would not end at $30$ weeks. Nonetheless, we note that any reduction in the pressure to search for employment prior to this date among the treated group would likely reduce the magnitude of our estimated treatment effects.
Given the importance of seasonal variation on job search, there may be differential trends in the rates of job-finding between individuals who became unemployed on different calendar dates. As we discuss in Section 2.3, such differential trends can be problematic if the distribution of the start dates of unemployment spells differs between the two cohorts. For this reason we apply the weighting scheme specified in ((ref)) to re-balance these distributions between the groups. This requires us to drop from the sample individuals who became unemployed on a date on which no untreated individual was made unemployed (there are $17$ such dates out of $240$).
Figure 4.1(a) plots the weighted mean outcomes for the two groups. Recall that the mean outcome is the proportion of individuals who have left unemployment by a given day into their spell. The red curve is the mean outcome for the untreated cohort and blue for the treated cohort. The series for the two cohorts are relatively close, with some apparent divergence immediately following day 210, which is marked with a vertical dashed line and is the point at which benefits expire for individuals in the untreated cohort. Over the period following the benefit extension the two curve eventually begin to converge. The rightmost vertical solid line marks day 273, at which time benefits expire for treated individuals.
Figure 4.1(b) contains the weighted time-average hazards for the two groups with the same color coding as in Figure 4.1(a). Note that these are logarithmic transformations of the values in Figure 4.1(a). We see that in the initial weeks following unemployment the time-average hazard rates for both groups increases steadily before stabilizing and declining. It then remains relatively steady for both groups following $30$ weeks with a modest decline for the untreated group.
There is very marked jump in the time-average hazard rate for untreated individuals following the end of their benefit period. This may suggest a sudden increase in job search intensity after unemployment benefits run out, or it may reflect a deliberate delay in the start date of new employment until exactly the date of benefit expiration. Notably, there also appears to be a slight increase for the treated group at this same period despite the benefits of these individuals persisting for an additional $9$ weeks. This may be explained by imperfect knowledge of the reform.
Following standard practice we apply our diff-in-diff approach using only periods within a relatively narrow window around the treatment date. We estimate the level difference between covariate weighted time-average hazards using the observations no more than eight weeks prior to the treatment date and we extrapolate forward no more than $125$ days after treatment. The point eight weeks prior to the treatment date is marked by the leftmost solid black vertical line in Figures 4.1(a) and 4.1(b).
Figure 4.2(a) shows the weighted time-average hazards over this period. The dashed line indicates the imputed values for the treated cohort under the no-treatment counterfactual. Note that within this window, pre-treatment parallel trends appear plausible.
Figure 4.2(b) plots the mean outcomes over this same window. Again, the dashed line plots the imputed mean outcomes for the treated group. The gray region gives the $95\%$ pointwise confidence bands. These were calculated using the bootstrap method described in Section 3.2. These are confidence bands for the average treatment effect added to the factual average outcomes for the untreated group.
In Figure 4.3(a) we plot the period-specific treatment effects. The treatment effect is strongly negative immediately following treatment but tends to decrease in magnitude over time. Notably, most of the steady reduction in magnitude follows the expiry of benefits for untreated individuals, which occurs at the time marked by the solid vertical line. The grays region are $95\%$ pointwise and uniform confidence bands. A close inspection reveals that the uniform bands do not contain zero at all periods, rather the upper band lies just below zero for a brief period shortly after treatment. That is, we find a statistically significant negative treatment effect.
Figure 4.3(b) visualizes our test for pre-treatment parallel trends. The green curve plots the difference in time-average hazard estimates relative to the value just prior to treatment. The vertical line indicates the treatment date. We see that this curve is strongly negative following treatment (in line with our negative treatment effect estimates), but is weakly positive prior to treatment. The $50\%$ uniform confidence bands contain zero over the entire pre-treatment period. This indicates a failure to reject pre-treatment parallel trends in the time-average hazards even at the highly conservative $50\%$ level.