Estimating Treatment Effects in Mover Designs

Estimating Treatment Effects in Mover Designs

}

Introduction

\setcounter{page}{1}

The rise of rich longitudinal data has broadened the scope for causal inference in economics. Rather than estimating a single cross-sectional treatment effect, researchers increasingly exploit variation in an individual's treatment choices over time -- such as the firm they work for, the city they live in, the teacher they learn from, or the doctor they are treated by -- in order to estimate a large number of causal effects simultaneously.\footnote{Recent examples include bdg12, chk_2013, jackson13, cfr:14a, fui2015, fgw16, sacarnyjmp, fghw17, molitor17, alcottetal17, and chetty/hendren:15.} Often these effects are obtained from a linear two-way fixed effects regression, motivated by a static, additive, and constant-effect model (e.g. akm99). It is usually not clear, however, what these sorts of “mover regressions” capture under misspecifications of the model, including heterogeneous treatment effects, outcome persistence, and time-varying shocks.

This paper explores the causal content of mover designs in a treatment effects framework, relaxing the canonical regression model with familiar quasi-experimental restrictions. In the simplest case of a binary treatment, two time periods, and no additional controls, I show that a mover regression identifies a weighted average of four difference-in-difference comparisons, and is therefore causally interpretable under a restriction on outcome persistance and a standard parallel trends assumption. This result links mover analyses to simpler quasi-experimental designs, but does not easily extend to settings with multiple unordered treatments. I show that, in general, mover regressions need not identify weighted averages of heterogeneous causal effects under parallel trends alone; rather they require additional restrictions on potential outcome heterogeneity across both treatments and time.

Motivated by these results, I develop a class of two-step mover average treatment effect (MATE) estimators for quasi-experimental mover designs. The estimators can be thought to extend the semiparametric difference-in-difference approach of abadie_05 to settings where individuals move both into and out of multiple treatments over time. Identification follows from conditional-on-covariate restrictions on trends and treatment effect heterogeneity, which are satisfied by a partially-separable model of dynamic potential outcomes. Certain MATEs are identified without direct restrictions on potential outcome persistence. The key effect homogeneity assumption permits extrapolation across observably-similar individuals from the many difference-in-difference comparisons embeded in mover designs and generates a large set of overidentifying restrictions. I characterize efficient MATE estimators and omnibus specification tests of these restrictions, both of which are straightforward to compute given a set of first-step propensity score estimates.

This analysis contributes to a small but growing econometric literature relaxing the canonical assumptions of two-way fixed effect regressions, typically as applied to matched worker-firm panels. ams15, for example, propose tests of the additivity restriction and develop a latent class model of non-random worker movement, while hlm17 leverage structural assumptions and long-run time series variation to estimate worker and firm effect ranks. Most recently, blm17 show how to accommodate discrete heterogeneity and Markovian job search patterns with certain forms of endogeneity. To my knowledge no paper has yet to study mover designs in a treatment effects framework, though in applications researchers sometimes appeal to the essential logic of parallel trends.\footnote{For example, fgw16) state their identifying assumption as the restriction that “trends do not vary systematically with the migrant's origin and destination.”}

This paper can also be thought to generalize classical and recent approaches to difference-in-differences estimation -- including ashcard85, hit_97,hist_98, abadie_05, and atheyimbens06 -- to settings with multiple non-absorbing treatment states. A related recent literature considers the treatment effect interpretations of so-called “event study” designs, in which individuals select into a binary treatment over multiple time periods (imaikim2016; bj_es; as_es; csa18; De Chaisemartin and D'Haultfoeuille, 2018\nocite{fm16}). Here I focus on issues raised by selection into and out of multiple treatments over two periods, though an appendix section extends the theory to multiple-period mover designs. Further afield, the conditional homogeneity restriction I propose is similar to those used by angrist_fernandez_val:13, angrist_rokkanen:15, and hull15 for extrapolating treatment effects within and across quasi-experimental instrumental variable and regression discontinuity designs.

More generally, this paper builds on the long and rich panel data literature chamberlain:1980, chamberlain:1982, chamberlain:1984, manski:1987, honore:1992, arellano2003 by allowing for certain forms of nonlinearity, non-additivity, and heterogeneity in causal response over time. Notably, the parallel trends assumptions I develop here are weaker than the “time ignorability” restrictions underlying recent approaches to non-separable panel identification (hahn01; wooldridge05; chernetal13), as the special structure of mover designs permits particular types of heterogeneous time-varying shocks.\footnote{graham_powell_2012 consider an alternative panel approach with continuous treatment variables, this theory is less relevant for mover selection among discrete unordered alternatives.} As mentioned above some of my proposed estimators also allow for persistent effects of treatment -- a feature that, as imaikim2016 point out, is typically ruled out in panel data frameworks.

The remainder of this paper is organized as follows. The next section develops the dynamic treatment effects framework, characterizes the causal content of conventional mover regressions, and builds intuition for the main identification result. Section 3 then develops the class of two-step estimators for mover average treatment effects and discusses both specification testing and efficiency. Section 4 concludes. Future drafts of this paper will illustrate the theory using the the Medicare patient mover design of fgw16. All proofs, along with the extension of the theory to multiple time periods, are contained in the appendix.

Interpreting Mover Regressions

Suppose we observe a panel of individuals $i$ over time periods $t=0,\dots, T-1$, including an outcome $Y_{it}$, a vector of covariates $X_{it}$, and an individual's repeated selection $J_{it}\in\{0,\dots,J-1\}$ among $J$ possible treatments. Let $D_{ijt}=\mathbf{1}[J_{it}=j]$ be an indicator for individual $i$ choosing treatment $j$ in time $t$. A typical mover analysis estimates the regression

Here $\alpha_i$ and $\tau_t$ denote individual and time fixed effects, while the coefficient $\beta_j$ is meant to capture the effect of treatment $j$ relative to the omitted treatment $0$. This mover regression may be estimated just on the set of individuals with $J_{is}\neq J_{it}$ for some $s\neq t$ (who I refer to as movers) or include other stayers with $J_{it}=\bar{J}_i$ for all $t$. In practice researchers often exclude stayers or include mover-specific fixed effects in ((ref)) in order to weaken the identifying assumptions (fgw16; chetty/hendren:15).\footnote{Mover regressions are often written $Y_{it} = \alpha_i + \tau_t + \beta_{J_{it}} +X_{it}^\prime \gamma + \epsilon_{it}$. One treatment category is always omitted from estimation, though sometimes the estimated $\beta_j$ are recentered to capture effects relative to the average $j$.}

I use a dynamic potential outcomes framework (robins86,robins97) to characterize the causal interpretation of equation (1). Let $Y_{it}^{k\rightarrow j}$ denote the outcome of individual $i$ in time $t$ if she were to select treatment $j$ in that period after previously following the treatment path of $k=(k_0,\dots,k_{t-1})^\prime$. These are well-defined random variables under the usual stable unit treatment value assumption (rubin80), which I maintain throughout. When not ambiguous I write $Y_{it}^j\equiv Y_{it}^{(J_{i0},\dots,J_{i,t-1})^\prime \rightarrow j}$ as the time-$t$ potential outcome of individual $i$ given her treatment choices $J_{is}$ for $s<t$: that is, $Y_{it}^j$ implicitly conditions on choices made in past periods. Point-in-time treatment effects relative to the omitted treatment are written $Y_{it}^j-Y_{it}^0$, and realized outcomes can be written

where $\overline{0}$ denotes a conforming vector of zeros. The first two terms of equation ((ref)) reflect potential outcomes at time $t$ if an individual had stayed in treatment $0$ in all previous periods $s<t$, while the third term captures differences in outcomes at time $t$ arising from different treatment histories.

A comparison of equations ((ref)) and ((ref)) suggests a set of sufficient conditions for mover regression estimates to be causally interpretable:

Potential outcomes are impersistent under Assumption IO in that they only depend on the contemporaneous treatment status and not on previous treatment choices. When this is the case the last term of equation ((ref)) is ignorable. When furthermore each of the period-specific treatment effects are constant across individuals (Assumption CE), we may write $Y_{it}=Y_{it}^0+\sum_{j\neq 0}\overline{\beta}^j D_{ijt}$. The regression coefficients $\beta_j$ then coincide with the causal effects $\overline{\beta}^j$ if we can decompose $Y_{it}^0 = \alpha_i + \tau_t + X_{it}^\prime \gamma + \epsilon _{it}$, with $\epsilon_{it}$ orthogonal to the vector of treatment choices (Assumption CO).\footnote{Sometimes mover regressions are motivated by a stronger conditional independence assumption, along with implicit impersistence and constant effects assumptions: see, e.g., equations (2) and (4) in ams15.}

Assumptions IO, CE, and CO are straightforward to state mathematically and may, as in fgw16, stem from an underlying economic model. Nevertheless, they may also prove strong and difficult to evaluate in practice: researchers might be reluctant to rule out any forms of outcome persistence or treatment effect heterogeneity, or find it challenging to assess the appropriateness of the conditional orthogonality restriction in different settings. Instead, researchers often motivate mover regressions with claims on the comparability of outcome trends across different types of movers, and validate their estimates with the kinds of pre- and post-trend analyses typically associated with difference-in-difference designs.\footnote{See, e.g., Figure 5 in chk_2013, Table 4 in jackson13, and Figure 6 in fgw16.} I therefore next consider what mover regressions identify when Assumptions IO and CE are relaxed and when Assumption CO is replaced with a quasi-experimental parallel trends assumption. To start simply and build intuition for later identification results, I first consider mover regressions with only two treatment states.

Binary Treatment Mover Regressions

Suppose there are only time periods ($T=2$). Then the mover treatment coefficients are equivalently defined by a first-differenced regression,

where $\Delta V_{i} = V_{i1}-V_{i0}$ denotes the first-difference operator applied to variable $V_{it}$ and $\tau=\tau_1-\tau_0$. When furthermore treatment is binary $(J=2)$ and there are no added covariates ($X_{it}=0$), we have a simple algebraic expression for the single mover regression coefficient $\beta_1$:

The proof of Lemma 1 uses omitted-variables bias algebra to write $\beta_1$ as a linear combination of the coefficients from a saturated model for $E[\Delta Y_i\mid \Delta D_{i1}]$, identifying mean outcome growth among stayers (with $\Delta D_{i1}=0$), those who move out of treatment 1 (with $\Delta D_{i1}=-1$), and those who move into treatment 1 (with $\Delta D_{i1}=1$). Lemma 1 thus shows that the simplest mover regression identifies a convex average of outcome growth comparisons between movers and stayers, across the two mover types. In the special case of no stayers this expression simplifies to the single comparison ((ref)) across the two mover groups.

Using Lemma 1, it is straightforward to show that a restriction on average outcome persistence, combined with a standard parallel trends assumption, renders binary treatment mover regressions causally interpretable:

In words, Proposition 1 states that the binary treatment mover regression identifies a convex combination of average treatment effects, across time and the two mover groups, under two assumptions. First, equation ((ref)) requires individuals that move into each treatment to have, on average, the same time-$1$ outcome as if they had always been there (an impersistence assumption, weakening Assumption IO). Second, equations ((ref))-((ref)) state that -- conditional on an individual being in treatment $j$ at any point -- the potential outcomes for different types of movers and stayers would have followed the same average growth path in the absence of a move (a parallel trends assumption, weakening Assumption CO). Note that Proposition 1 does not directly restrict treatment effect heterogeneity, relaxing Assumption CE.

Intuition for the link between equation ((ref)) and equation ((ref)) comes from classic difference-in-differences logic. Under parallel trends, the difference in outcome growth rates between those moving into treatment $1$ at $t=1$ (with $\Delta D_{i1}=1$) and treatment $0$ stayers (with $D_{i00}=D_{i01}=1$) identifies the average time-$1$ treatment effect of the former group,

while the average time-$1$ treatment effect for the other mover group (with $\Delta D_{i1}=-1$) is identified by subtracting their outcome growth from that of the other stayer group (with $D_{i10}=D_{i11}=1$):

Similarly, when the assumptions of Proposition 1 hold, comparisons of outcome growth among (i) $\Delta D_{i1}=1$ movers and treatment $1$ stayers and (ii) treatment $0$ stayers and $\Delta D_{i1}=-1$ movers identify average time-$0$ treatment effects. Thus each of the two terms in Lemma 1 can be written as a weighted average of difference-in-difference comparisons identifying average causal effects under the assumptions.

Figure 1(a) summarizes the four difference-in-difference comparisons combined in the simple mover regression. Outcome growth contrasts within the dark- and light-colored groups identify average time-$1$ effects under the assumptions of Proposition 1, while comparisons within the dashed- and solid-lines group identify average time-$0$ effects. Interestingly, only the parallel trends assumption is needed to identify the former, whereas the impersistence restriction is also used to ensure the time-$0$ comparisons are causally interpretable. This is because time-$0$ outcomes can always be “differenced off” when comparing within the light- dark-colored groups of Figure 1(a), as in standard difference-in-differences, but the time-$1$ outcomes in the “reverse” difference-in-differences of the dashed- and solid groups are not comparable when potential outcomes systematically persist.

The appendix proof to Proposition 1 shows that the weights aggregating these difference-in-difference comparisons depend on the population shares of different mover and stayer types. Clearly if there are no movers with $\Delta D_{i1}=-1$ then $\beta_1$ represents a weighted average of causal effects for movers with $\Delta D_{i1}=1$, and vice-versa. How much weight is placed on effects from time $0$ versus time $1$ moreover depends on the proportion of stayers in each treatment: at the extremes if there are no stayers at either $d=0$ or $d=1$, then the mover regression weights together time-$d$ effects for movers with $\Delta D_{i1}=1$ and time-$(1-d)$ effects for movers with $\Delta D_{i1}=-1$. Thus when there are no initially-treated individuals, the mover regression identifies $E[Y_{i1}^1-Y_{i1}^0\mid \Delta D_{i1}=1]=E[Y_{i1}^1-Y_{i1}^0\mid D_{i11}=1]$, the average treatment effect on the treated, as in a standard difference-in-difference design.

Finally, it is worth exploring the special situation in which there are no stayers of either type. This follows from the second part of Lemma 1:

As mentioned, researchers may exclude stayers from a mover regression in order to weaken the identifying assumptions: here note that whenever the parallel trends assumption in Proposition is satisfied with stayers it is also satisfied when they are excluded. The corollary shows that without stayers the simple mover regression identifies a simply-weighted average of mover treatment effects across time. Interestingly, this estimand can be expressed in two ways: as the average of time-$t$ effects for movers into treatment 1 and time-$(1-t)$ effects for movers out of treatment 1, for either $t=0$ or $t=1$. The equivalence of ((ref)) and ((ref)) is an algebraic consequence of imposing parallel trends for both treatments.

A pessimistic interpretation of Proposition 1 and its corollary is that mover regression coefficients may be difficult to interpret even in the binary treatment case. That is, two mover experiments with the same joint distribution of causal effects and treatment choices (and thus the same average treatment effects for different types of movers and stayers) may produce different regression coefficients, depending on the marginal distribution of initial treatment. Researchers interested in the external validity of mover regression estimates, or in comparing estimates across different experiments, may view this as an important limitation.\footnote{yitzhaki96 argues this point for regression-weighted averages of causal parameters in general; see also angrist98 and, more recently, BoFE18.} Nevertheless, the above discussion suggests this limitation can be easily overcome: when the assumptions of Proposition 1 hold, a researcher wishing to estimate a time-specific average causal effect could construct the relevant difference-in-difference comparisons in Figure 1(a) and weight them together as they please. This reweighted difference-in-difference logic is at the core of the general strategy for identifying mover average treatment effects in Section 3. Before formalizing the strategy, however, we first consider additional complications arising from mover designs with multiple unordered treatments.

Multiple-treatment Mover Regressions

In practice, mover regressions involve choices across many different treatments. fgw16, for example, estimate models with $305$ healthcare market treatments, while chk_2013 study worker movement across over a million German firms. It may be reasonable to expect that the basic difference-in-difference logic extends to multiple-treatment regressions, so that they again capture some weighted average of causal effects under weak quasi-experimental restrictions. The following result, however, shows that this is not the case:

The appendix proof of Proposition 2 shows that even with strongly impersistent outcomes and conventionally parallel trends, the multiple treatment coefficients in equation ((ref)) combine a set of average treatment effects with a set of non-causal terms. The latter are of the form

and

for treatments $j>0$ and $k\neq j$ and for time periods $t$, $s$, and $u$. These capture, respectively, differences in trends among stayers in different treatment states and combinations of causal effects for movers across different treatments and times. Although each of these comparisons are ignorable in the canonical additively-separable and constant-effects model (or imposed directly by Assumption CO and CE), in general they are non-zero under the parallel trends assumption, rendering the mover coefficients causally uninterpretable.

At first this shortcoming of multi-treatment mover regressions may appear a puzzle. After all additional treatment choices simply yield more difference-in-difference comparisons, each of which are causally interpretable under parallel trends and Assumption IO. The issue, as with the weighting scheme of Proposition 1, is that the mover regression combines the set of quasi-experiments in a way that is sensible when the canonical mover model is correctly specified, but need not be when there are heterogeneous treatment effects or time-varying shocks.

As a simple example of the issue, consider a three-treatment design with the two mover and stayer groups illustrated in Figure 1(b). Following the logic of the previous subsection, there are two time-$1$ average causal effects identified under parallel trends: the effect of treatment $1$ relative to treatment $0$ for movers from $0$ to $1$ and the effect of treatment $2$ relative to treatment $1$ for movers from $1$ to $2$. The average time-$0$ effect of treatment $1$ relative to treatment $0$ is moreover identified for movers from $0$ to $1$ if we add an outcome impersistence condition. As in Figure 1(a), the difference-in-difference comparisons identifying each of these effects are given by contrasts within similarly-colored and similarly-patterned line groups.

How does the mover regression combine these effects? Using formulas from the proof to Proposition 2, we can show that the treatment coefficients satisfy

where $p_0$ denotes the proportion of stayers in treatment $0$. Thus, while the treatment-$1$ mover regression coefficient continues to identify a convex average of time-$0$ and time-$1$ treatment effects, the coefficient on treatment $2$ is not causal. Lacking any difference-in-difference comparison identifying the effect of treatment $2$ relative to treatment $0$, the mover regression sums the first two causal effects in equation ((ref)). Under a constant effects assumption this is sensible, since then

though with meaningful treatment effect heterogeneity this sum need not be causally interpretable. Equation ((ref)) moreover includes a term capturing the difference in outcome growth rates for the two stayer groups. This is again sensible under Assumptions CE and CO, since then

though in general this term need not be zero. Note that the left-hand side of equation ((ref)) is an example of equation ((ref)), where the third undefined term is arbitrarily set to zero, while the left-hand side of equation ((ref)) is an example of equation ((ref)).

A researcher pessimistic about the weighting scheme in Proposition 1 may therefore have even more cause for pessimism with multiple-treatment mover regressions. Despite the availability of multiple difference-in-difference comparisons, conventional mover analyses need not even have a weighted causal effect interpretation when $J>2$. Nevertheless, as with the binary treatment case, one could imagine individually extracting and more sensibly combining the component difference-in-difference quasi-experiments to overcome the limitations of mover regressions. This combination may involve extrapolating some causal effects from others, just as the above regression example does in order to identify the average effect of treatment 2 relative to treatment 0 under constant effects. Weaker extrapolations, however, may only combine difference-in-difference experiments capturing treatment effects from the same time period and for observably-similar movers, weakening the constant effects assumption. I next develop a class of two-step estimators that enact this logic.

Estimating Mover Average Treatment Effects

In general there may be many combinations of heterogeneous treatment effects that are of interest in a mover design. To discipline the initial theoretical approach I focus on mover average treatment effects, defined for each treatment $j>0$ as

where $\Delta D_i$ is a vector collecting the set of $\Delta D_{ij}$. Here $MATE_{jt}$ captures the average time-$t$ effect of treatment $j$ relative to treatment $0$, among individuals that change treatment status.\footnote{Recall that while for simplicity we restrict attention here to two periods, the appendix generalizes what follows to the multiple period case. For this I define $\Delta D_i$ as a vector collecting the set of $D_{ijt}-D_{ijs}$ for all $t\neq s$, so that $MATE_{jt}$ captures the average effect for individuals moving at any point.} That mover designs tend to reveal effects on mobile individuals is often implicit in applications, just as is that standard difference-in-difference estimation tends to capture average effects for those who become treated (Abadie, 2005). Of course, if there are no stayers in the study population the MATEs become average treatment effects (ATEs).

Identifying Assumptions

The key quasi-experimental assumption I leverage is that of conditional parallel trends, which generalizes the trend restrictions in Proposition 1. Formally, with $X_i$ denoting a vector of controls (including, perhaps, some elements of the $X_{i0}$ and $X_{i1}$ from the mover regression and other time-invariant observables), consider

Under Assumption CPT, the average treatment-$j$ outcomes for different types of movers into or out of $j$ and stayers at $j$ would have followed parallel paths if not for the move, conditional on the controls in $X_i$. In many settings it may be plausible that an individual's treatment selection is only driven by potential outcome dynamics through a set of contemporaneous or lagged observables, as with the famous “Ashenfelter dip” of pre-treatment income for those entering job training programs ashenfelterdip, ashcard85. Clearly, the parallel trends assumption in Proposition 1 is a special case of Assumption CPT, for which $X_{i}=0$. It is also straightforward to verify that the “time ignorability” identifying assumptions developed in the recent literature on non-separable panel models (e.g. chernetal13) imply Assumption CPT, but are not implied by it.\footnote{chernetal13 consider models of the form $Y_{it}^{k\rightarrow j}=g(j,X_i,\alpha_i,\epsilon_{it})$, where the distribution of $\epsilon_{it}$ does not depend on $t$ given $(\alpha_i,J_{i0},J_{i1},X_i)$. Then $E[Y_{i1}^{j\rightarrow j}-Y_{i0}^j\mid J_{i0},J_{i1},X_i]=0$, satisfying Assumption CPT. Identification here allows for heterogeneous time-varying shocks by leveraging the particular structure of mover designs.}

As with multiple-treatment mover regressions, combining causal effects across many difference-in-difference experiments requires further homogeneity restrictions. Here I adopt a weaker assumption than the canonical model: that mover treatment effects are on average comparable, conditional on observables:

Restrictions on the conditional heterogeneity of average causal effects have been previously used in the treatment effects literature to extrapolate within and across different quasi-experiments angrist_fernandez_val:13,angrist_rokkanen:15,hull15. Here Assumption CEH states that differences in average causal effects for movers with different origins-destination pairs are driven only by the set of observed contemporaneous or lagged controls in $X_i$. In applications researchers sometimes gauge mover effect homogeneity by tests of outcome trend symmetry (e.g. chk_2013); Assumption CEH can thus be thought to relax the assumptions motivating such tests.

Finally, as in the previous section, I use a restriction on average potential outcome persistence to identify time-$0$ causal effects from “reverse” difference-in-difference quasi experiments. Again this can be made conditional on observables:

Under Assumption COI, the mean outcome for movers into each treatment $j$ from each treatment $k$ is the same as it would be if the movers had always chosen $j$, conditional on the controls in $X_i$. Thus any potential for persistence in average outcomes for movers must be driven by time-varying or invariant observables, relaxing the usual panel data restriction of complete outcome impersistence (imaikim2016).

Together, these three assumptions relax those of standard mover regressions. In particular Assumptions CPT and CEH are implied by a partially-separable model of dynamic potential outcomes,

where $\epsilon_{ij1}-\epsilon_{i0}$ is mean-independent of treatment choices conditional on the controls and on $i$ being a $j$-mover or $j$-stayer (that is, of $(J_{i0},J_{i1})$ conditional on $X_i$ and $D_{ij0}+D_{ij1}\neq 0$). As shown below, with enough stayers this model permits identification of time-$1$ MATEs, here taking the form

Estimating time-$0$ MATEs or omitting stayers will additionally require Assumption COI, which here restricts $E[\epsilon_{ik1}\mid J_{i0},J_{i1},X_i]=E[\epsilon_{im1}\mid J_{i0},J_{i1},X_i]$ for all $k,m$. In contrast, the mover regression assumptions IO, CE, and CO are satisfied when the $\epsilon_{ikt}$ do not depend on $k$ and are mean-independent of $(J_{i0},J_{i1})$, and when $\beta_{jt}(X_i)$ is additively-separable in treatment, time, and a fixed linear combination of the controls. The following identification results thus allow for both time-varying shocks and treatment effect heterogeneity to vary flexibly with observables.

Identification with and without Stayers

With the three key assumptions in hand, I next establish MATE identification in two salient cases. First, I suppose a researcher is willing to assume potential outcome trends are conditionally comparable between movers and stayers, and that stayers are sufficiently dispersed to make feasible a set of conditional difference-in-difference comparisons linking treatments $j$ and $0$. Enumerating this set requires some additional notation; I let $C_j$ denote the set of all variable-length n-tuples $C_{j \ell}=(c_{j \ell0},\dots,c_{j \ell M_{j \ell}})$, where $M_{j\ell}+1$ denotes the length of $C_{j\ell}$ and where $c_{j\ell m} \in \{0,\dots,J-1\}$, $c_{j\ell 0}=0$, $c_{j\ell M_{j\ell}}= j$, and $c_{j\ell m}\neq c_{j\ell n}$ for all $m\neq n$. Thus $C_j$ collects all of the $\ell$ possible paths (or chains) from treatment $0$ to treatment $j$ via other treatment states, where no intermediate treatment is included as a link in the chain more than once. We then have the following result:

In words, Proposition 3 states that the time-$1$ mover average treatment effect for treatment $j$ is identified by a particular weighted average of outcome growth $\Delta Y_i$ under conditional parallel trends and effect homogeneity, and when there exists a set of difference-in-difference comparisons linking treatment $j$ to the reference treatment $0$. This set is given by the chain $C_{j\ell}=(0,c_{j\ell1},\dots,c_{j\ell,M_{j\ell}-1},j)$, where between any two links $c_{j\ell,m-1}$ and $c_{j\ell m}$ there exist either (i) movers from treatment $c_{j\ell,m-1}$ into treatment $c_{j\ell m}$ and stayers at treatment $c_{j\ell,m-1}$ or (ii) movers from treatment $c_{j\ell m}$ into treatment $c_{j\ell,m-1}$ and stayers at treatment $c_{j\ell m}$, conditional on the controls. The proof to Proposition 3 shows that under these assumptions the $m$-specific terms in the weighting scheme ((ref)) identify $E[Y_{i1}^{c_{j\ell,m}}-Y_{i1}^{c_{j\ell,m-1}}\mid \Delta D_i\neq 0]$, so that summing over the links of the chain identifies $E[Y_{i1}^{j}-Y_{i1}^{0}\mid \Delta D_i\neq 0]=MATE_{j1}$. An analogous result follows for $t=0$ effects when potential outcomes are conditionally impersistent.

Unpacking this result further, note that each summand of the weighting scheme ((ref)) is in turn a linear combination of two terms given by equation ((ref)). The $\rho_{it}^{c,d}$ depend on the conditional frequency of different groups of movers and stayers given the controls, as summarized by the propensity scores $E[D_{idt}D_{ics}\mid X_i]$ and $P(\Delta D_i\mid X_i)$. The appendix proof shows that weighting $\Delta Y_i$ by $\rho_{it}^{c,d}$ and $-\rho_{it}^{d,c}$ replicates and averages together conditional difference-in-difference comparisons between the different mover and stayer groups illustrated in Figure 1(a). For example,

which is a weighted comparison of the conditional difference in outcome growth between movers from treatments $c$ to $d$ and stayers at treatment $c$. Similarly, $E[\Delta Y_i (-\rho_{i1}^{d,c})\mid X_i]$ identifies a weighted conditional contrast of outcome growth between stayers at treatment $d$ and movers from treatments $d$ to $c$. As before when the conditional parallel trends assumption holds these identify conditional weighted average treatment effects of the respective mover groups, which are assumed to both be representative of all movers under conditional effect homogeneity. Thus any weighted average of $E[\Delta Y_i \rho_{i1}^{c,d}\mid X_i]$ and $E[\Delta Y_i (-\rho_{i1}^{d,c})\mid X_i]$ identifies $E[Y_{i1}^{c_{j\ell,m}}-Y_{i1}^{c_{j\ell,m-1}}\mid \Delta D_i\neq 0,X_i]\frac{P(\Delta D_i\neq 0\mid X_i)}{P(\Delta D_i\neq 0)}$ under the assumptions; averaging these averages over the marginal distribution of the controls then identifies $E[Y_{i1}^{c_{j\ell,m}}-Y_{i1}^{c_{j\ell,m-1}}\mid \Delta D_i\neq 0]$.

Proposition 3 can be thought to generalize Abadie (2005)'s approach to identification in difference-in-difference designs. Specifically, suppose $J=2$ and there are no movers from treatment $1$ to treatment $0$. Then there is only one chain $C=(0,1)$ satisfying (i), with $w_1=1$ and Assumption CEH satisfied trivially, and the weighting scheme identifying $MATE_{11}=E[Y_{i1}^1-Y_{i0}^0\mid D_{i1}=1]$ coincides with that of abadie_05.

It is also worth noting that the logic of Proposition 3 implies a weaker approach to MATE identification in binary treatment mover designs, using particular data-driven weights to avoid restricting treatment effect heterogeneity:

As in Proposition 3, the weighting scheme in equation ((ref)) combines two sets of conditional difference-in-difference experiments via the $\rho_{it}^{0,1}$ and $(-\rho_{it}^{1,0})$ terms. Here however these terms are combined via weights ((ref)) that are proportional to the share of movers of each type -- either out of or into treatment 1 -- in order to avoid mixing effects across the two groups. This corollary thus gives a flexible way to estimate pairwise average treatment effects in a mover design for movers across two treatments, though such effects need not be comparable across different treatments without an effect homogeneity assumption.

To apply Proposition 3 or its corollary, a researcher must be willing to assume that any systematic differences in the potential outcome trends of movers and stayers arise from a set of observable controls. As in the previous section, we may also consider identification under a weaker parallel trends assumption in which the trends of stayers are unrestricted. The following result shows that in such cases one may still identify the average of $MATE_{jt}$ across time $t$:

Proposition 4 generalizes the basic logic of the corollary to Proposition 1, along the lines of Proposition 3. Here each summand in equation ((ref)) identifies a weighted difference-in-difference comparison between movers from treatment $c$ to treatment $d$ and movers from treatment $d$ to treatment $c$, such that, under the assumptions,

Similar to before, weighting and adding together these comparisons along the links of any chain $C_{j}$ from treatment $0$ to treatment $j$ thus identifies $\frac{1}{2}(MATE_{j0}+MATE_{j1})$. For this to be feasible there must exist both types of movers at each link, conditional on the controls, as no stayer variation is used. The cost of the weaker parallel trends assumption is a somewhat less informative estimand: the weighting scheme ((ref)) is not able to separately identify time-$0$ and time-$1$ effects.

Estimation and Testing

As in abadie_05, Propositions 3 and 4 suggest a straightforward two-step approach to estimating mover average treatment effects from an i.i.d. sample of size $N$. In a first step, a researcher computes a set of propensity score estimates $\widehat{E}[D_{id0}D_{ic1}\mid X_i]$, along with the sample proportion of movers $\widehat{P}(\Delta D_i\neq 0)$. For a given chain $C_{j\ell}$, she then forms the sample analogue of either equation ((ref)) or equation ((ref)), with the former also requiring specification of a vector of weights $w_{j\ell}$. When the first-step estimates are consistent, so too will be the second-step weighting estimator under the assumptions of Proposition 3 or 4. The asymptotic behavior of this estimator depends on the properties of the data-generating process and whether or not the first-step propensity score estimates are parametric. Future drafts will establish this behavior formally using standard first-order asymptotic theory (as in Abadie (2005)); for now I take the $\sqrt{N}$-consistency and asymptotic normality of the described two-step estimators as given, in order to focus on additional conceptual issues arising from overidentification in mover designs.

When both conditions (i) and (ii) of Proposition 3 hold for some of the links of a chain $C_{j\ell}$, one faces the choice of which set of weights $w_{j\ell m}$ to use for estimating $MATE_{jt}$. Further degrees of overidentification arise when multiple $C_{j\ell}$ satisfy the assumptions of either Propositions 3 or 4: in general the number of possible chains grows rapidly with the number of institutions, though in practice the graph connecting any two institutions by difference-in-difference experiments may be sparse.\footnote{To be precise, there are $\sum_{k=0}^{J-2}k!\binom{J-2}{k}^2$ possible chains given $J$ institutions.} Testing the equality of two estimates formed by different chains or weights constitutes an omnibus specification test of the identifying assumptions which may be used to gauge the plausibility of the quasi-experimental framework.

A potentially more powerful test of this joint null hypothesis uses a conditionally-efficient estimator, optimally combining different quasi-experiments to minimize the asymptotic estimator variance. To characterize this procedure, let $\widehat{\rho}_{it}^{c,d}$ and $\widehat{\kappa}_{it}^{c,d}$ be consistent estimates of $\rho_{it}^{c,d}$ and $\kappa_{it}^{c,d}$, and let $\widehat{M}$ denote a $K\times 1$ vector collecting the set of either all positive and negative $\widehat{E}[\Delta Y_i \widehat{\rho}_{it}^{c,d}]$ or of all $\widehat{E}[\Delta Y_i \widehat{\kappa}_{it}^{c,d}]$ that are well-defined for treatments $(c,d)$, where $\widehat{E}[V_{it}]$ denotes the sample average of a variable $V_{it}$. Next, let $S_{j}$ be a $P\times K$ matrix with elements $S_{jpk}\in\{0,1\}$, such that, for all $p$, $\sum_k S_{jpk}\widehat{M}_k$ is consistent for $MATE_{jt}$ under Proposition 3 or identifies $\frac{1}{2}(MATE_{j0}+MATE_{j1})$ under Proposition 4. Then a general method of moments estimator for the target causal parameter $\beta_j$, which combines mover variation across all available chains, is given by

where $W$ is some $P\times P$ positive-definite weighting matrix. Given the first-step propensity scores underlying $\widehat{M}$, a conditionally-efficient estimator $\widehat{\beta}_j^*$ is obtained by setting $W=(S_j\widehat{\Omega}S_j^{\prime})^{-1}$, where $\widehat{\Omega}$ consistently estimates the asymptotic variance of $\widehat{M}$. Solving ((ref)), we then have

where $\iota$ denotes a $P\times 1$ vector of ones.

This estimate of the target causal parameter (either $MATE_{jt}$ if $\widehat{M}$ collects moments involving $\widehat{\rho}_{it}^{c,d}$ or $\frac{1}{2}(MATE_{j0}+MATE_{j1})$ if $\widehat{M}$ collects moments involving $\widehat{\kappa}_{it}^{c,d}$) is an optimally-weighted combination of all relevant difference-in-difference quasi-experiments, with weights proportional to the elements of $\iota^\prime (S_j\widehat{\Omega}S_j^\prime)^{-1}S_j$. In particular, it satisfies

where $\widehat{\Omega}\xrightarrow{p}\Omega$. As usual, an omnibus specification test based on ((ref)) is given by

which has an asymptotic $\chi^2_{P-1}$ distribution under the joint null of the identifying assumptions. Implicitly, ((ref)) checks whether the same estimate of $\beta_j$ is obtained from any two difference-in-difference chains, weighting pairwise comparisons by the efficient weights.

It appears relatively straightforward to compute equations ((ref)) and ((ref)) in practice. Estimation of the large set of mover propensity scores can be distributed over multiple computational resources, while the second-step calculation of the estimated asymptotic variance of $\widehat{M}$ and the second-step estimator ((ref)) is likely to be quite simple. Future drafts of this paper will include simulations of the computational demands and finite-sample performance of these estimators, as well as an application to a real-world movers design.

Conclusions

Although retaining the flavor of simpler difference-in-differences designs, quasi-experimental mover designs require additional restrictions. Mover regression estimates, while causally interpretable in the binary treatment case, fail in general to recover weighted averages of heterogeneous treatment effects under a parallel trends assumption alone. In contrast, the two-step estimators developed here accommodate heterogeneous treatment effects and time-varying shocks, provided they are not correlated with individual movement conditional on controls. Certain mover average treatment effects are moreover identified without a direct restriction on potential outcome persistence, provided the potential trends of movers and stayers are conditionally comparable. As argued above, the computation of conditionally-efficient treatment effect estimates is likely to be light in practice, even relative to recent advances in traditional two-way fixed effect regressions ack2002, guimaraes/portugal:10, gaure13, correia2016.

As always, whether the assumptions considered here are more plausible than other approaches to mover designs , such as hlm17 and blm17, will be a matter of context. The restrictions of conditional parallel trends and conditional effect homogeneity are likely to have the advantage of both being familiar to applied researchers and relatively straightforward to consider in applications. At a minimum, the quasi-experimental approach may allow researchers to verify the robustness of substantive conclusions drawn from conventional mover regressions and more recent variations.

Figures and Tables