Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
105,544 characters · 11 sections · 95 citation commands
\doublespacing \pagestyle{plain}
Abstract: { We propose a novel approach for causal mediation analysis based on changes-in-changes assumptions restricting unobserved heterogeneity over time. This allows disentangling the causal effect of a binary treatment on a continuous outcome into an indirect effect operating through a binary intermediate variable (called mediator) and a direct effect running via other causal mechanisms. We identify average and quantile direct and indirect effects for various subgroups under the condition that the outcome is monotonic in the unobserved heterogeneity and that the distribution of the latter does not change over time conditional on the treatment and the mediator. We also provide a simulation study and an empirical application to the Jobs II programme.}
{ Keywords: Direct effects, indirect effects, mediation analysis, changes-in-changes, causal mechanisms, treatment effects. }\newline { JEL classification: C21.}
\thispagestyle{empty} { \setcounter{footnote}{0} \setcounter{footnote}{0} \setcounter{page}{1} }
Causal mediation analysis aims at disentangling a total treatment effect into an indirect effect operating through an intermediate variable -- commonly referred to as mediator -- as well as the direct effect. The latter includes any causal mechanisms not operating through the mediator of interest. Even when the treatment is random, direct and indirect effects are generally not identified by simply controlling for the mediator without accounting for its potential endogeneity, as this likely introduces selection bias, see RoGr92.
This paper suggests a novel identification strategy for causal mediation analysis based on changes-in-changes (CiC) as suggested by at06 for evaluating (total) average and quantile treatment effects. We adapt the approach to the identification of the direct effect and the indirect effect running through a binary mediator. The outcome variable must be continuous and is assumed to be observed both prior to and after treatment and mediator assignment as it is the case in repeated cross sections or panel data. The key identifying assumptions imply that the continuous outcome is strictly monotonic in unobserved heterogeneity and that the distribution of unobserved heterogeneity does not change over time conditional on the treatment and the mediator (the latter assumption is also known as stationarity). Given appropriate common support conditions, this permits identifying direct effects on subpopulations conditional on the treatment and the mediator states, even if both treatment and mediator assignment are endogenous.
Augmenting the assumptions by random treatment assignment and weak monotonicity of the mediator in the treatment allows for causal mediation analysis in subpopulations defined upon whether and how the mediator reacts to the treatment. Specifically, we show the identification of direct effects among those whose mediator is always one an96 and never one (never-takers) irrespective of treatment assignment, respectively. Furthermore, we identify the total, direct, and indirect treatment effects on those whose mediator value complies with treatment assignment (compliers). For any set of assumptions, we discuss the identification of both average and quantile direct and indirect effects. We note that if appropriately weighted, the respective average effects among compliers, always-takers, and never-takers add up to the average direct and indirect effects in the population.
Identification in the earlier mediation literature typically relied on linear models for the mediator and outcome equations and often neglected endogeneity issues, see for instance Co57, JuKe81, and BaKe86. More recent contributions use more general identification approaches based on the potential outcome framework and take endogeneity issues explicitly into consideration. Examples include RoGr92, Pearl01, Robins2003, PeSiva06, VanderWeele09, ImKeYa10, Hong10, AlbertNelson2011, ImYa2011, TchetgenTchetgenShpitser2011, VansteelandtBekaertLange2012, and Huber2012. The vast majority of the literature assumes that the covariates observed in the data are sufficiently rich to control for treatment and mediator endogeneity. Also in empirical economics, there has been an increase in the application of such selection on observables approaches, see for instance SimonsenSkipper2006, FlFl09, HeckmanPintoSavelyev2013, Huber2015, Keeleetal2015, ContiHeckmanPinto2016, HuberLechnerMellace2017, BijwaardJones2018, BellaniBia2018, HuberLechnerStrittmatter2018, and doe19. Comparably few studies in economics develop or apply instrumental variable approaches for disentangling direct and indirect effects, see for instance FrHu17, PowdthaveeLekfuangfuWooden2013, BrunelloFortSchneeweisWinterEbmer2016 and Chenetal2017. Our paper provides another, CiC-based identification strategy that neither rests on selection on observables assumptions nor on instrumental variables for the treatment or the mediator.
While most studies aim at evaluating direct and indirect effects in the total population, a smaller strand of the literature uses the principal stratification framework of FrangakisRubin02 to investigate effects in subpopulations (or principal strata) defined upon whether and how the mediator reacts to the treatment, see Ru04. This approach has been criticized for typically focussing on direct effects on populations whose mediator is constant (i.e.\ always- and never-takers) rather than decomposing direct and indirect effects on compliers and for considering subpopulations rather than the total population, see VanderWeele08 and VanderWeele12. Deuchertetal2017 suggest a difference-in-differences (DiD) strategy that alleviates such criticisms. Identification relies on a randomized treatment, monotonicity of the (binary) mediator in the treatment, and particular common trend assumptions on mean potential outcomes across principal strata. The latter imply that mean potential outcomes under specific treatment and mediator states change by the same amount over time across specific subpopulations. Depending on the strength of common trend and effect homogeneity assumptions across principal strata, direct and indirect effects are identified for different subpopulations and under the strongest set of assumptions even for the total population.
Our paper contributes to this literature on principal strata effects, but relies on different identifying assumptions than Deuchertetal2017. While differential time trends across subpopulations are permitted, our approach restricts the conditional distribution of unobserved heterogeneity over time. The two sets of assumptions are not nested and their appropriateness is to be judged in the empirical context at hand. However, both approaches could be used simultaneously for testing the joint validity of the identifying assumptions of either method, in which case both CiC and DiD converge to the same, true average direct and indirect effects. As a further distinction to Deuchertetal2017, our method also permits assessing quantile treatment effects (QTEs) rather than average effects only.
In independent work, saw19 proposes a CiC strategy to tackle non-compliance in randomized experiments when the exclusion restriction of random assignment is violated. While there is an overlap in some identification results of his study and ours (e.g.\ concerning the direct effect on never-takers), there are also important differences. First, saw19 predominantly focusses on the average treatment effect on the treated under one-sided non-compliance (ruling out always-takers), which then corresponds to the total effect on compliers. Our paper in addition disentangles the total complier effect into direct and indirect components. Second, under two-sided non-compliance (i.e.\ the existence of both never- and always-takers), saw19 identifies the total complier effect by assuming homogeneity of the direct effect, while we extend the CiC assumptions to the always-takers for identifying (direct, indirect, and total) complier effects as well as the direct effect among always-takers. Third and in contrast to saw19, we also provide identification results in the absence of randomization and monotonicity of the mediator in the treatment. On the other hand, saw19 in contrast to our study demonstrates that the CiC strategy does not necessarily require pre-treatment outcomes, but may exploit any pre-treatment variable that has similar rank orders (as a function of unobserved heterogeneity) like the outcome of interest.
We provide a simulation study in which we compare the CiC to the DiD approach to illustrate our identification results. We also consider an empirical application to the Jobs II programme previously analysed by VinokurPriceSchul1995, a randomized job training intervention designed to analyse the impact of job training on labour market and mental health outcomes. We investigate the direct effect of the randomized offer of treatment on a depression index, as well as its indirect effect through actual participation in the programme as mediator. The reason for investigating the direct effect is that treatment assignment could have a motivation or discouragement effect on those randomly offered or not offered the training. We, however, find the direct effect estimates to be close to zero and statistically insignificant and therefore no indication for the violation of the exclusion restriction when using treatment assignment as instrumental variable for actual participation. In contrast, the moderately negative total and indirect effects on those induced to participate by assignment are statistically significant at least at the 10% level in all but one case and very much in line with the estimate obtained by instrumental variable regression.
The remainder of this study is organized as follows. Section (ref) introduces the notation and defines the direct and indirect effects of interest. Section (ref) presents the assumptions underlying our CiC approach as well as the identification results. Section (ref) provides a simulation study. Section (ref) provides an application to Jobs II. Section (ref) concludes.
Let $D$ denote a binary treatment (e.g., receiving the offer to participate in a training programme) and $M$ a binary intermediate variable or mediator that may be a function of $D$ (e.g., the actual participation in a training programme). Furthermore, let $T$ indicate a particular time period: $T=0$ denotes the baseline period prior to the realisation of $D$ and $M$, $T=1$ the follow up period after measuring $D$ and $M$ in which the effect of the outcome is evaluated. Finally, let $Y_t$ denote the outcome of interest (e.g., health measures) in period $T=t$. Indexing the outcome by the time period $t \in \{0,1\}$ implies that it is measured both in the baseline period and after the realisation of $D$ and $M$. To define the parameters of interest, we make use of the potential outcome notation, see for instance rub74, and denote by $Y_t(d,m)$ the potential outcome for treatment state $D=d$ and mediator state $M=m$ in time $T=t$, with $d, m, t, \in \{0,1\}$. Furthermore, let $M(d)$ denote the potential mediator as a function of the treatment state $d \in \{0,1\}$. For notational ease, we will not use any time index for $D$ and $M$, because either is assumed to be measured at a single point in time between $T=0$ and $T=1$, albeit not necessarily at the same point, as $D$ causally precedes $M$. Therefore, $D$ and $M$ correspond to the actual treatment and mediator status in $T=1$, while it is assumed that no treatment or mediation takes place in $T=0$.
Using this notation, the average treatment effect (ATE) in the ex-post period is defined as $\Delta_1 =E[Y_1(1,M(1))-Y_1(0,M(0))]$. That is, the ATE corresponds to the effect of $D$ on the outcome that either affects the latter directly (net of any effect on the mediator) or indirectly through an effect on $M$. Indeed, the total ATE can be disentangled into the direct and indirect effects, denoted by $\theta_1(d) = E[Y_1(1,M(d))-Y_1(0,M(d))]$ and $\delta_1(d)= E[Y_1(d,M(1)) -Y_1(d,M(0))]$, by adding and subtracting $Y_1(1,M(0))$ or $Y_1(0,M(1))$, respectively:
Distinguishing between $\theta_1(1)$ and $\theta_1(0)$ or $\delta_1(1)$ and $\delta_1(0)$, respectively, implies the possibility of interaction effects between $D$ and $M$ such that the direct and indirect effects could be heterogeneous across values $d=1$ and $d=0$.
In our approach, we consider the concepts of direct and indirect effects within specific subpopulations. The latter are either defined conditional on the treatment and mediator values or conditional on potential mediator values under either treatment states, which matches the so-called principal stratum framework of FrangakisRubin02. As outlined in an96 in the context of instrumental variable-based identification, any individual $i$ in the population belongs to one of four strata, henceforth denoted by $\tau$, according to their potential mediator status under either treatment state: always-takers ($a$: $M(1)=M(0)=1$) whose mediator is always one, compliers ($c$: $M(1)=1$, $M(0)=0$) whose mediator corresponds to the treatment value, defiers ($de$: $M(1)=0$, $M(0)=1$) whose mediator opposes the treatment value, and never-takers ($n$: $M(1)=M(0)=0$) whose mediator is never one. Note that $\tau$ cannot be pinned down for any individual, because either $M(1)$ or $M(0)$ is observed, but never both.
Let $\Delta_1^{\tau} = E[Y_1(1,M(1))-Y_1(0,M(0))|\tau]$ denote the ATE conditional on $\tau \in \{a,c,de,n\}$; $\theta_1^{\tau}(d)$ and $\delta_1^{\tau}(d)$ denote the corresponding direct and indirect effects. Because $M(1)=M(0)=0$ for any never-taker, the indirect effect for this group is by definition zero $(\delta_1^{n}(d)=E[Y_1(d,0) -Y_1(d,0)|\tau=n]=0)$ and $\Delta_1^{n} = E[Y_1(1,0)-Y_1(0,0)|\tau=n]=\theta_1^{n}(1)=\theta_1^{n}(0)=\theta_1^{n}$ equals the direct effect for never-takers. Correspondingly, because $M(1)=M(0)=1$ for any always-taker, the indirect effect for this group is by definition zero $(\delta_1^{a}(d)=E[Y_1(d,1) -Y_1(d,1)|\tau=a]=0)$ and $\Delta_1^{a} = E[Y_1(1,1)-Y_1(0,1)|\tau=a]=\theta_1^{a}(1)=\theta_1^{a}(0)=\theta_1^{a}$ equals the direct effect for always-takers. For the compliers, both direct and indirect effects may exist. Note that $M(d)=d$ due to the definition of compliers. Accordingly, $\theta_1^{c}(d) = E[Y_1(1,d)-Y_1(0,d)|\tau=c]$ equals the direct effect for compliers, $\delta_1^{c}(d)= E[Y_1(d,1) -Y_1(d,0)|\tau=c]$ equals the indirect effect for compliers, and $\Delta_1^{c}= E[Y_1(1,1) -Y_1(0,0)|\tau=c]$ equals the total effect for compliers. In the absence of any direct effect, the indirect effects on the compliers are homogeneous, $\delta_1^{c}(1)=\delta_1^{c}(0)=\delta_1^{c}$, and correspond to the local average treatment effect an96. Analogous results hold for the defiers.
As already mentioned, we will also consider direct effects conditional on specific values $D=d$ and mediator states $M=M(d)=m$, which are denoted by $\theta_1^{d,m}(d)=E[Y_1(1,m)-Y_1(0,m)|D=d,M(d)=m]$. These parameters are identified under weaker assumptions than strata-specific effects, but are also less straightforward to interpret, as they refer to mixtures of two strata. For instance, $\theta_1^{1,0}(1)=E[Y_1(1,0)-Y_1(0,0)|D=1,M(1)=0]$ is the effect on a mixture of never-takers and defiers, as these two groups satisfy $M(1)=0$. Likewise, $\theta_1^{0,0}(0)$ refers to never-takers and compliers satisfying $M(0)=0$, $\theta_1^{0,1}(0)$ to always-takers and defiers satisfying $M(0)=1$, and $\theta_1^{1,1}(1)$ to always-takers and compliers satisfying $M(1)=1$.
We denote by $F_{Y_{t}(d,m)}(y) = \Pr(Y_t(d,m) \leq y)$ the cumulative distribution function of $Y_t(d,m)$ at outcome level $y$. Its inverse, $F_{Y_{t}(d,m)}^{-1}(q) = \inf \{y : F_{Y_t(d,m)}(y) \geq q \}$, is the quantile function of $Y_t(d,m)$ at rank $q$. The total QTE are denoted by $\Delta_1(q) = F_{Y_1(1,M(1))}^{-1}(q) -F_{Y_1(0,M(0))}^{-1}(q)$. The QTE can be disentangled into the direct quantile effects, denoted by $\theta_1(q,d) = F_{Y_1(1,M(d))}^{-1}(q) -F_{Y_1(0,M(d))}^{-1}(q)$, and the indirect quantile effects, denoted by $\delta_1(q,d) = F_{Y_1(d,M(1))}^{-1}(q) -F_{Y_1(d,M(0))}^{-1}(q)$.
The conditional distribution function in stratum $\tau$ is $F_{Y_{t}(d,m)|\tau}(y) = \Pr(Y_t(d,m) \leq y |\tau)$ and the corresponding conditional quantile function is $F_{Y_t(d,m)|\tau}^{-1}(q) = \inf \{y : F_{Y_{t}(d,m)|\tau}(y) \geq q \}$ for $\tau \in \{a,c,d,n\}$. Using the previously described stratification framework, we define the QTE conditional on $\tau \in \{a,c,de,n\}$: $\Delta_1^{\tau}(q) = F_{Y_1(1,M(1))|\tau}^{-1}(q)-F_{Y_1(0,M(0))|\tau}^{-1}(q)$. The direct quantile treatment effect among never-takers equals $\Delta_1^{n} (q)= F_{Y_1(1,0)|n}^{-1}(q)-F_{Y_1(0,0)|n}^{-1}(q) =\theta_1^{n}(q)$. The direct quantile effect among always-takers equals $\Delta_1^{a} (q)= F_{Y_1(1,1)|a}^{-1}(q)-F_{Y_1(0,1)|a}^{-1}(q) =\theta_1^{a}(q)$. The total QTE among compliers equals $\Delta_1^{c}(q) = F_{Y_1(1,1)|c}^{-1}(q)-F_{Y_1(0,0)|c}^{-1}(q)$, the direct quantile effect among compliers equals $\theta_1^{c}(q,d) = F_{Y_1(1,d)|c}^{-1}(q)-F_{Y_1(0,d)|c}^{-1}(q)$, and the indirect quantile effect among compliers equals $\delta_1^{c}(q,d) = F_{Y_1(d,1)|c}^{-1}(q)-F_{Y_1(d,0)|c}^{-1}(q)$. Finally, we define the direct quantile treatment effects conditional on specific values $D=d$ and mediator states $M=M(d)=m$,
with the quantile function $F_{Y_t(d,m)|D=d,M(d)=m}^{-1}(q) = \inf \{y : F_{Y_{t}(d,m)|D=d,M(d)=m}(y) \geq q \}$ and the distribution function $F_{Y_{t}(d,m)|D=d,M(d)=m}(y) = \Pr(Y_t(d,m) \leq y |D=d,M(d)=m)$.
We subsequently define various functions of the observed data required for the identification results. The conditional distribution function of the observed outcome $Y_t$ conditional on treatment value $d$ and mediator state $m$, is given by $F_{Y_{t}|D=d,M=m}(y) = \Pr(Y_t \leq y |D=d,M=m)$ for $d,m \in \{0,1\}$. The corresponding conditional quantile function is $F_{Y_{t}|D=d,M=m}^{-1}(q) = \inf \{y : F_{Y_t|D=d,M=m}(y) \geq q \}$. Furthermore,
is the quantile-quantile transform of the conditional outcome from period 0 to 1 given treatment $d$ and mediator status $m$. This transform maps $y$ at rank $q$ in period 0 ($q = F_{Y_{0}|D=d,M=m}(y)$) into the corresponding $y'$ at rank $q$ in period 1 ($y'= F_{Y_{1}|D=d,M=m}^{-1}(q)$).
This sections discusses the identifying assumptions along with the identification results for the various direct and indirect effects. We note that our assumptions could be adjusted to only hold conditional on a vector of observed covariates. In this case, the identification results would hold within cells defined upon covariate values. In our main discussion, however, covariates are not considered for the sake of ease of notation. For notational convenience, we maintain throughout that $\Pr(T=t, D=d, M=m)>0$ for $t,d,m$ $\in\{1,0\}$, implying that all possible treatment-mediator combinations exist in the population in both time periods. Our first assumption implies that potential outcomes are characterized by a continuous nonparametric function, denoted by $h$, that is strictly monotonic in a scalar $U$ that reflects unobserved heterogeneity.\\ Assumption 1: Strict monotonicity of continuous potential outcomes in unobserved heterogeneity.\\ The potential outcomes satisfy the following model: $Y_t(d,m)= h(d,m, t, U)$, with the general function $h$ being continuous and strictly increasing in the scalar unobservable $U \in \mathbb{R}$ for all $d,m,t \in \{0,1\}$.\\ Assumption 1 requires the potential outcomes to be continuous implying that there is a one-to-one correspondence between a potential outcome's distribution and quantile functions, which is a condition for point identification. For discrete potential outcomes, only bounds on the effects could be identified, in analogy to the discussion in at06 for total (rather than direct and indirect) effects. Assumption 1 also implies that individuals with identical unobserved characteristics $U$ have the same potential outcomes $Y_t(d,m)$, while higher values of $U$ correspond to strictly higher potential outcomes $Y_t(d,m)$. Strict monotonicity is automatically satisfied in additively separable models, but Assumption 1 also allows for more flexible non-additive structures that arise in nonparametric models.
The next assumption rules out anticipation effects of the treatment or the mediator on the outcome in the baseline period. This assumption is plausible if assignment to the treatment or the mediator cannot be foreseen in the baseline period, such that behavioral changes affecting the pre-treatment outcome are ruled out.\\ Assumption 2: No anticipation effect of $M$ and $D$ in the baseline period.\\ $Y_0(d,m) - Y_0(d',m') = 0\mbox{, for } d, d', m, m' \{1,0\}.$ \\ Similarly, at06 and cha15 assume the assignment to the treatment group does not affect the potential outcomes as long as the treatment is not yet realized.
Furthermore, we assume conditional independence between unobserved heterogeneity and time periods given the treatment and no mediation. \\ Assumption 3: Conditional independence of $U$ and $T$ given $D=1,M=0$ or $D=0,M=0$.\\ (a) $U\perp\!\!\!\perp T|D=1,M=0$,\\ (b) $U\perp\!\!\!\perp T|D=0,M=0$.\\ Under Assumption 3a, the distribution of $U$ is allowed to vary across groups defined upon treatment and mediator state, but not over time within the group with $D=1,M=0$. Assumption 3b imposes the same restriction conditional on $D=0,M=0$. Assumption 3 thus imposes stationarity of $U$ within groups defined on $D$ and $M$. This assumption is weaker than (and thus implied by) requiring that $U$ is constant across $T$ for each individual $i$. For example, Assumption 3 is satisfied in the fixed effect model $U = \eta + v_t$, with $\eta$ being a time-invariant individual-specific unobservable (fixed effect) and $v_t$ an idiosyncratic time-varying unobservable with the same distribution in both time periods.
at06 and cha15 impose time invariance conditional on the treatment status, $U\perp\!\!\!\perp T|D=d$, to identify the average treatment effect on the treated, $\varphi_1=E[Y_1(1,M(1))-Y_1(0,M(0))|D=1]$ or local average treatment effect, $\varphi_1=E[Y_1(1,M(1))-Y_1(0,M(0))|\tau=c]$, respectively. We additionally condition on the mediator status to identify direct and indirect effects.
For our next assumption, we introduce some further notation. Let $F_{U|d,m}(u) )= \Pr(U \leq u |D=d,M=m)$ be the conditional distribution of $U$ with support $\mathbb{U}_{dm}$. \\ Assumption 4: Common support given $M=0$.\\ (a) $\mathbb{U}_{10}\subseteq \mathbb{U}_{00}$,\\ (b) $\mathbb{U}_{00}\subseteq \mathbb{U}_{10}$.\\ Assumption 4a is a common support assumption, implying that any possible value of $U$ in the population with $D=1,M=0$ is also contained in the population with $D=0,M=0$. Assumption 4b imposes that any value of $U$ conditional on $D=0,M=0$ also exists conditional on $D=1,M=0$. Both assumptions together imply that the support of $U$ is the same in both populations, albeit the distributions may generally differ.
Assumptions 1 to 3 permit identifying direct effects on mixed populations of never-takers and defiers as well as never-takers and compliers, respectively, as formally stated in Theorem 1.\\ Theorem 1: Under Assumptions 1–3,
Proof. See Appendix (ref).
To identify direct effects on further populations, we invoke a conditional independence assumption that is in the spirit of Assumption 3, but refers to different combinations of the treatment and the mediator.\\ Assumption 5: Conditional independence of $U$ and $T$ given $D=0,M=1$ or $D=1,M=1$.\\ (a) $U\perp\!\!\!\perp T|D=0,M=1$,\\ (b) $U\perp\!\!\!\perp T|D=1,M=1$.\\ Under Assumption 5a, the distribution of $U$ is allowed to vary by treatment and mediator group, but not over time conditional on $D=0,M=1$. Assumption 5b imposes the same restriction conditional on $D=1,M=1$.
Assumption 6 is similar to Assumption 4, but imposes common support conditional on $M=1$ rather than $M=0$.\\ Assumption 6: Common support given $M=1$.\\ (a) $\mathbb{U}_{01}\subseteq \mathbb{U}_{11}$,\\ (b) $\mathbb{U}_{11}\subseteq \mathbb{U}_{01}$.\\ Assumptions 6a implies that any possible value of $U$ in the population with $D=0,M=1$ is also contained in the population with $D=1,M=1$. Assumptions 6b states that any value of $U$ conditional on $D=1,M=1$ exists conditional on $D=0,M=1$.
Theorem 2 shows the identification of the direct effects on mixed populations of always-takers and defiers as well as always-takers and compliers.\\ Theorem 2: Under Assumptions 1-2, 5,
Proof. See Appendix (ref).
In the instrumental variable framework, any direct effects of the instrument are typically ruled out by imposing the exclusion restriction, in order to identify the causal effect of an endogenous regressor on the outcome, see for instance Imbens+94. By considering $D$ as instrument and $M$ as endogenous regressor, $\theta_1^{1,0}(1)=\theta_1^{0,0}(0)=\theta_1^{0,1}(0)=\theta_1^{1,1}(1)=0$ yield testable implications of the exclusion restriction under Assumptions 1-6.
So far, we did not impose exogeneity of the treatment or mediator. In the following, we assume treatment exogeneity by invoking independence between the treatment and the potential post-treatment variables.\\ Assumption 7: Independence of the treatment and potential mediators/outcomes.\\ $ \{Y_t(d,m),M(d)\} \perp\!\!\!\perp D \mbox{, for all } d,m,t, \in \{0,1\}.$\\ Assumption 7 implies that there are no confounders jointly affecting the treatment on the one hand and the mediator and/or outcome on the other hand. It is satisfied under treatment randomization as in successfully conducted experiments. This allows identifying the ATE: $\Delta_1 = E[Y_1|D=1] -E[Y_1|D=0]$.
Furthermore, we assume the mediator to be weakly monotonic in the treatment.\\ Assumption 8: Weak monotonicity of the mediator in the treatment.\\ $\Pr(M(1) \geq M(0)) =1.$\\ Assumption 8 is standard in the instrumental variable literature on local average treatment effects when denoting by $D$ the instrument and by $M$ the endogenous regressor, see Imbens+94 and an96. It rules out the existence of defiers.
As discussed in the Appendix (ref), the total ATE $\Delta_1= E[Y_1 |D=1]- E[Y_1 |D=0]$ and QTE $\Delta_1(q) = F_{Y_{1} |D=1}^{-1}(q)- F_{Y_{1} |D=0}^{-1}(q)$ for the entire population are identified under Assumption 7. Furthermore, Assumptions 7 and 8 yield the strata proportions, denoted by $p_{\tau}= \Pr(\tau)$, as functions of the conditional mediator probabilities given the treatment, which we denote by $p_{(m|d)}=\Pr(M=m|D=d)$ for $d, m \in \{0,1\}$ (see Appendix (ref)):
Furthermore, Assumptions 2, 7, and 8 imply that (see Appendix (ref))
Therefore, a rejection of the testable implication $E[Y_0|D=1] -E[Y_0|D=0]=0$ in the data would point to a violation of these assumptions.
Assumptions 7 and 8 permit identifying additional parameters, namely the total, direct, and indirect effects on compliers, and the direct effects on never- and always-takers, as shown in Theorems 3 to 5. This follows from the fact that defiers are ruled out and that the proportions and potential outcome distributions of the various principal strata are not selective w.r.t.\ the treatment.\\ Theorem 3: Under Assumptions 1–3, 7-8,
Proof. See Appendix (ref).
Theorem 4: Under Assumptions 1–2, 5, 7-8,
Proof. See Appendix (ref).
Theorem 5: Under Assumptions 1-3, 5, 7-8,
Proof. See Appendix (ref).
As in Assumption 5.1 of at06, we assume standard regularity conditions, namely that conditional on $T=t$, $D=d$, and $M=m$, $Y$ is a random draw from that subpopulation defined in terms of $t,d,m$ $\in$ $\{1,0\}$. Furthermore, the outcome in the subpopulations required for the identification results of interest must have compact support and a density that is bounded from above and below as well as continuously differentiable. Denote by $N$ the total sample size across both periods and all treatment-mediator combinations and by $i$ $\in$ $\{1,...,N\}$ an index for the sampled subject, such that $(Y_i,D_i,M_i,T_i$) correspond to sample realizations of the random variables $(Y,D,M,T$).
The total, direct, and indirect effects may be estimated using the sample analogy principle, which replaces population moments with sample moments ma88. For instance, any conditional mediator probability given the treatment, $\Pr(M=m|D=d)$, is to be replaced by an estimate thereof in the sample, $ \frac{\sum_{i=1}^{N} I\{M_i=m,D_i=d\}}{\sum_{i=1}^{N} I\{D_i=d\}}$. A crucial step is the estimation of the quantile-quantile transforms. The application of such quantile transformations dates at least back to juh91, see also cha15, wue19, and str19 for recent applications. First, it requires estimating the conditional outcome distribution, $F_{Y_t|D=d,M=m}(y)$, by the conditional empirical distribution $\hat{F}_{Y_t|D=d,M=m}(y)=\frac{1}{\sum_{i=1}^{n}I\{D_i=d, M_i=m, T_i=t\}}\sum_{i:D_i=d,M_i=m,T_i=t}I\{Y_i\leq y\}$. Second, inverting the latter yields the empirical quantile function $\hat{F}_{Y_t|D=d,M=m}^{-1}(q)$. The empirical quantile-quantile transform is then obtained by
This permits estimating the average and quantile effects of interest. Average effects are estimated by replacing any (conditional) expectations with the corresponding sample averages in which the estimated quantile-quantile transforms enter as plug-in estimates. Taking $\theta_1^{1,0}$ (see Theorem 1) as an example, an estimate thereof is
Likewhise, quantile effects are estimated based on the empirical quantiles.
For the estimation of total ATE and QTE, at06 show that the resulting estimators are $\sqrt{N}$-consistent and asymptotically normal, see their Theorems 5.1 and 5.3. These properties also apply to our context when splitting the sample into subgroups based on the values of a binary treatment and mediator (rather than the treatment only). For instance, the implications of Theorem 1 in at06 when considering subsamples with $D=1$ and $D=0$ carry over to considering subsamples with $D=1, M=0$ and $D=0, M=0$ for estimating the average direct effect on never-takers. In contrast to at06, however, some of our identification results include the conditional mediator probabilities $\Pr(M=m|D=d)$. As the latter are estimated with $\sqrt{N}$-consistency, too, it follows that the resulting effect estimators are again $\sqrt{N}$-consistent and asymptotically normal. We use a non-parametric bootstrap approach to calculate the standard errors. cha15 show the validity of the bootstrap approach for such kind of estimators, which follows from their asymptotic normality.
For the case that identifying assumptions to only hold conditional on observed covariates, denoted by $X$, estimation must be adapted to allow for control variables. Following a suggestion by at06 in their Section 5.1, basing estimation on outcome residuals in which the association of $X$ and $Y$ has been purged by means of a regression is consistent under the additional assumption that the effects of $D$ and $M$ are homogeneous across covariates. As an alternative, mel15 propose a flexible semiparametric estimator that does not impose such a homogeneity-in-covariates assumption and show $\sqrt{N}$-consistency and asymptotic normality.
To shape the intuition for our identification results, this section presents a brief simulation based on the following data generating process (DGP):
independent of each other, and
Treatment $D$ as well as the observed time period $T$ are randomized, while the mediator-outcome association is confounded due to the unobserved time constant heterogeneity $U$. The potential outcome in period $1$ is given by $Y_1(d,M(d'))=\Lambda((1+d+M(d')+d\cdot M(d'))+U)$, where $\Lambda$ denotes a link function. If the latter corresponds to the identity function, our model is linear and implies a homogeneous time trend $T$ equal to 1. If $\Lambda$ is nonlinear, the time trend is heterogeneous, which invalidates the common trend assumption of difference-in-differences models. $M$ is not only a function of $D$ and $U$, but also of the unobserved random term $V$, which guarantees common support w.r.t.\ $U$, see Assumptions 4 and 6. Compliers, always-takers, and never-takers satisfy, respectively: $c=I\{U+V\leq 0, 1+U+V>0\}$, $a=I\{U+V>0\}$, and $n=I\{1+U+V\leq0\}$.
In the simulations with 1,000 replications, we consider two sample sizes ($N=1,000, 4,000$) and investigate the behaviour of our change-in-changes methods as well as the difference-in-differences approach of Deuchertetal2017 in both a linear ($\Lambda$ equal to identity function) and nonlinear outcome model where $\Lambda$ equals the exponential function. To implement the change-in-changes estimators in the simulations as well as the application in Section (ref), we make use of the `cic' command in the qte R-package by Callaway2016 with its default values.
Table (ref) reports the bias, standard deviation (`sd'), root mean squared error (`rmse'), true effect (`true'), and the relative root mean squared error in percent of the true effect (`relr') of the respective estimators of $\theta_1^n$, $\theta_1^a$, $\Delta_c$, $\theta_1^c(1)$, $\theta_1^c(0)$, $\delta_1^c(1)$, and $\delta_1^c(0)$ for the linear model. In this case, the identifying assumptions underlying both the change-in-changes (Panel A.) and difference-in-differences (Panel B.) estimators are satisfied. Specifically, the homogeneous time trend on the individual level satisfies any of the common trend assumptions in Deuchertetal2017, while the monotonicity of $Y$ in $U$ and the independence of $T$ and $U$ satisfies the key assumptions of this paper. For this reason any of the estimates in Table (ref) are close to being unbiased and appear to converge to the true effect at the parametric rate when comparing the results for the two different sample sizes.
Table (ref) provides the results for the exponential outcome model, in which the time trend is heterogeneous and interacts with $U$ through the nonlinear link function. While the change-in-changes assumptions hold (Panel A.), average time trends are heterogeneous across complier types such that the difference-in-differences approach (Panel B.) of Deuchertetal2017 is inconsistent. Accordingly, the biases of the change-in-changes estimates generally approach zero as the sample size increases, while this is not the case for the difference-in-differences estimates. Change-in-changes yields a lower root mean squared error than the respective difference-in-differences estimator in all but one case (namely $\hat{\delta}_1^c(0)$ with $N=1,000$) and its relative attractiveness increases in the sample size due to its lower bias.
In our final simulation design, we maintain the exponential outcome model but assume $D$ to be selective w.r.t.\ $U$ rather than random. To this end, the treatment model in (ref) is replaced by $D=I\{U+Q>0\}$, with the independent variable $Q\sim N(0,1)$ being an unobserved term. Under this violation of Assumption 7, complier shares and effects are no longer identified, which is confirmed by the simulation results presented in Table (ref). The bias in the change-in-changes based total, direct, and indirect effects on compliers do not vanish as the sample size increases. Furthermore, under non-random assignment of $D$ (while maintaining monotonicity of $M$ in $D$), the never-takers' and always-takers' respective distributions of $U$ differ across treatment. Therefore, average direct effects among the total of never or always-takers, respectively, are not identified. Yet, $\theta_1^{1,0}$, which is still identified by the same estimator as before, yields the direct effect among never-takers with $D=1$ (as defiers do not exist). Likewise, $\theta_1^{0,1}$ corresponds to the direct effect on always-takers with $D=0$. Indeed, the results in Table (ref) suggest that both parameters are consistently estimated with the change-in-changes model (Panel A.).
Our empirical application is based on the JOBS II data by VinokurPrice1999. JOBS II was a randomized job training intervention in the US, designed to analyse the impact of job training on labour market and mental health outcomes, see VinokurPriceSchul1995. It was a modified version of the earlier JOBS programme, which had been found to improve labour market outcomes such as job satisfaction, motivation, earnings, and job stability, see Caplanetal1989 and VinokurRynGramlichPrice1991, as well as mental health, see VinokurPriceCaplan1991. According to the results of VinokurPriceSchul1995, the JOBS II programme increased reemployment rates and improved mental health outcomes, especially for participants having an elevated risk of depression. The JOBS interventions had an important impact in the academic literature Wanberg2012,Liuetal2014 and the methodology was implemented in field experiments in Finland Vuorietal2002,Vuorietal2005 and the Netherlands Brenninkmeijer2011, suggesting positive effects on labour market integration in either case.
The JOBS II intervention was conducted in south-eastern Michigan, where 2,464 job seekers were eligible to participate in a randomized field experiment, see VinokurPrice1999. In a baseline period prior to programme assignment, individuals responded to a screening questionnaire that collected pre-treatment information on mental health. Based on the latter, individuals were classified as having either a high or low depression risk and those with a high risk were oversampled before the training was randomly assigned.\footnote{In the JOBS II intervention, randomization was followed by yet another questionnaire sent out two weeks before the actual job training, see VinokurPriceSchul1995, which also provided information on whether an individual had been assigned the training. Consequently, the data collected in that questionnaire must be considered post-treatment as they could be affected by learning the assignment. Therefore, we rely on the earlier screening data as the relevant pre-treatment period prior to random programme assignment.} The job training consisted of five 4-hours seminars conducted in morning sessions during one week between March 1 and August 7, 1991. Members of the treatment group who participated in at least four of the five sessions received USD 20. Each of the standardized training sessions consisted, among other aspects, of the learning and practicing of job search and problem-solving skills.\footnote{When compared to the earlier JOBS programme Caplanetal1989, the job training sessions of JOBS II focused more strongly on building a sense of mastery, personal control and self- efficacy in job search. Previous research had suggested that an increase in this sense of mastery, control and self-efficacy improved observed effort in job search behaviour EdenAviram1993. Results in MarshallLang1990 suggest that mastery is a strong predictor of depression symptoms among women. For a detailed discussion of the literature, the exact sampling process, the training programme, and further aspects, see VinokurPriceSchul1995.} The control group received a booklet with information on job search methods VinokurPriceSchul1995.
We analyse the impact of job training on mental health, namely symptoms of depression 6 months after training participation. The health outcome ($Y$) is based on a 11-items index of depression symptoms of the Hopkins Symptom Checklist. For example, respondents were asked how much they were bothered by symptoms such as crying easily, feeling lonely, feeling blue, feeling hopeless, having thoughts of ending their lives, or experiencing a loss of sexual interest. The questions were coded on a 5-point scale, going from `not at all' (1) to `extremely' (5), and summarized in a depression variable that consists of the average across all questions.
One-sided non-compliance with the random assignment is a major issue in JOBS II. While the study design rules out always-takers because members of the control group did not have access to the job training programme, 45% of those assigned to training in our data did not participate and are therefore never-takers, the remaining 55% are compliers. In order to avoid selection bias w.r.t\ actual participation, the original JOBS II study by VinokurPriceSchul1995 analysed the total effect of the policy (i.e.\ the intention-to-treat effect), including those who, despite receiving an offer to participate, did not take part in the job training. In contrast, we use our methodology to separate the direct effect of mere training assignment, which is our treatment $D$, from the indirect effect operating through actual training participation, which is our mediator $M$, among compliers.\footnote{ImKeTi09 analyse Jobs II in a mediation context as well, but consider a different mediator, namely job search self-efficacy, and a different identification strategy based on selection on observables.} We also consider the direct effect on never-takers, which likely differs from that on the compliers. While being offered (or not offered) the job training might affect compliers' mental health by inducing motivation/enthusiasm (or discouragement), it may not have the same effect among never-takers, who do not attend such seminars whatsoever.
More concisely, we base identification on Theorem 3a with Assumption 4a for the average direct effect on never-takers, $\theta_1^n$, on Theorem 3b with Assumption 4 for the direct effect on compliers under $d = 0$, $\theta_1^c(0)$, and Theorem 5 with Assumptions 4b and 6a for the indirect effect on compliers under $d = 1$, $\delta_1^c(1)$. None of these approaches requires the presence of always-takers in the sample. We also note that if random assignment operated through other mechanisms than actual participation in any of the subpopulations as it may appear reasonable in the context of mental health outcomes, this would violate the exclusion restriction when using assignment as instrumental variable for actual participation in a two stage least squares regression. Given that our identifying assumptions hold, our approach can therefore be used to statistically test the exclusion restriction.
Our evaluation sample consists of a total of 3,360 observations in the pre-treatment and post-mediator periods with non-missing information for $D$, $M$, and $Y$. It is an unbalanced panel due to attrition of roughly 13% of the initial respondents between the two periods. Table (ref) provides summary statistics for the outcome in the total sample as well as by treatment group over time. We verify whether randomization was successful by comparing the outcome means of the treatment and control groups in the pre-treatment period ($T=0$) just prior to the randomization of $D$. The small difference of 0.01 is not statistically significant according to a two sample t-test. Furthermore, the standardized difference test suggested by RosenbaumRubin1985 yields a value of just 1.68 and is thus far below 20, a threshold frequently chosen for indicating problematic imbalances across treatment groups. To test for potential attrition bias we also consider these statistics in the pre-treatment period exclusively among the panel cases that remain in the sample in the post-mediator period (not reported in Table (ref)). The p-value of the t-test amounts to 0.52 and the standardized difference of 3.5 is low such that attrition bias does not appear to be a concern. We therefore do not find statistical evidence for a violation of the random assignment of $D$ in our sample. Table (ref) also reports the mean difference in outcomes in the post-mediator period ($T=1$) 6 months after participation, which is an estimate for the total (or intention-to-treat) effect of $D$. The difference of 0.08 is statistically significant at the 5% level.
Table (ref) presents the estimation results based on our CiC approach and the DiD strategy of Deuchertetal2017 when (linearly) controlling for the gender of respondents in either case. Standard errors rely on cluster bootstrapping the direct and indirect effects 1999 times, where clustering is on the respondent level. The CiC and DiD estimates of the direct effects on never-takers, $\hat{\theta}_1^n(0)$, as well as on compliers, $\hat{\theta}_1^c(0)$, are not statistically significant at conventional levels. Hence, we do not find statistical evidence for a direct effect of the mere assignment into the training programme on the depression outcome, which would point to a violation of the exclusion restriction when using assignment as instrument for participation. In contrast, we find for both CiC and DiD negative total effects among compliers $\hat{\Delta}_c$ that are statistically significant at least at the 10% level. In the case of CiC, also the negative indirect effect among compliers, $\hat{\delta}_1^c(1)$, is significant at the 5% level, while this is not in the case for DiD. By and large, our results point to a moderately negative treatment effect on depressive symptoms through actual programme participation, rather than through other (i.e.\ direct) mechanisms. The CiC estimates $\hat{\delta}_1^c(1)$ and $\hat{\Delta}_c$ are in fact rather similar to the result of a two stage least squares regression relying on the exclusion restriction by using $D$ as instrument for $M$. The latter approach yields a local average treatment effect on compliers in the post-mediator period of -0.14 with a heteroskedasticity-robust standard error of 0.07 (significant at the 5% level).
We proposed a novel identification strategy for causal mediation analysis with repeated cross sections or panel data based on changes-in-changes (CiC) assumptions that are related but yet different to at06 considering total treatment effects. Strict monotonicity of outcomes in unobserved heterogeneity and distributional time invariance of the latter within groups defined on treatment and mediator states are key assumptions for identifying direct effects within these groups. Additionally assuming random treatment assignment and weak monotonicity of the mediator in the treatment permits identifying direct effects on never-takers and always-takers as well as total, direct, and indirect effects on compliers. We also provided a brief simulation study and an empirical application to the Jobs II programme.