On the plausibility of the latent ignorability assumption

Abstract: { The estimation of the causal effect of an endogenous treatment based on an instrumental variable (IV) is often complicated by attrition, sample selection, or non-response in the outcome of interest. To tackle the latter problem, the latent ignorability (LI) assumption imposes that attrition/sample selection is independent of the outcome conditional on the treatment compliance type (i.e.\ how the treatment behaves as a function of the instrument), the instrument, and possibly further observed covariates. As a word of caution, this note formally discusses the strong behavioral implications of LI in rather standard IV models. We also provide an empirical illustration based on the Job Corps experimental study, in which the sensitivity of the estimated program effect to LI and alternative assumptions about outcome attrition is investigated. }

{ Keywords: instrument, non-response, attrition, sample selection, latent ignorability.}\\ { JEL classification: C21, C24, C26. }

{ { Address for correspondence: Martin Huber, University of Fribourg, Bd.\ de P\'{e}rolles 90, 1700 Fribourg, Switzerland, [email removed].}\thispagestyle{empty} }

{ \setcounter{footnote}{0} \setcounter{footnote}{0} \setcounter{page}{1} }

Introduction

A frequently encountered complication when estimating the effect of a potentially endogenous treatment based on an instrumental variable (IV) methods is attrition/sample selection/non-response bias in the outcome. To account for this problem, the missing at random (MAR) assumption (e.g.\ \citeasnoun{Ru76b}), for instance, requires outcome attrition to only depend on observable variables. Alternatively, \citeasnoun{FrangakisRubin99} propose a latent ignorability (LI) restriction, which assumes attrition to be independent of the outcome conditional on the instrument and the treatment compliance type (i.e. whether one is a complier or non-complier in the notation of \citeasnoun{Angrist+96}). In the IV framework both assumptions can be combined (e.g.\ \citeasnoun{MealliImbens04}), imposing independence conditional on the compliance type, the instrument, and further observables.

We argue that LI is nevertheless quite restrictive, as attrition is not allowed to be related to unobservables affecting the outcome in a very general way. Section (ref) formally discusses the strong behavioral implications of LI in standard IV models with non-response. This assumption should therefore be cautiously scrutinized in applications. As an example, consider \citeasnoun{Barnard+03}, who assess a randomized voucher program for private schooling with noncompliance (where the IV is the randomization and the treatment is private schooling) and attrition in the test score outcomes, because some children did not take the test. Unobservables as ability or motivation likely affect both test taking and test scores. LI (combined with MAR) requires that conditional on the compliance type (i.e.\ private schooling as a function of voucher receipt), voucher assignment, and observed covariates, test taking is not related to ability or motivation (and thus, test scores). Among compliers (only in private schooling when randomized in), those taking the test must thus have the same distribution of ability and motivation as those abstaining. However, even within compliers, heterogeneity in ability and motivation may be sufficiently high to selectively affect test taking such that LI fails. Section (ref) provides an empirical illustration using the Job Corps experimental study, in which the estimated program effect under LI is compared to alternative assumptions about outcome attrition.

IV models with nonresponse

Assume the following parametric IV model with nonresponse:

$Y$ is the outcome of interest, $D$ is the binary (and potentially endogenous) treatment, and $R$ is the response indicator. Note that $1(\cdot)$ is the indicator function that is equal to one if its argument is satisfied and zero otherwise. $Y$ is only observed if $R=1$ and unknown if $R=0$, implying non-response, sample selection, or attrition. $Z$ is a randomly assigned instrument affecting $D$ (but not directly $Y$ or $R$) and assumed to be binary, e.g., the randomization indicator in an experiment. $U,V,W$ denote arbitrarily associated unobservables, $\alpha_0,\alpha_1,\beta_0,\beta_1,\gamma_0,\gamma_1$ are coefficients.

\citeasnoun{Angrist+96} define four compliance types, denoted by $T$, based on how the potential treatment status depends on the instrument: An individual is a complier (defier) if her potential treatment state is one (zero) in the presence and zero (one) in the absence of the instrument and an always-taker (never-taker) if the potential treatment is always (never) one, independent of the instrument. Assume that $\beta_1$ is positive (a symmetric case could be made for a negative $\beta_1$). Then, an individual is a complier if $\beta_0+\beta_1 \geq V>\beta_0$, an always taker if $\beta_0 \geq V$, and a never taker if $\beta_0+\beta_1<V$. Defiers do not exist due to the positive sign of $\beta_1$.

We now impose the following latent ignorability (LI) assumption, see \citeasnoun{FrangakisRubin99}, and critically assess it in the light of our standard IV model with attrition:\\ Assumption 1 (latent ignorability): $Y \bot R | Z, T$ (where `$\bot$' denotes independence),\\ which is equivalent to $Y \bot R | Z, D,T$ as $Z$ and $T$ perfectly determine $D$. Furthermore, we assume that the error term $U$ is continuous, such that $Y$ is continuous. Finally, for the moment we also impose that $U=V=W$ such that the same unobservable (e.g.\ motivation) affects the outcome (e.g.\ test score), treatment (e.g.\ private schooling), and response (e.g.\ test taking).

Note that Assumption 1 implies that the distribution of $U$ among compliers is the same across response states given the instrument:

where $f(\cdot)$ denotes an arbitrary function with a finite expectation and the second line follows from the parametric model in (ref). Obviously, the joint satisfaction of $U=V=W$ and (ref) is impossible in this context, as the distribution of $U$ conditional on $\gamma_0+\gamma_1 \geq U$ and $\gamma_0+\gamma_1 < U$, respectively, is non-overlapping. An analogous impossibility result holds for $E(f(Y)| Z=0, T=c, R=1)=E(f(Y)|Z=0, T=c, R=0)$, which is also implied by Assumption 1.

Imposing $U=V=W$ seems too extreme for most applications and was chosen for illustrative purposes. However, even if the unobserved terms in the various equations are not the same, but non-negligibly correlated as commonly assumed in IV models, identification may seem questionable. Suppose, for instance, that $W=\delta_1 V + \epsilon$, where $\epsilon$ is random noise and $\delta_1$ is a coefficient. Then, Assumption 1 and the model in (ref) imply that

If $U$ is associated with either $\epsilon$, $V$, or both, the latter equality does not hold in general, but only if the association of $U, \epsilon$, $V$ is of a very specific form, which raises concerns about Assumption 1.

Finally, we investigate an in terms of functional form assumptions more general IV model, where $Y$, $D$, and $R$ are given by nonparametric functions denoted by $\phi$, $\psi$, and $\eta$, respectively:

Under this model, Assumption 1 implies that

This can be satisfied in special cases, for instance if $U=\pi 1(\psi(1,V)\geq 0, \psi(0,V)< 0)+\varepsilon$, with $\pi$ denoting the (homogeneous) effect of being a complier and $\varepsilon$ being random noise. Then, (ref) simplifies to $E(f(\epsilon)| Z=1, \psi(1,V)\geq 0, \psi(0,V)< 0, \eta(1, W) \geq 0 ) =E(f(\epsilon)| Z=1, \psi(1,V)\geq 0, \psi(0,V)< 0, \eta(1, W) < 0)$, which holds because $\epsilon$ is independent of $W$. In general, identification requires that $T$ is a sufficient statistic to control for the endogeneity introduced by conditioning on $R$. This, however, implies that the association between $U$, $V$, and $W$ is quite specific, otherwise Assumption 1 does not hold.

Empirical illustration

As an illustration for treatment evaluation under LI and alternative assumptions about attrition, we consider the experimental evaluation of the U.S.\ Job Corps program (see for instance \citeasnoun{ScBuGl01}), providing training and education for young disadvantaged individuals. We aim at estimating the effect of program participation ($D$) in the first or second year after randomization into Job Corps ($Z$) on log weekly wages of females in the third year ($Y$). Of the 4,765 females in the experimental sample with observed treatment status, wages are only observed for 3,682 individuals ($R=1$), while 1,083 do not report to work.

Reconsidering the IV model of ((ref)), we assume that in each of $\phi$, $\psi$, and $\eta$ a vector of observed covariates, denoted by $X$, may enter as additional explanatory variables. Similar to \citeasnoun{FrHu14}, Section 2.2, we assume that (i) Assumption 1 holds conditional on $X$ (thus combining LI and MAR), (ii) $U \bot Z | X, T$ such that the instrument affects the outcome only through the treatment, (iii) $T \bot Z| X$ which is implied by random assignment, (iv) $\Pr(T=c)>0$ and $\Pr(T=d)=0$ so that compliers exist and defiers are ruled out, and (v) $0<\Pr(Z=1|X)<1$, ensuring common support in the covariates across instrument states. $X$ (measured prior to randomization) includes education, ethnicity, age and its square, school and working status, and receipt of Aid to Families with Dependent Children (AFDC) and food stamps.

We compare sempiparametric LATE estimation based on the latter assumptions (see Theorem 1 in \citeasnoun{FrHu14}) to (i) MAR-based LATE estimation as in Section 2.3 of \citeasnoun{FrHu14} (assumptions: $Y \bot R | X, Z, D$, $(U,T) \bot Z | X$, $\Pr(T=c)>0$, $\Pr(T=d)=0$, $0<\Pr(Z=1|X)<1$), (ii) the Wald estimator among those with $R=1$ (ignoring sample selection), and (iii) the method of \citeasnoun{FrFrHuLe2015}, which tackles sample selection and treatment endogeneity by two distinct instruments. In the latter approach, which allows for non-ignorable selection related to $U$ in a more general way than LI, we use the number of kids younger than 6 in the household 2.5 years after random assignment as instrument for $R$. We apply a semiparametric version of the estimator outlined in equation (23) of \citeasnoun{FrFrHuLe2015} along with the weighting function in their expression (21).

Table (ref) provides descriptive statistics for the covariates, the treatment, and the instruments in the total sample and for working and not working females. Across the latter groups for instance education, aid receipt, previous job status, and Job Corps participation differ importantly, pointing to non-random selection into employment. Table (ref) presents the effect estimates, standard errors, and p-values based on 1999 bootstraps using the quantile method. The effect under LI + MAR (based on Theorem 1 of \citeasnoun{FrHu14}) of 0.12 log points virtually identical to the Wald estimator which ignores sample selection bias, and both are statistically significantly different from zero. The MAR-based estimate is one third higher, but not significantly differently so. The method of \citeasnoun{FrFrHuLe2015} based on two instruments (2 IVs) yields virtually the same effect as MAR and is neither statistically significantly different from any other estimator, nor from zero at any conventional level.

It seems important to understand the differences in the behavioral assumptions of the estimators. LI + MAR, for instance, assumes that given the covariates and program assignment, unobservables like ability and motivation do not jointly affect employment and wages among compliers. In constrast, the method of \citeasnoun{FrFrHuLe2015} does not rely on this restriction and allows for more general forms of sample selection, at the cost of also requiring a valid instrument for employment. In our illustration, the results turned out to be rather robust to the different assumptions considered, which need not necessarily hold in other contexts.

\setlength\baselineskip{14.0pt} \scriptsize