Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
174,177 characters · 28 sections · 88 citation commands
Inference on LATEs with covariates
\thispagestyle{empty} \setcounter{page}{1}
With endogenous treatment and a binary instrument, imbens1994identification show that the two-stage least squares (TSLS) estimand has a causal interpretation as a local average treatment effect (LATE). Empirically, instrument validity is generally argued conditional on available controls. The causal interpretation of the TSLS estimand is lost if we linearly include these controls unless this is a correct parametric assumption \citep*{blandhol2022tsls}. In the absence of a credible justification to linearly include the controls, angrist1995two show that TSLS can consistently estimate a weighted average of LATEs provided that the number of possible values of the vector of controls is fixed: a researcher can select a saturated specification that includes (i) a dummy for each unique realized value of the vector of controls and (ii) interactions of the instrument with these dummies. However, in most empirical settings the vector of controls has rich support, and saturated TSLS breaks down. As a result, applied work continues to use the more parsimonious linear specification at the risk of targeting a non-causal estimand.
This paper proposes a new method for inference on the weighted average of LATEs targeted by saturated TSLS. We provide asymptotically valid tests and confidence intervals with sufficient power to detect empirically realistic causal effect sizes, while allowing the number of control dummies and instrument interactions to be of the same order as the sample size. As saturated specifications contain only dummy variables, we can provide low-level assumptions for our results. The most important and rather mild assumption is that each covariate group contains at least two individuals for which the instrument is active and two for which the instrument is inactive. Since the conditional validity of an instrument often relies on categorical controls representing different groups or strata, and our method extends to multivalued treatments and multiple or multivalued instruments, our method applies to a wide variety of empirically relevant settings in applied economics.
Conducting valid inference in a saturated specification is a challenging task mogstad2024instrumental. Even with 10 binary controls, there are 1,024 possible values of the covariate vector, and for each possible value a covariate-group dummy and an instrument interaction is included in the regression. When the individuals induced to change treatment status by the instrument and the variation in the instrument are not evenly distributed across the covariate groups, interacting a strong instrument with control dummies may result in instrument interactions that are only weakly related to the treatment. Hence, even in the most favorable saturated specification with a single strong binary instrument, inference has to take many and possibly weak instruments into account. Standard inferential procedures for instrumental variables are known to be biased in the presence of many and weak instruments bekker1994alternative,staiger1997instrumental.
While there is an extensive literature that combines the notion of many and weak instruments, this either excludes unobserved heterogeneity in treatment effects or a large number of covariate groups. For instance, \citet*{bekker2003finite, chao2005consistent,hausman2012instrumental, crudu2021inference,mikusheva2021inference,matsushita2020jackknife,lim2022conditional}, among others, focus on the linear instrumental variable model with a homogeneous slope coefficient. lee2018consistent develops a TSLS variance estimator that allows for heterogeneous treatment effects, which however is not robust to the large number of instrument interactions in typical saturated specifications. kolesar2013estimation proposes estimators free of many and weak instrument bias while allowing for heterogeneous treatment effects. However, inferential procedures based on these estimators have only been developed under the assumption that the number of control dummies is a negligible fraction of the sample size evdokimov2018inference.
This paper conducts inference on the weighted average of LATEs that is robust against the key features of saturated economic data: unobserved treatment effect heterogeneity, many control dummies and instrument interactions, weak identification strength, and conditional heteroskedasticity. We first show that the recently proposed estimator from \citet*{chao2023jackknife}, designed to estimate a homogeneous slope coefficient in a panel data setting, consistently estimates a weighted average of LATEs in the setting of saturated instrumental variable estimation (SIVE). We then show that in the present context, it is crucial to take treatment effect heterogeneity into account in the inference stage, as it has a nontrivial effect on the variance that standard variance estimators do not capture. Throughout, the analysis allows the number of control dummies and instrument interactions to be asymptotically non-negligible relative to the sample size. For consistency, the condition that we impose on the identification strength has been shown by mikusheva2021inference to be the weakest possible. Our results also provide a fully identification-robust procedure.
We conduct a series of Monte Carlo simulations which set-up mimics key features of the data used in card1995using, and has also been used by blandhol2022tsls. The results illustrate that our proposed estimator is median unbiased, even with a large number of covariate groups and weak instrument strength. Our inference method yields nominal size control regardless of the instrument strength when the number of covariate groups is small, and becomes progressively more conservative when the number of covariate groups increases under weak identification. At the same time, our method has sufficient statistical power to detect empirically realistic causal effect sizes. In contrast, linear and fully saturated TSLS and various jackknife estimators incur a bias that increases with the number of covariate groups and as the strength of the instrument decreases, and the corresponding tests show large size distortions.
The empirical relevance of our method for treatment evaluation is illustrated with empirical applications from different fields of applied economics: a judge design study from stevenson2018distortion, an experiment on banking access by dupas2018banking, and in the supplementary material, we further study the effect of schooling card1995using, health insurance \citep*{finkelstein2012oregon}, and an educational program \citep*{muralidharan2019disrupting}. We find that our method produces realistic LATE estimates in settings where the literature has found issues with existing methods for inferring LATEs. In settings in which assumptions underlying TSLS are harmless, our method produces very similar results with a negligible loss in precision. Hence, our inference method is able to detect statistically significant LATEs, while the applied researcher is assured that results are not driven by the often implicitly made assumptions underlying existing methods.
To the best of our knowledge, we are the first to tackle the statistical challenges in inferring the weighted average of covariate-specific LATEs as derived by angrist1995two that explicitly allows for weak monotonicity: the instrument may induce treatment take-up for some covariate groups, while it may discourage take-up for other groups. Weak monotonicity has been shown to be empirically relevant in judge designs aizer2015juvenile,mueller2015criminal,sigstad2023monotonicity, information provision experiments vilfort2023interpreting, and a wide range of other applied economic settings de2017tolerating. sloczynski2020should provides a review of 25 articles published in AEA journals between 2009 and 2015 and finds that 22 papers report at least one TSLS regression with individuals induced in and out of treatment.
A strand of related literature aims to infer the unconditional LATE with semiparametric estimators abadie2003semiparametric, non-parametric estimators frolich2007nonparametric, and machine learning techniques \citep*{chernozhukov2018double}. The unconditional LATE weighs the covariate-specific reduced forms and first stages with the probability density function of the covariates. This parameter only has a causal interpretation under the notion of strong monotonicity: The instrument moves treatment take up in the same direction for all covariate groups. Under this assumption, the instrument interactions are not required for identification, which removes the inferential challenges with many and weak instruments.
The remainder of this article is organized as follows. Section (ref) explains the current practice of inferring LATEs with covariates from the data and its challenges. Section (ref) introduces our proposed causal estimand and its inference procedure, supported by large sample theoretical results. Section (ref) discusses the Monte Carlo simulations, Section (ref) the empirical applications, and Section (ref) concludes.
Suppose we are interested in the causal effect of a binary treatment $T_i$ on an outcome $Y_i$, for individuals $i=1,\dots,n$. For each individual, define the potential outcomes $Y_i(1)$ and $Y_i(0)$ corresponding to the values of $Y_i$ if individual $i$ is treated or not treated, respectively. Hence, the treatment effect is defined as $Y_i(1)-Y_i(0)$. The treatment is potentially endogenous, and a binary instrument $Q_i$ and a vector of covariates $X_i$ are available to help to identify a causal effect. The developed theory applies equally well to the extensions to multivalued treatments and instruments in angrist1995two.
Define $\mathbb{X}=\{x_1,\dots,x_G\}$ as the set of all possible $G$ unique realizations of $X_i$. Denote the potential treatment statuses $T_i(1)$ and $T_i(0)$ corresponding to the values of $T_i$ if individual $i$'s treatment assignment is given by $Q_i=1$ and $Q_i=0$, respectively. In case the outcome $Y_i$ also directly depends on $Q_i$, its corresponding potential outcomes are given by $Y_i(Q_i,T_i)$. If we condition on the covariates, the four instrumental variable (IV) assumptions in the imbens1994identification framework are the following.
These assumptions allow for complete treatment effect heterogeneity across all individuals, and do not impose any parametric assumptions. Following the exclusion restriction, we write $Y_{i}(T_{i})$ for the potential outcomes. The relevance assumption holds if there are individuals with $T_i(1)\neq T_i(0)$ for each $X_i$. These individuals are either compliers ($T_i(1) > T_i(0)$) or defiers ($T_i(1) < T_i(0)$). We weaken the relevance assumption in Section (ref) to allow for subpopulations with no compliers and no defiers. The monotonicity assumption is referred to in blandhol2022tsls as weak monotonicity, because it allows the effect of the instrument on the treatment to have a different direction for each $X_i$. In other words, we can have compliers for some values of $X_i$ and defiers for other values of $X_i$. Strong monotonicity requires $\mathbb{P}[T_{i}(1)\geq T_{i}(0)]=1$ or $\mathbb{P}[T_{i}(1)\leq T_{i}(0)]=1$, and therefore assumes that the effect of switching on the instrument on potential treatment status is (weakly) in the same direction for all individuals.
Within the LATE framework, causal effects are estimated of the form
The causal effect is then a positively weighted average of covariate-specific LATEs. The following well-known result, see for instance angrist2009mostly, shows that the covariate-specific LATEs $\tau(x_g)$ are indeed identified.
In practice, the number of observations corresponding to each $X_i$ is usually small, and the moments in Lemma (ref) cannot be accurately estimated. Therefore, practitioners commonly rely on regression to estimate a weighted average of the covariate-specific LATEs as the parameter of interest, where the weights are automatically selected through the regression model that is specified. The default regression specification includes the covariates linearly in the first and second stage. However, blandhol2022tsls show that if this parametric assumption is incorrect, then TSLS has no causal interpretation.
angrist1995two show that $\tau$ can be estimated by TSLS in saturated specifications: the first stage includes dummies for each possible value of $X_i$ and a full set of interactions between these dummies and the instrument, and the second stage includes the treatment variable and the control dummies. We define the $G\times1$ vector $W_i$ with elements $W_{ig}={1}[X_i=x_g]$, indicating the covariate group of individual $i$. The $G\times1$ vector $Z_i$ contains the instrument interactions $Z_{ig}=Q_i{1}[X_i=x_g]$. Note that $\sum_g W_{ig}=1$ and $\sum_g Z_{ig} = Q_i$. Define $n_g = \sum_i W_{ig}$ as the number of individuals in covariate group $g$, and $m_{1,g} = \sum_i Z_{ig}$ and $m_{0,g} = n_g-m_{1,g}$ as the number of individuals in covariate group $g$ with an active and inactive instrument, respectively.
Both the full set of covariate group indicators and the full set of instrument interactions are required for nonparametric estimation of $\tau$ by TSLS. In this specification, the conditional expectation of the instrument given the covariate groups is linear in the covariate group indicators. This allows for correctly partialling out the covariates, which otherwise may induce negative weights into $\tau$. Without the instrument interactions, the first stage does not necessarily reproduce the direction of the monotonicity assumption in all covariate groups. Hence, omission of the instrument interactions requires the direction of the monotonicity to be invariant to the covariate group.
It is clear that saturation can only work if we observe each covariate value more than once. Moreover, to achieve identification, each covariate group has to include both individuals with an active and an inactive instrument. The setup we consider, with many control dummies and many instrument interactions, requires the number of observations in each group to satisfy the following assumption.
This assumption requires that both the number of individuals with an active instrument and nonactive instrument has to be larger than one in each covariate group. Hence, without additional assumptions, our setup focuses on discrete controls. This applies to a wide range of applications, as the conditional validity of instrumental variables is most commonly argued on the basis of categories or strata. Assuming that both values of the instrument can be observed in each covariate group, is similar to an overlap or common support assumption.
While saturation results in a causal TSLS estimand if the number of possible covariate values is small, it is not straightforward to find a causal estimand in the empirically more common setting in which the controls have rich support. Since each group requires an indicator, and the instrument is interacted with these indicators, this automatically results into a large set of control dummies and a large set of instruments. In this setting, TSLS is known to be biased, see e.g.\ bekker1994alternative.
Moreover, the instrument interactions may weaken the instrument strength. Instrument strength is measured by the first stage signal, which can be written as $\text{FS}=\sum_g \mathbb{P}[X_i=x_g]\pi(x_g)^2 \mathbb{V}[Q_i|X_i=x_g]$ with complier or defier shares $\pi(x_g)=\mathbb{P}[T_i(1) \neq T_i(0)|X_i=x_g]$ and treatment assignment variation $\mathbb{V}[Q_i|X_i=x_g]$. If the complier or defier share and the variation in treatment assignment are homogeneous across covariate groups, that is $\pi(x_g)=\pi$ and $\mathbb{V}[Q_i|X_i=x_g]=\mathbb{V}[Q_i]$, the strength of the instrument interactions $Z$ equals the strength of the instrument $Q$. However, in settings where $\pi(x_g)$ is large in groups with a low number of treated or untreated units, while $\pi(x_g)$ is small in groups with a number of treated units close to half of the number of group members, the first stage is likely weak. A large number of potentially weak instruments can exacerbate the bias in TSLS, see e.g.\ chao2005consistent. We impose the following assumption on the identification strength, which ensures existence of a causal estimand and weakens Assumption (ref).3.
Under Assumption (ref), we need to have a nonzero probability to find compliers or defiers in at least one covariate group. Even for that covariate group, the number of compliers or defiers may be small, as Assumption (ref) allows $\mathbb{P}[T_i(1) \neq T_i(0)|X_i=x_g]$ to converge to zero as the sample size increases.
Estimators that have been proposed in settings with many or weak instruments often assume the first stage and reduced form errors to be independent across observations:
We follow suit, and maintain this assumption throughout this section to illustrate the estimation challenges with existing methods. In Supplemental Appendix (ref), we discuss how to relax this assumption to allow for more general correlation structures in our proposed method.
Define the $n$-dimensional vectors $ Y = (Y_{1},\ldots,Y_{n})'$ and $ T = (T_{1},\ldots,T_{n})'$, and the $n \times G$-dimensional matrices $ W = (W_{1}',\ldots, W_{n}')'$ and $Z = ( Z_{1}',\ldots, Z_{n}')'$. Define the residual maker matrix $M_{W} = I_n - W(W'W)^{-1}W'$ with $I_n$ the $n$-dimensional identity matrix. The TSLS estimand is commonly defined as
where $P=M_{W}Z(Z'M_{W}Z)^{-1}Z'M_{W}$ partials out the controls $W$ from the first stage and the second stage. However, this estimand does not identify $\tau$ when the number of covariate groups is large as the following result makes precise.
This result can be found in evdokimov2018inference. We provide a short proof in Appendix (ref) as some of the steps are used to derive subsequent results. Note that $\tilde{\text{P}}[X_i=x_g]$, $\tilde{\text{V}}[Q_i|X_i=x_g]$, and $\tilde{\text{P}}[Q_i=1|X_i=x_g]$ are the sample analogues of ${\mathbb{P}}[X_i=x_g]$, ${\mathbb{V}}[Q_i|X_i=x_g]$, and ${\mathbb{P}}[Q_i=1|X_i=x_g]$.
Lemma (ref) shows that when both the second term in the numerator and the second term in the denominator go to zero faster than the first term in the numerator and denominator for $\beta^{\text{TSLS}}$, we identify $\tau$ with the weights in (ref) equal to $\omega^{\text{TSLS}}(x_g) = \frac{\tilde{\emph{P}}[X_i=x_g]\pi(x_g)^2 \tilde{\emph{V}}[Q_i|X_i=x_g]}{\sum_g \tilde{\emph{P}}[X_i=x_g]\pi(x_g)^2 \tilde{\emph{V}}[Q_i|X_i=x_g]}$. It is clear that this is the case when $P_{ii}$ is small. Since $ P_{ii}=\sum_g W_{ig}\left(Z_{ig}\frac{1}{n_g}\frac{m_{0,g}}{m_{1,g}} + (1-Z_{ig})\frac{1}{n_{g}}\frac{m_{1,g}}{m_{0,g}}\right)$, TSLS has a causal interpretation if in each covariate group the number of individuals for which the instrument is active ($m_{1,g}$) and the number of individuals for which the instrument is inactive ($m_{0,g}$) is large. This requirement becomes more stringent as the strength of the instrument interactions measured by $\sum_g \tilde{\emph{P}}[X_i=x_g]\pi(x_g)^2 \tilde{\emph{V}}[Q_i|X_i=x_g]$ decreases.
In empirical applications, the number of covariate groups is generally large and nonnegligible relative to the number of individuals. In this case, at least a number of covariate groups has to contain a small number of observations, and the additional terms in Lemma (ref) do not go to zero. This bias is known as the many instrument bias of TSLS. Under homogenous treatment effects, and assuming homoskedastic errors, one can follow bekker1994alternative in using LIML to avoid this bias. However, kolesar2013estimation points out that with treatment effect heterogeneity, the estimand of LIML is generally not causal.
It follows from Lemma (ref) that the bias in TSLS is due to the diagonal elements $P_{ii}$. A frequently used approach to reduce many instrument bias is to employ a jackknife-style correction \citep*{angrist1999jackknife,ackerberg2009improved}. In the current setting, we could remove the diagonal of $P$ using the diagonal matrix $D_P$ with $[D_{P}]_{ii}=P_{ii}$, to obtain the estimand
This diagonal removal has been the basis of recent papers in the literature on identification-robust inference under many instrument sequences \citep*{mikusheva2021inference,crudu2021inference,matsushita2020jackknife}. It ensures that the many instrument bias present in TSLS disappears. However, in the presence of a potentially large set of control variables, the consequence of removing the diagonal elements $P_{ii}$ is that the controls are no longer projected out.
An alternative jackknife approach is to first partial out the controls before removing the diagonal. This gives the following estimand
ackerberg2009improved refer to this estimand as the improved JIVE (IJIVE), where they additionally scale the matrix $(P-D_P)$ with $(I-D_P)^{-1}$ in (ref). Although IJIVE appropriately handles the controls, it reintroduces a many instrument bias as $[M_{W}(P-D_{P})M_{W}]_{ii}=[P-M_{W}D_{P}M_{W}]_{ii}\neq 0$. Supplemental Appendix (ref) derives the bias expressions for JIVE and IJIVE.
We now consider an estimand identical to that of TSLS when the number of possible values of the vector of controls is fixed, but that does not suffer from the many instrument bias when the number of covariate values grows. Define the matrix $V=[W,Z]$ consisting of the covariate group indicators and the instrument interactions, and the residual maker matrix ${M}_{W,Z}=I-V(V'V)^{-1}V'$. The SIVE estimand is specified as
where $D$ is a diagonal matrix with diagonal elements such that $P_{ii}=[{M}_{W,Z}D{M}_{W,Z}]_{ii}$. It follows that (ref) is a jackknife estimand, which removes the diagonal of $P$ and hence the many instrument bias in the TSLS estimand. At the same time, by pre- and post-multiplying $D$ by ${M}_{W,Z}$, the controls are projected out correctly and the bias in JIVE is prevented. In addition to the controls, ${M}_{W,Z}$ also projects out the instrument interactions, and therefore does not affect the causal estimand that TSLS is targeting.
The SIVE estimator has been proposed by \citet*{chao2023jackknife} as Fixed Effect Jackknife IV (FEJIV). They show that the diagonal elements of $D$ can be obtained by solving a system of linear equations with a unique solution, and derive consistency results in a linear instrumental variable panel data model with fixed effects, where the number of instruments increases slowly with the sample size. Within our saturated setting, we can derive a closed-form expression for $D$, and show that the estimand identifies a weighted average of covariate-specific LATEs. These results require a weaker set of assumptions as we allow the number of control dummies and instrument interactions to be asymptotically non-negligible relative to the sample size. Furthermore, Assumption (ref) relaxes Assumption 6 in chao2023jackknife that requires $m_{1,g}\geq 3$ and $m_{0,g}\geq 3$.
The following result shows that the diagonal matrix $D$ in (ref) exists under Assumption (ref).
The proof is deferred to Appendix (ref). The result shows that when the number of covariate values is small and both $m_{1,g}$ and $m_{0,g}$ are large, the diagonal elements $D_{ii}$ are small and the estimator reduces to the TSLS estimator.
Because the term that is subtracted in the numerator and denominator of (ref) is orthogonal to the instruments and controls, it is straightforward to establish that the SIVE estimand has a causal interpretation that is identical to the unbiased TSLS estimand, without requiring the number of control dummies or instrument interactions to be small.
The proof is deferred to Appendix (ref).
While having a causal estimand is a crucial first step, we also need to be able to infer the estimand from the data. In this section, we therefore consider the testing problem
for a given $\beta_{0}$. The testing procedure is standard and based on the fact that under $H_{0}$,
where we provide conditions on the strength of the instrument interactions under which $\lambda=1$ or $\lambda\leq 1$. In (ref), $\hat{\beta}^{\text{SIVE}}$ is the sample analogue of (ref) and given by,
The crucial part to make the test operational is to find an estimator for the variance of $\hat{\beta}^{\text{SIVE}}$ that appropriately accounts for treatment effect heterogeneity, as well as the presence of many control dummies and many instrument interactions. Let $v_{i} = \varepsilon_{i}-u_{i}\beta^{\text{SIVE}}$ and denote by $\sigma_{u,i}^2 = \mathbb{E}[u_{i}^2|Q,X]$, $\sigma_{v,i}^2 = \mathbb{E}[v_{i}^2|Q,X]$, and $\sigma_{uv,i} = \mathbb{E}[u_{i}v_{i}|Q,X]$. We propose the following variance estimator for $\hat{\beta}^{\text{SIVE}}$ that is decomposed into two parts,
The first part of (ref) is given by
where $A=P-M_{W,Z}DM_{W,Z}$ with $D$ as defined in Lemma (ref), and $D_{\hat{\sigma}_{u}^2}$, $D_{\hat{\sigma}_{v}^2}$ and $D_{\hat{\sigma}_{uv}}$ are diagonal matrices with $[D_{\hat{\sigma}_{u}^2}]_{ii}=\hat{\sigma}_{u,i}^2$, $[D_{\hat{\sigma}_{v}^2}]_{ii}=\hat{\sigma}_{v,i}^2$, and $[D_{\hat{\sigma}_{uv}}]_{ii}=\hat{\sigma}_{uv,i}$ on their respective diagonals. In the presence of many instruments, standard heteroskedasticity robust Eicker-Huber-White variance estimators are inconsistent cattaneo2018inference. We therefore consider the hartley1969variance (HRK) variance estimators that are given by,
When $\hat{\beta}^{\text{SIVE}}$ is replaced by its population counterpart $\beta^{\text{SIVE}}$, the estimators are unbiased conditional on the instrument and covariates. This removes the main driver of the inconsistency of standard heteroskedasticity robust variance estimators.
The second part of (ref) corrects for an upward bias in (ref) stemming from the use of the HRK variance estimators instead of the corresponding population quantities. In Appendix (ref) we show how to estimate these interactions to form the bias-correction term $\hat{\mathbb{V}}_{2}$. This term eliminates the bias if $m_{1,g}\geq 4$ and $m_{0,g}\geq 4$ and partially removes the bias when $m_{1,g}\geq 3$ and $m_{0,g}\geq 3$.
To study the asymptotic properties of the test statistic in (ref) with variance estimator (ref), we impose the following assumptions. Throughout, $c$ denotes a generic positive constant that can differ between occurrences.
Assumption (ref) ensures that the distribution of the test statistic is nondegenerate and controls the behavior of the estimators for the conditional variances of $u_{i}$ and $\varepsilon_{i}$.
Assumptions (ref) to (ref) allow for the inference on the weighted average of LATEs in (ref) in a wide range of empirically relevant settings. First, the treatment effects $Y_i(1)-Y_i(0)$ are allowed to be heterogeneous across all $i$. It follows that treatment effects may vary across the covariate groups defined by the elements of $\mathbb{X}$, and hence SIVE identifies a weighted average of potentially heterogeneous conditional treatment effects. This is in line with the LATE identification literature. Note that the literature on inference in IV models often assumes that the data satisfies a model along the lines of
in which the treatment effect of $T_i$ on $Y_i$ is modelled with $\beta$ which is specified to be homogeneous. Since we conduct inference with saturated instruments and covariates, no parametric assumptions or a model specification is required.
Second, the first stage and reduced form errors $u_i$ and $\varepsilon_i$ are allowed to be heteroskedastic. Although heteroskedasticity has been accounted for in existing methods with many instruments (e.g.\ hausman2012instrumental, mikusheva2021inference, crudu2021inference), these consider IV models as in (ref) with homogeneous effects. In our setting, assuming homoskedastic errors would also implicitly restrict treatment effect heterogeneity.
Third, inference based on our test statistic in (ref) is asymptotically valid under a minimal assumption on the identification strength. That is, the test works with sets of instrument interactions that have a strong or a semi-strong signal. The instrument interactions are referred to as strong when the concentration parameter
Since $|\pi(x_g)|\leq 1$ and $\text{Var}[Q_i|X_i=x_g]\leq \frac{1}{4}$, in the scenario with the strongest identification possible within each covariate group, we require $\frac{n}{4} \gg G$ for the instrument interaction set to be considered as strong. In this case, we show that the SIVE estimator is consistent, the variance estimator (ref) is consistent, and the asymptotic confidence intervals constructed from the test statistic attain nominal coverage. Under semi-strong identification, that is if$\frac{n}{\sqrt{G}}\text{FS}\rightarrow\infty$ and $G\rightarrow\infty$, the SIVE estimator remains consistent and inference may be conservative if there exist covariate groups with $m_{1,g}\leq 3$ or $m_{0,g}\leq 3$. mikusheva2021inference show that this is the weakest identification strength under which a consistent estimator exists. In the case that one is worried that the identification is even weaker so that $\frac{n}{\sqrt{G}}\text{FS} \rightarrow C<\infty$ for some constant $C\in[0,\infty)$ and $G \rightarrow \infty$, the results we provide allow for the construction of fully identification robust confidence intervals by inverting a score statistic as in kleibergen2005testing.
We first provide a consistency result for $\hat{\beta}^{\text{SIVE}}$.
The proof is deferred to Appendix (ref). The condition on the identification strength covers strong identification and semi-strong identification as defined in Section (ref). The most stringent case occurs when $G\propto n$, in which case we require that $\sqrt{n}\text{FS}\rightarrow_{p}\infty$. Even then, $\text{FS}$ is allowed to {decrease} to zero asymptotically.
The following result shows the asymptotic validity of a standard $t$-test that uses the variance estimator from (ref). What is particularly important to note is that the theorem does not limit the rate at which $G$ can grow with $n$. In particular, we allow $G/n\rightarrow c\in (0,1)$ which are the many-instrument sequences by bekker1994alternative.
The proof is deferred to Appendix (ref). Theorem (ref) shows that under strong identification or with sufficiently large groups through a strengthening of Assumption (ref), we achieve asymptotically nominal coverage. In the presence of smaller groups and weaker identification, the procedure becomes conservative. If $m_{1,g}\geq 3$ and $m_{0,g}\geq 3$, Theorem (ref) shows that we can precisely quantify how conservative the procedure maximally is. Technically, the most challenging part to establish Theorem (ref) is that the variance estimator converges to a quantity that is equal to, or at least as large as the population variance, conditional on the covariates $X$ and instrument $Q$.
We illustrate the finite sample properties of the proposed inference method relative to existing methods in an empirically relevant setting. We build on the Monte Carlo experiment considered in blandhol2022tsls that is designed to match some key features of the data used by card1995using. We consider the inference method proposed in Section (ref) and the estimators discussed in Section (ref): TSLS including the controls linearly (TSLS-L), TSLS with a saturated specification (TSLS-S), and the jackknife estimators JIVE, IJIVE, and UJIVE.
Set-up: Consider a sample with $n=3,000$ observations. The endogenous treatment and the outcome variable are generated as
where $\Phi(.)$ denotes the cumulative distribution function of a standard normal random variable. We generate $(u_{i},\varepsilon_{i})'\sim N( 0,\Sigma)$ where $[\Sigma]_{11}=[\Sigma]_{22}=1$ and $[\Sigma]_{12}=0.527$. We set $\gamma_i=1.2$ across all $i$, which corresponds to $\tau=1.2$. We extend this to heterogeneous treatment effects in Supplemental Appendix (ref). We set $p_{0}=0.22$ and vary the identification strength through $p_{1}\in\{0.39,0.49,0.59,0.69\}$. The binary instrument satisfies
We have a single control variable $X_{i}$ that can take on $G=\{1,50, 200\}$ values taken from a one-dimensional Halton sequence.
Bias: We first study the bias of the estimators. Figure (ref) shows the absolute median bias as a function of the instrument strength $p_1$, where values for the absolute bias are truncated at one. The small circles correspond to $G=1$ covariate groups, the medium-sized circles to $G=50$ groups, and the large circles to $G=200$ groups.
We find that SIVE is unbiased across all settings under consideration. For estimators commonly used in applied work, TSLS-L and TSLS-S, the bias becomes substantial when the number of covariate groups increases. In this setting, TSLS-L does not correspond to a causal estimand and TSLS-S suffers from many instrument bias, which is more pronounced in settings where the instruments are weak. From the jackknife estimators, JIVE is not performing much better. Since the controls are not appropriately projected out in this estimator, it incurs a large omitted variable bias. IJIVE and UJIVE are performing considerably better, with the former only incurring bias when there are many weak instruments and the latter is unbiased in all settings.
Size: Second, we examine the empirical size of a test of $H_{0}\colon \beta=\tau$ with $\tau=1.2$ and a nominal test size of 5%. Figure (ref) shows the test size as a function of the instrument strength, where values for the size are truncated at 0.2.
SIVE obtains close to nominal size control for $G=1$ for all values of the instrument strength. As follows from Theorem (ref), the test becomes progressively more conservative for a larger number of covariate groups and increasing the instrument strength makes the test less conservative. For TSLS-L, TSLS-S, and JIVE we observe substantial size distortions for all instrument strengths. This can be explained by the biases observed in Figure (ref). IJIVE and UJIVE also show substantial deviations from nominal size. This can be explained by the assumptions under which the variance estimators for these estimators are proposed: They require the number of control dummies to be a negligible fraction of the sample size.
In settings with heterogeneous treatment effects, we find similar or slightly less conservative sizes with SIVE compared to the homogeneous settings, but the size of the other methods further moves away from the nominal size. Instead of $\gamma_i=1.2$ in (ref), we set $\gamma_i=6$ for observations in covariate groups with fewer observations than the median sized covariate group, and $\gamma_i=1.2$ for the remaining observations. Detailed results are reported in Supplemental Appendix (ref). This appendix also shows that potential alternative variance estimators, such as the ones proposed by lee2018consistent and chao2023jackknife, do not control size in settings with treatment effect heterogeneity and/or many instrument interactions.
Power: Third, we consider the statistical power of a test of $H_{0}\colon \beta=0$ with a nominal test size of 5%. Since our proposed method is the only one that accurately controls size, we focus on the power of SIVE. Figure (ref) shows the power curves of SIVE with different numbers of covariate groups and varying instrument strength. We consider values of $\beta$ between -1.2 and 1.2, from which the latter is set as an empirically realistic effect in this setting by blandhol2022tsls. Hence, we consider power curves in an empirically relevant range of treatment effects. Although Figure (ref) shows that SIVE can be conservative, we find that the test still has power even when the number of covariate groups is large and the instruments are weak. For modest or strong instruments, power is close to one when $\beta=1.2$, even with a large number of covariate groups.
This section illustrates the empirical relevance of the proposed method for treatment evaluation. We consider the LATE of pretrial detention using a judge design and the LATE of banking access using a field experiment. Supplemental Appendix (ref) discusses empirical applications to the LATEs of schooling, health insurance, and an educational program.
First, we discuss the LATE of pretrial detention on conviction using the judge design data studied by stevenson2018distortion. The data has also been recently analyzed by mogstad2024instrumental and sloczynski2020should. The initial study already discusses violations of strong monotonicity caused by judges being relatively strict for certain types of defendants but relatively lenient for other types of defendants.
The sample consists of 331,971 arrests between 2006 and 2013 from the Philadelphia court records. Each case is randomly assigned to one of seven judges. For ease of exposition, we select the two judges with the lowest and highest proportion of cases in which the defendant is detained pretrial in 2006 and 2007. To ensure the exogeneity of the judge selection, we then consider the sample of 86,765 cases assigned to one of these two judges between 2008 and 2013. The binary outcome variable equals one if the case outcome is conviction, and the binary treatment variable equals one if the defendant is detained pretrial. The binary instrument equals one if the case was assigned to the more lenient judge in 2006-2007. Following one of the control specifications in sloczynski2020should, our analysis includes 19 controls: 17 offense type dummies and two race dummies.
The first three columns of Table (ref) show the TSLS-L, TSLS-S, and SIVE estimates, with their corresponding standard errors. TSLS-L relies on the parametric assumption of a linear relation between the controls and the outcome and treatment, and a strong monotonicity assumption on the instrument. The estimator is inconsistent if either of those assumptions is violated, which probably explains the implausible magnitude of the estimate. The TSLS-S estimate is more realistic, but may suffer from a many and potentially weak instrument bias. The SIVE estimate is slightly larger than the TSLS-S estimate, but substantially smaller than the TSLS-L estimate. Using the appropriate variance estimator, the standard error is considerably smaller than using the standard variance estimator for TSLS-L and slightly larger than the standard error of TSLS-S. We conclude that our proposed method is able to correct for issues in LATE estimation in the commonly used judge design. The SIVE estimate is in line with the literature, corrects for the bias in TSLS estimates caused by violations of the linearity assumption on the controls or the strong monotonicity assumption on the instruments, and is statistically significant.
Second, we consider a field experiment for which we have no a priori indications that the TSLS estimates are problematic. dupas2018banking study a randomized opportunity to open a free bank account to estimate the LATE of opening a bank account on savings balance. We focus on the LATE on savings in formal financial institutions in Uganda. The outcome variable is savings stocks in 2010 US dollars at formal financial institutions including commercial banks, microfinance banks, and savings and credit cooperatives. The binary treatment variable equals one for households who actually opened the accounts and made at least one deposit. The binary instrument equals one for households offered a voucher to open bank accounts with no financial costs. The analysis includes 43 controls; 41 strata dummies cover the stratified randomization by gender, occupation, and bank branch. Two wave dummies indicate the three rounds of follow-up surveys, administered approximately 6, 12, and 18 months after accounts were opened. The sample size is 6,007, standard errors are clustered at the household level, and our sample includes 2,085 households.
The final three columns of Table (ref) show the estimates for the LATE of banking access. The estimate in the first column is identical to the estimate in dupas2018banking. The differences between the estimates of TSLS-L, TSLS-S, and SIVE are small. The standard error of SIVE is slightly larger, but this does not change the qualitative results. Supplemental Appendix (ref) finds that SIVE also produces similar results to TSLS in other carefully conducted field experiments. Using SIVE in these settings, the practitioner can be assured that the results are not driven by any implicit assumptions. The SIVE confidence intervals respect the full nonparametric nature of the LATE framework, and this does not necessarily imply a substantial loss in precision.
Although TSLS with covariates is known to require saturation to obtain a causal interpretation, one of the main concerns of applied economists are the statistical challenges involved in inference with a saturated specification. We propose a method that tackles these challenges. This method requires a valid instrument that satisfies the standard imbens1994identification assumptions, and minimal assumptions on the number of observed individuals in each covariate group and the identification strength. Unlike existing procedures, we allow arbitrary treatment effect heterogeneity and covariates with rich support, while still being able to detect empirically relevant effect sizes.
{2pt plus 0.3ex}
\setcounter{page}{1}
\setcounter{equation}{0} \setcounter{figure}{0} \setcounter{table}{0} \setcounter{section}{0}
This supplemental appendix has five parts:
Consider a setting where the projection errors $u_{i}$ and $\varepsilon_{i}$ are clustered in $K$ groups indexed by $k=1,\ldots,K$ and $n_{k}$ observations per cluster. Denote by $L_{K}$ an $n\times K$ matrix with $L_{K,ik}=1$ if observation $i$ is a member of cluster $k$. For any vector $v$, denote by $v_{(k)}$ the subvector of individuals from cluster $k$. For any matrix $V$, denote by $V_{(kl)}$ the submatrix with the rows (columns) corresponding to individuals from cluster $k$ ($l$). We can accommodate the following clustering structure.
Part 1 is the analogue of the lower bounds on the variances and upper bound on the correlation in Assumption (ref) in a clustered setting. To illustrate Part 2, consider the case where clustering takes place at the household level. Assumption (ref) states that each person in a household has a different covariate vector, i.e.\ each person is unique within their household. This implies that we always have at least as many clusters as we have covariate groups. The following identification result extends Theorem (ref) to a clustered setting.
The proof is an immediate consequence of Assumption (ref). Bias under clustering could be caused by the term $u'A \varepsilon$. Although by construction $\mathbb{E}[u_i\varepsilon_{i}|Q,X]A_{ii}=0$, potentially $\mathbb{E}[u_i\varepsilon_{j}|Q,X]A_{ij}\neq 0$ if $i$ and $j$ are in the same cluster. However, for any pair of indices $\{i,j\}$ that are in the same cluster, $A_{ij}=0$ as $i$ and $j$ are not allowed to be in the same covariate group. If there are in fact multiple identical individuals within each household, Lemma (ref) would not hold. In this case, we would have to remove the off-diagonal elements from the matrix $A$ for people in the same covariance group and the same cluster.
We also have the following result.
Proof: With the notation as in Appendix (ref), under Assumption (ref) it still holds that $\mathbb{E}[S_{n}|Q,X]=0$. For the variance of $S_{n}$, we have
The second line is obtained as in ligtenberg2023inference, the first inequality uses Assumption (ref) and the final inequality is the same as in Appendix (ref). Likewise, we obtain that $\mathbb{E}[D_{n}|Q,X]=r_{n}^{1/2}H_{n}$ and $\mathbb{V}(D_n|Q,X)\leq c$ $a.s.$ The proof is completed as in Appendix (ref).
For inference, we adapt (ref) using the generalization of the HRK variance estimators to the clustered setting considered in boot2023unbiased.
Let $F$ be an $n^2\times \sum_{k=1}^{K}n_{k}^2$ matrix that collects the nonzero terms of $\text{vec}(\Sigma_{u})=Fh$. Define
Consider $\text{vec}(\hat{\Sigma}_{u})$. It is straightforward to show that this is unbiased,
The following result gives the expression for the JIVE estimand from (ref). It shows that while removing the diagonal elements has removed the many instrument bias, the estimand can be substantially different from $\tau$ and is difficult to interpret.
Proof: The JIVE estimand is
For the second term in the numerator, using (ref),
where $W'D_PW=I_G$ as $\sum_i W_{ig}^2 P_{ii}=1$, and $Z'D_PZ=W'D_PZ$ is a $G \times G$ diagonal matrix with elements $\sum_i Z_{ig}^2 P_{ii}=\frac{m_{0,g}}{n_{g}} $. Similarly, we get
Using the results in Appendix (ref), we arrive at the expression for $\beta^{\text{JIVE}}$ in Lemma (ref).
The following result shows that IJIVE defined in (ref) reintroduces the many instrument bias.
The proof is similar as for Lemma (ref) and omitted. The causal estimand again is different from TSLS, but under Assumption (ref) the weights on $\tau(x_g)$ will lie between 0 and the TSLS weights. To quantify the many instrument bias, consider a homoskedastic setting where $\mathbb{E}[u_{i}\varepsilon_{i}|X, Q] = \sigma_{u\varepsilon}$. We see that $B_{Y,2}=\frac{\sigma_{u\varepsilon}}{n}\sum_{g=1}^{G}\frac{1}{n_{g}}$ compared to the bias in TSLS that is $\frac{\sigma_{u\varepsilon}}{n}\sum_{g=1}^{G}1$. We conclude that IJIVE offers a substantial bias reduction, especially when $n_{g}$ is large. However, with rich support for the covariates, $n_{g}$ is fixed, and the bias is of the same order as for TSLS.
To show consistency of the variance estimator, we need to upper bound terms that are of the form $ S( A, L) = \sum_{i_{1},\ldots,i_{16}}|A_{i_{1}i_{2}}||A_{i_{3}i_{4}}||A_{i_{5}i_{6}}||A_{i_{7}i_{8}}|L_{i_{9}i_{10}}L_{i_{11}i_{12}}L_{i_{13}i_{14}}L_{i_{15}i_{16}}$, with some restrictions on the indices over which we sum. We will first find a matrix $ B$ such that $S( A, L)\leq S( B, L)$. To save on notation, define
We then have
where $\leq$ indicates an elementwise inequality, so that $S( A, L)\leq S( B, L)$.
We can now establish the following results
Proof: (R1) follows from the fact that $ H_{g}' H_{g} = E_{g}^{-1}$ and $\iota_{2}' E_{g}\iota_{2} = \frac{n_{g}}{m_{1,g}m_{0,g}}$. Since $\text{rank}( B_{(g)})=1$, (R2) follows from (R1). (R3) follows from the fact that $ H_{g}' H_{g} = E_{g}^{-1}$. For (R4)--(R8), note that $ L_{(g)} = H_{g} E_{g} H_{g}'$. Using this result and $ H_{g}' H_{g} = E_{g}^{-1}$ yields (R4). For (R5), we note that $ \iota_{n_{g}}'( B_{(g)}\odot L_{(g)})\iota_{n_{g}} = \text{tr}( B_{(g)} L_{(g)}) = \text{tr}( B_{(g)})$ with the last equality by (R4). For (R6) and (R7) we calculate the elementwise product,
We can now explicitly calculate bounds for (R6) and (R7),
For (R8), we have a similar result as,
This completes the proof for $(R1)-(R8)$.
To prove $(R9)-(R11)$, using the definition of $B_{(g)}$ in (ref),
Using (ref) and the fact that $H_{g}'H_{g} = E_{g}^{-1}$, $(R9)$ follows from
as $m_{1,g}m_{0,g}\geq 2(n_{g}-2)\geq 2(n_{g}-n_{g}/2) = n_{g}$ since $n_{g}\geq 4$ by Assumption (ref).
For $(R10)$, using first (ref) and then (ref),
For $(R11)$, using (ref) and (ref),
For $\mathbb{V}(B.2a.1|Q,X)$ in (ref), we need to bound terms of type
We list these terms in Table (ref) and bound them by $cr_{n}^{-2}G$.
For $\mathbb{V}(B.2a.2|Q,X)$ in (ref), we need to bound terms of type
We list these terms in Table (ref) and (ref) and bound them by $cr_{n}^{-2}G$.
The variance in (ref) converges to zero almost surely by the bounds:
Likewise, the variance in (ref) converges to zero almost surely by the bounds:
Define
For consistency, we need to show that the following converges to zero.
We will focus on the first term, where $\hat{\xi}_{11,ij}$ is mean zero by construction. Consider
We now subtract all terms with nonzero expectation and consider the following five cross-products, from which the others follow: $(I)\colon (a)\times (d)$, $(II)\colon (b)\times (d)$, $(III)\colon (a)\times (e)$, $(IV)\colon (b)\times (e)$ and $(V)\colon (c)\times (e)$.
Let $y_{ki}$ denote a random variable independent over $i=1,\ldots,N$. Let $k=1,\ldots,4$. For example, we could choose $y_{1i}=u_{i}$, $y_{2i}=u_i$, $y_{3i}=v_{i}$, $y_{4i}=v_i$, and $N=m_g$. After subtracting terms with nonzero expectation, we obtain terms of the following general structure.
We will show that the variance of these terms is at most of order $N^2$. Starting with $(I)$,
Since by Assumption (ref) we can assume that $y_{kj}$ has bounded moments up to order 8, it holds that $ \mathbb{V}(I.a)\leq cN$, $\mathbb{V}(I.b)\leq c$, $\mathbb{V}(I.c)\leq cN^2$, $\mathbb{V}(I.d)\leq cN$.
Considering $(II)$,
The variance of $(II.a)$ and $(II.b)$ can be bounded as $\mathbb{V}(II.a)\leq c$, $\mathbb{V}(II.b)\leq c.$
Considering $(III)$,
The variance of $(III.a)$ and $(III.b)$ can be bounded as $\mathbb{V}(III.a)\leq c$, $\mathbb{V}(III.b)\leq cN^2$.
Considering $(IV)$,
The variance of $(IV.a)$ and $(IV.b)$ can be bounded as $ \mathbb{V}(IV.a)\leq c$, $\mathbb{V}(IV.b)\leq c$.
Finally, considering $(V)$:
The variance of $(V.a)$ and $(V.b)$ can be bounded as $\mathbb{V}(V.a)\leq cN$, $\mathbb{V}(V.b)\leq cN^2$.
Taking $N=m_{1,g}$, these results show that
Analogous calculations show that the remaining terms in (ref) converge to zero.
Consider a group with $m_{1,g}=2$ or $m_{0,g}=2$. To avoid singularity in the variance estimators of Appendix (ref), we can use the following rescaled version of the conventional variance estimator for the observations that only share their instrument status with one other observation in the group,
Notice that here $L_{ij}=1/2$. Similarly,
The first term in the expressions coincides with the first terms in (ref). The same holds up to a scaling constant for the second term. Those terms are therefore covered by the proof in Appendix (ref). The only new terms are the last terms of the respective expressions that yield a positive bias in the variance estimator. Substituting these terms into (ref), we see that the bias contribution by observation $i$ if observation $j(i)$ is the only observation in the same covariate group with the same instrument status, is
where $ R(|\hat{\beta}-\beta|)\rightarrow_{p} 0$. This shows that the bias is positive up to an $o_{p}(1)$ term.
This appendix discusses the results from a set of additional Monte Carlo experiments. First, we consider experiments similar to the ones in Section (ref) extended with treatment effect heterogeneity. Second, we run experiments with additional variance estimators to the ones used in Section (ref).
Treatment effect heterogeneity: Consider the setup in Section (ref), with $\gamma_i=5\times1.2$ for observations in covariate groups with less observations than the median sized covariate group, and $\gamma_i=1.2$ for the remaining observations. Figure (ref) shows the absolute median bias and test size as a function of instrument strength. We find thet treatment effect heterogeneity does not affect the bias, but the test size further deteriorates for all methods but SIVE when the number of covariate groups increases.
Alternative variance estimator SIVE: For SIVE, we use the variance estimator given in (ref), which is robust to treatment effect heterogeneity. We now assess the effect of treatment effect heterogeneity on tests that rely on the variance estimator proposed by chao2023jackknife:
where $J = (M_{W}\odot M_{W})^{-1}$, $[D_{1}]_{i} = [J(\hat{\varepsilon}\odot\hat{\varepsilon})]_{i}$, $\hat{\varepsilon} = M_{W,Z}(y-T\hat{\beta}^{\text{SIVE}})=M_{W,Z}(\varepsilon-u\hat{\beta}^{\text{SIVE}})$ and $\hat{u} = M_{W,Z}T=M_{W,Z}u$. This variance estimator is proposed in the context of a linear panel data model with homogeneous slope coefficient.
Consider the setup in Section (ref), with $\gamma_i=h\times1.2$ for observations in covariate groups with less observations than the median sized covariate group, and $\gamma_i=1.2$ for the remaining observations. The treatment effect heterogeneity level $h$ ranges from 1 to 5. Figure (ref) shows the size of the test of $H_{0}\colon \beta=\tau$ in a setting with weak instruments ($p(2)=0.39$, left panel) and with strong instruments ($p(2)=0.69$, right panel). On the $x$-axis we vary the treatment effect heterogeneity level.
The red circles in the left panel confirm that our proposed method offers a conservative test under weak instruments. As expected based on the theory, the level of treatment effect heterogeneity has no effect on the size of the test. For the green circles representing the alternative variance estimator, we see that increasing the level of treatment effect heterogeneity leads to a oversized test. The fact that treatment effect heterogeneity has only a mild effect in this case is due to the fact that it is flooded by the uncertainty introduced by the presence of many weak instruments. When the instruments are strong, as in the right panel of Figure (ref), accounting for treatment effect heterogeneity becomes more important. Again, SIVE shows no dependence on the level of treatment effect heterogeneity. The alternative variance estimator now becomes progressively more oversized as the level of heterogeneity increases for all values of the number of covariate groups.
Alternative variance estimator TSLS: lee2018consistent proposes a variance estimator for TSLS that is valid in overidentified specifications where each instrument identifies a different LATE. It therefore allows for treatment effect heterogeneity. However, the analysis in lee2018consistent assumes that the number of instruments is fixed relative to the sample size.
Consider the setup in Section (ref), with $\gamma_i=5\times1.2$ for observations in covariate groups with less observations than the median sized covariate group, and $\gamma_i=1.2$ for the remaining observations. Figure (ref) shows that TSLS-S with the variance estimator proposed by lee2018consistent (TSLS-S-Lee) shows excellent size control when the number of covariate groups equals one. Also with $G=50$, TSLS-S-Lee shows a substantial improvement over the standard Eicker-Huber-White variance estimator used by TSLS-S. However, when the number of covariate groups increases, the many instrument bias manifests itself and the size increases above its nominal value.
This appendix discusses additional empirical applications to the LATEs of schooling, health insurance, and an educational program.
First, we discuss the LATE of years of schooling on log wage by card1995using. The linear TSLS estimate for this LATE has been investigated by blandhol2022tsls due to concerns about this estimator in this setting. The sample consists of the National Longitudinal Survey of Young Men (NLSYM) respondents who reported educational attainment and wages in the 1976 interview. The survey started in 1966 with 14-24 year old men. The outcome variable is log hourly wage and the treatment variable is years of education. The binary instrument equals one if there was an accredited four-year college in the local labor market of the respondent when the respondent was 14 years old. The analysis includes fifteen controls: years of potential experience itself and squared, a binary race indicator for Black, binary indicators for living in the South and in a standard metropolitan statistical area (SMSA), a set of binary indicators for region of residence in 1966, and a binary indicator for residence in an SMSA in 1966. The sample size is 3010.
The TSLS estimator linearly includes these controls (TSLS-L), which in this case does not coincide with a saturated specification. Since years of experience consists of only 24 unique values, and the remaining controls are binary, the set of covariates can be saturated. After saturation, we (1) remove the covariate groups without either an active or nonactive instrument, and (2) remove the observations without a covariate group. The resulting number of covariate groups and number of observations in the saturated specification (TSLS-S) equals 264 and 1864, respectively. Since SIVE can only consider covariate groups with at least two active and two nonactive instruments, the number of covariate groups and observations is further reduced to 111 and 1229, respectively.
The first three columns of Table (ref) show the TSLS-L, TSLS-S, and SIVE estimates, with their corresponding standard errors. The estimate in the first column is identical to the estimates in card1995using and blandhol2022tsls, and the estimate in the second column is identical to blandhol2022tsls. TSLS-L relies on the parametric assumption of a linear relation between the controls and the outcome and treatment, and a strong monotonicity assumption on the instrument. The estimator is inconsistent if either of those assumptions is violated. The TSLS-S estimate suffers from a many and potentially weak instrument bias. These biases are also pointed out by blandhol2022tsls.
The SIVE estimate is similar to the TSLS-S estimate, and albeit substantially smaller than the TSLS-L estimate, it resides in the 95% confidence intervals of both estimates. However, there is a substantial increase in the magnitude of its standard error, which accounts both for many controls, many and weak instruments, and treatment effect heterogeneity. We conclude that based on this sample, we cannot reject that the LATE of returns to schooling differs from zero, in contrast to what is suggested by the TSLS estimates.
We conclude that our proposed method is able to correct for common issues in LATE estimation in a widely studied example. The SIVE estimates for the returns to schooling are in line with the literature, and correct for the upward bias in TSLS estimates caused by violations of the linearity assumption on the controls or the strong monotonicity assumption on the instruments. In contrast to existing estimators robust to these violations, SIVE does not suffer from many or weak instrument bias, and its standard errors allow for treatment effect heterogeneity. Hence, our inference method is robust against violations of the (implicit) assumptions made by the researcher in an empirically relevant setting.
Second, we consider a carefully conducted field experiment for which we have no a priori indications that the TSLS estimates are problematic. finkelstein2012oregon use the randomized opportunity to apply for Medicaid in the Oregon Health Insurance Experiment (OHIE) to study the effects of Medicaid enrollment. In a follow-up paper, finkelstein2016effect study the impact on emergency department use in a sample of 24,646 individuals with Portland-area zip codes. The binary instrument equals one if an individual was randomly given the opportunity to apply for Medicaid. The binary outcome variable equals one if an individual visited an emergency department within 180 days of the lottery notification, and the binary treatment variable equals one if an individual was enrolled in Medicaid during the same 180-day period. The IV analysis includes twelve controls: three household size dummies, eight lottery draw dummies, and the pre-lottery value of the binary outcome variable. Standard errors are clustered at the household level, and our sample includes 22,188 different households.
Table (ref) shows the TSLS-L, TSLS-S, and SIVE estimates for the LATE of Medicaid, with their corresponding standard errors. The estimate by TSLS-L is identical to the estimate in finkelstein2016effect. The estimators using a saturated specification result in a slightly smaller LATE with very similar standard errors, leading to the same conclusions as with TSLS-L.
Although the conditional validity of instrument variables is most commonly argued on the basis of discrete controls/strata, a valid question is what happens in the presence of continuous controls. In this case, covariate groups have to be merged before the methods in this paper can be applied. The results we derive hold exactly when the merged covariate groups $g$ and $g'$ satisfy the following conditions. For $q\in\{0,1\}$,
Although restrictions on treatment effect heterogeneity are often considered to be strong, the assumptions in (ref) are considerably weaker than the necessary conditions required for a causal interpretation of the widely used linear TSLS estimand with covariates. As an alternative to (ref), one can impose a smoothness assumption on the conditional expectations and exploit the fact that we can finely discretize the control as the number of control dummies can be of the same order as the sample size. In this case, results from the matching literature abadie2006large show that the resulting bias is small when the number of continuous controls is small.
This appendix illustrates the effect of discretizing a continuous control in an empirical application. muralidharan2019disrupting use a randomized opportunity to receive free access to a technology-led instructional program (called Mindspark) to estimate the LATE of attending the Mindspark centers on math and language test scores. The study considers students from public middle schools in Delhi, who are tested before and after the 4.5-month-long intervention. We focus on the outcome variable that equals the endline math test score, which is standardized to have a mean of zero and standard deviation of one in the baseline test before the intervention. The treatment variable is the number of days a student logged into the Mindspark system. The binary instrument equals one for students who were randomly selected for the Mindspark voucher offer. The controls include 18 randomization strata fixed effects and the baseline math test score. The sample size is 535 students, and we report Huber-White standard errors.
The TSLS-L estimator linearly includes the controls, and due to the continuous baseline math test score this does not coincide with a saturated specification. The baseline score consists of 528 unique values, which makes saturation in a sample of size 535 observations infeasible. However, instead of discretizing the continuous control to its most granular level, we can include a set of dummies representing equally sized intervals of the sorted values in the control. Setting the number of dummies equal to one is identical to including an intercept, and the continuous control is not taken into account, while ten mutually exclusive dummies approximate the variation in the control.
Figure (ref) shows the TSLS-L estimate (black line) and the corresponding 95% confidence interval (gray shaded area). These estimates (Estimated LATE of 0.0067 with a standard error equal to 0.0011) are identical to the estimate in muralidharan2019disrupting. The red solid line represents the TSLS-S estimates for an increasing number of dummies representing the continuous control, and the dashed red lines represent the lower and upper bounds of the corresponding confidence intervals. Similarly, the solid and dashed blue lines represent the SIVE estimates and corresponding confidence intervals. For any level of granularity of the continuous control, the TSLS-S and SIVE estimate lie in the TSLS-L confidence interval. Saturation only slightly inflates the confidence intervals, even with ten dummies accounting for the continuous control.
We conclude that saturation with a discretized continuous control does not necessarily result in a substantial loss of statistical power. In case the continuous control affects the validity of the instrument, instead of governing the precision of the LATE estimate in the application here, discretizing the continuous control may effect the validity of the instrument. However, the conditional validity of an instrument almost always relies on categorical controls, representing different groups or strata.