Endogenous Treatment Effect Estimation with some Invalid and Irrelevant Instruments

Endogenous Treatment Effect Estimation with some Invalid and Irrelevant Instruments

Keywords: Endogenous treatment effect, instrumental variable selection, high-dimensionality, economic growth and trade. \doublespacing

Introduction

Instrumental variables (IV) method is useful for the identification of the endogenous treatment effects in empirical studies. The internal validity of IV analysis requires that instruments are associated with the treatment (A1: Relevance Condition), have no direct pathway to the outcome (A2: Exclusion Condition), and are not related to unmeasured variables that affect the treatment and the outcome (A3: Exogenous Condition). Finding the instruments that satisfy assumptions (A1)-(A3) is often quite a challenging task for empirical researchers. The inclusion of many redundant instruments (violation of A1) provides poor finite sample properties of estimators. Two-stage least squares (2SLS) tend to have large biases when many weak instruments are used Bekker1994Alternative. An instrument is deemed invalid, if it has a direct effect on the outcome (violation of A2), or an indirect association with the outcome through unobserved confounders (violation of A3). Using an invalid instrument will lead to inconsistency of the 2SLS and LIML estimators Kolesar2015Identification. It thus motivated this study to develop a method that could select relevant and valid instruments from a large set of candidate instruments.

There is a strand of literature on instruments selection under the exclusion condition and exogenous condition. In a seminal work on IV selection, Donald2001Choosing consider a procedure that minimizes higher-order asymptotic MSE which relies on a priori knowledge of the order of instruments. Kuersteiner2010 use model averaging to construct optimal instruments. Okui2011 considers ridge regression for estimating the first-stage regression. \ In high-dimensional setting, Gautier2011 propose an estimation method related to the Dantzig selector and the square-root Lasso when the structural equation in an instrumental variables model is itself very high-dimensional. Belloni2012sparse introduce the Post Lasso method which extends IV estimation to high dimension with heteroscedastic and non-Gaussian random disturbances. Caner2015Hybrid use the adaptive Lasso estimator to eliminate the irrelevant instruments. Lin2015Regularization propose a two-stage regularization framework for identifying and estimating important covariates' effects while selecting and estimating optimal instruments where the dimensions of covariates and instruments can both be much larger than the sample size. Fan2018Nonparametric study a nonparametric approach regarding a general nonlinear reduced form equation to achieve a better approximation of the optimal instruments with the adaptive group Lasso.

In the invalid IV settings, liao2013 proposes an adaptive GMM shrinkage estimator to select valid moment consistently. Also in GMM framework, Xu2015Select use an information-based adaptive GMM shrinkage estimation to select both the relevant and valid moments consistently and Caner2018invalid develop the adaptive Elastic-Net generalized method of moments (GMM) estimator in large dimensional models with potentially (locally) invalid moment conditions. The aforementioned papers need a prior knowledge about a subset of instruments that is known to be valid and contains sufficient information for identification and estimation of the causal effects. In contrast, Kang2015Instrumental propose a Lasso type procedure to identify and select the set of invalid instruments without any prior knowledge about which instruments are potentially valid or invalid. Windmeijer2018 suggest a consistent median estimator using adaptive Lasso which obtains the oracle properties. Guo2018 propose a Two-Stage Hard Thresholding (TSHT) with voting procedure that selects relevant and valid instruments and produces valid confidence intervals for the causal effect, which also works in the high dimension setting.

In this paper, we develop a method that extends the scope of sisVIVE Kang2015Instrumental by considering a true high-dimensional instrumental variable setting. Without knowing which instrument is irrelevant or invalid, we consider a large set of candidate instruments $\mathcal{IV}=\mathcal{IV}_{1}\cup\mathcal{IV}_{2}\cup\mathcal{IV}_{3}\cup\mathcal{IV}_{4}$, where $\mathcal{IV}_{1}$ denotes the set of relevant and valid instruments, $\mathcal{IV}_{2}$ denotes the set of relevant and invalid instruments, $\mathcal{IV}_{3}$ denotes the set of irrelevant and valid instruments and $\mathcal{IV}_{4}$ denotes the set of irrelevant and invalid instruments. We propose a three step procedure to achieve the estimation of endogenous treatment effects. Firstly, we consider a nonparametric additive reduced form model and estimate it by adaptive group Lasso, which select relevant instruments (denoted by $\mathcal{A}_{R}^{*}$) consistently and estimate the optimal instruments $\mathbf{D}^{*}$. This data-driven approach can usually adopt the linearity form automatically for a truly linear reduced form model. In the second step, we replace the original treatment variable by its estimated value and select the valid instruments (denoted by $\mathcal{A}_{V}^{*}$) consistently by adaptive Elastic-Net. In the final step, we take the selected invalid instruments (denoted by $\mathcal{A}_{I}^{*}=(\mathcal{A}_{V}^{*})^{c}$) as covariates and run a least squares regression to obtain the treatment effect estimator. It is shown that our procedure can select relevant and invalid instruments consistently\footnote{In the common sense of IV selection, we need to select the strong and valid IV. In the perspective of variable selection using Lasso type procedures, as we will demonstrate in our model part, the invalid instruments have non-zero coefficients in the structural equation and they should be selected. When we say we can select the invalid IV, it is not to be confused with the ultimate goal of selecting the $\mathcal{IV}_{1}$.}. The estimator has the desired theoretical properties such as consistency and asymptotic normality. Therefore, it is called {\bf{R}}obust {\bf{IV}} {\bf{E}}stimator to both the {\bf{I}}nvalid and {\bf{I}}rrelevant instruments (R2IVE), where the number 2 in R2IVE denotes both types of bad instruments. Monte Carlo simulations demonstrate that our estimator performs better than the existing IV estimators (such as 2SLS, NAIVE Fan2018Nonparametric and sisVIVE Kang2015Instrumental) for the endogenous treatment effects. In the empirical study, we revisit the classic question of trade and growth. It shows that our R2IVE estimators can select the invalid and relevant instruments and provide a more robust result than Frankel1999Trade and NAIVE Fan2018Nonparametric when there exists potentially invalid instruments.

The paper is organized as follows. Section 2 introduces the model setting, identification and estimation. Section 3 presents theoretical results. Section 4 collects simulation results to evaluate the finite sample performance of the proposed estimator. In section 5, we illustrate the usefulness of our estimator by revisiting the trade and growth study. Section 6 concludes. Technical proofs are given in the Appendix.

The following notations are used throughout the paper. For any $n$ by $L$ matrix $\mathbf{X}$, we denote the $(i,j)$-th element of matrix $\mathbf{X}$ as $X_{ij}$, the $i$th row as $\mathbf{X}_{i.}$, and the $j$th column as $\mathbf{X}_{.j}$. $\mathbf{X}^{T}$ is the transpose of $\mathbf{X}$. $\mathbf{\mathcal{M}}_\mathbf{X} = \mathbf{I}_n-\mathbf{\mathcal{P}}_\mathbf{X}$, where $\mathbf{\mathcal{P}}_\mathbf{X} = \mathbf{X}(\mathbf{X}^{T}\mathbf{X})^{-1}\mathbf{X}^{T}$ is the projection matrix onto the column space of $\mathbf{X}$, and $\mathbf{I}_n$ is the $n$-dimensional identity matrix. Let $\boldsymbol{\iota}_{s}$ denote a $1\times s$ vector of ones. The $l_p$-norm is denoted by $\left\|\cdot\right\|_p$, and the $l_0$-norm, $\left\|\cdot\right\|_0$, denotes the number of non-zero components of a vector. $\left\|\cdot\right\|$ is $l_2$-norm following conventions. For any set $A$, we denote $A^{c}$ to be its complement and $|A|$ is the cardinality of set $A$.

Model

Model Setting

For $i=1,\ldots, n$, let $Y_i^{(0,\mathbf{0})}\in \mathbb{R}$ be the potential outcome without any treatment and instruments, $Y_{i}^{(d,\mathbf{z})} \in \mathbb{R}$ be the potential outcome if the individual $i$ were to have treatment $d$ and instruments values $\mathbf{z}$. $D_i^{(\mathbf{z})}\in \mathbb{R}$ is the endogenous treatment if the individual $i$ were to have instruments $\mathbf{z}$, $d,d^{\prime} \in \mathbb{R}$ are two possible values of the treatment, and $\mathbf{z},\mathbf{z}^{\prime} \in \mathbb{R}^{L}$ are two possible values of the instruments. We consider a true high-dimensional instrumental variable setting such that the dimensionality $L$ is allowed to be potentially larger than the sample size $n$. Without loss of generality, we consider the univariate endogenous treatment variable. Suppose we have the following potential outcomes model,

where $\beta^{*} \in \mathbb{R}$ is the treatment parameter of interest, $\boldsymbol{\alpha}_1^{*} \in \mathbb{R}^{L}$ represents the direct effect of $\mathbf{z}$ on $Y$, and $\boldsymbol{\alpha}_2^{*}\in \mathbb{R}^{L}$ represents the presence of unmeasured confounders that affect both the instruments and the outcomes. An instrument $Z_j$ therefore does not satisfy the exclusion condition if $\alpha_{1j} \neq 0$ and does not satisfy the exogeneity condition if $\alpha_{2j}\neq 0$.

For each individual, only one possible realization of $Y_i^{(d,\mathbf{z})}$ and $D_i^{(\mathbf{z})}$ are observed, which are denoted as $Y_i$ and $D_i$, respectively, with the observed instrument values $\mathbf{Z}_{i.}$. We study the endogenous treatment effect through a random sample $\left\{Y_i; D_i; \mathbf{Z}_{i.}^{T}\right\}_{i=1}^{n}$. Let $\boldsymbol{\alpha}^{*}=\boldsymbol{\alpha}_1^{*}+\boldsymbol{\alpha}_2^{*}$, and $\varepsilon_{i}=Y_{i}^{(0,\bf{0})}-E\left[Y_{i}^{(0,\bf{0})} | \mathbf{Z}_{i .}\right]$. Combining ((ref)) and ((ref)), the baseline model is given by

where $D_i$ is an endogenous treatment variable, that is, $E(\varepsilon_{i}|D_i)\neq 0$\footnote{Our study focuses on a known endogenous treatment effect model. Guo2018b study the testing endogeneity problem and propose a new test that has better power than the Durbin-Wu-Hausman (DWH) test in high dimensions.}. Note that the model can include exogenous variables $\mathbf{X}_{i} \in \mathbb{R}^{p}$ and if so we can replace the variables $Y_i$, $D_i$ and $\mathbf{Z}_{i.}$ with the residuals after regressing them on $\mathbf{X}$ (e.g., replace $\mathbf{Y}$ by $\mathcal{M}_{\mathbf{X}}\mathbf{Y}$) Zivit1998Inference. For simplicity, we assume that $\mathbf{Y}$, $\mathbf{D}$, and the columns of $\mathbf{Z}$ are centered, which can be obtained from a residual transformation with $\mathbf{X}$ containing only the constant term.

Definition 1: Instrument $j\in \left\{1,\ldots,L\right\}$ is valid if $\alpha_j^{*} =0$, which means the instrument $j$ satisfies both condition (A2) and (A3), and it is invalid if $\alpha_j^{*} \neq 0$. Let $\mathcal{A}_{V}^{*}$ denote the set of valid instruments and $\mathcal{A}_{I}^{*}=(\mathcal{A}_{V}^{*})^{c}$ denote the set of invalid instruments.

We also extend the sisVIVE Kang2015Instrumental by considering a nonparametric additive reduced form model with a large number of possible instruments.

where $f_{j}(\cdot)$ is the $j$th unknown smooth univariate function and $\xi_{i}$'s are i.i.d random errors with mean 0 and finite variance. For the model identification, we assume that all functions $f_j(\cdot)$'s are centered, that is, $E[f_j(Z_j)] = 0$, $1 \leq j \leq p$, where $Z_j$ denotes the $j$th instrument. As it is more flexible and generally applicable than the ordinary linear model, the nonparametric additive reduced form model (ref) could achieve a better approximation to the optimal instruments $D_{i}^{*}=E(D_i|\mathbf{Z}_i.)$ AmemiyaThe,NeweyEfficient. The resulting estimator based on (ref) is expected to be more efficient compared to the linear IV estimator, which will be confirmed both theoretically and numerically in the later sections.

Definition 2: Instrument $j\in \left\{1,\ldots,L\right\}$ is a non-redundant IV that satisfies (A1), if $f_j(Z_j)\neq 0$. Let $\mathcal{A}_{R}^{*}$ denote the set of these instruments that are able to approximate the conditional expectation of the endogenous variable.

Identification and Estimation

In this section, we illustrate the identification and estimation of models (ref) and (ref). Without knowing which instrument is irrelevant or invalid, we consider a large set of candidate instruments, $\mathcal{IV}=\mathcal{IV}_{1}\cup\mathcal{IV}_{2}\cup\mathcal{IV}_{3}\cup\mathcal{IV}_{4}$ as introduced in Section (ref). Here, $\mathcal{IV}_{1}$ is the set of ideal instruments satisfying the conditions (A1)-(A3). $\mathcal{IV}_{2}$ could contribute to the construction of optimal instruments and the correct specification of model (ref). $\mathcal{IV}_{3}$ should be excluded in the models, otherwise, the reduced form equation (ref) will provide poor finite sample properties of estimators and the structural model (ref) will be overfitted. $\mathcal{IV}_{4}$ is the set of instruments that all conditions (A1)-(A3) are not satisfied. These instruments should be excluded from the model (ref) but included in the model (ref). The model (ref) will not be correctly specified if we delete these instruments mistakenly, that is, we take invalid instruments as valid. Our goal is therefore to select relevant instruments (denoted by $\mathcal{A}_{R}^{*}=\mathcal{IV}_{1}\cup\mathcal{IV}_{2}$) consistently for the model (ref) and invalid instruments (denoted by $\mathcal{A}_{I}^{*}=\mathcal{IV}_{2}\cup\mathcal{IV}_{4}$) consistently for the model (ref). Specifically, we first estimate the equation (ref) by adaptive group Lasso, and then substitute the estimated optimal instruments into (ref) to select the valid instruments using adaptive Elastic-Net.

Many researchers have studied additive nonparametric models Linton1970. The nonparametric study of reduced form equation is often troubled by the curse of dimensionality NeweyEfficient, which has been the focus of a substantial body of recent literature on high-dimensional problems. Huang2010 proposed a variable selection procedure in nonparametric additive models using the adaptive group Lasso based on a spline approximation to the nonparametric component. Fan2018Nonparametric extend it to IV estimation, which selects the strong instruments consistently and adopts the linearity form automatically for a truly linear reduced form model. Here, we estimate the optimal instruments and select the relevant instruments consistently following Fan2018Nonparametric.

Let $\mathcal{S}_n$ be the space of polynomial splines of degrees $h \geq 1$ and $\left\{\phi_{k},k=1,\ldots,m_n\right\}$ be the normalized B-spline basis functions for $\mathcal{S}_n$, where $m_n$ is the sum of the polynomial degree $h$ and the number of knots. Let $\psi_k(Z_{ij}) = \phi_k(Z_{ij}) - n^{-1} \sum_{i=1}^{n}\phi_k(Z_{ij})$ be the centered B spline basis function for the $j$th instrument. Thus, for each $f_{nj} \in \mathcal{S}_{n}$, it can be represented by the linear combination of normalized B-spline series

Under suitable smoothness conditions, the function $f_j(Z_{ij})$ in (ref) can be well approximated by the function $f_{nj}(Z_{ij})$ in $\mathcal{S}_n$ by carefully choosing the coefficients $\left\{\gamma_{j1},...,\gamma_{jm_n}\right\}$ Stone1985. Then the model ((ref)) can be rewritten as

Denote $\mathbf{D}=\left(D_1,...,D_n\right)^{T}$ as the $n\times1$ vector of endogenous treatment variable. Let $\boldsymbol{\gamma}_j= (\gamma_{j1},\gamma_{j2},\ldots,\gamma_{jm_n})^{T}$ be the $m_n \times 1$ vector of parameters corresponding to the $j$th instrument in ((ref)) and $\boldsymbol{\gamma}=(\boldsymbol{\gamma}_1^{T},\ldots,\boldsymbol{\gamma}_L^{T})^{T}$ be the $m_nL\times1$ vector of parameters. Let $\boldsymbol{U}_{ij}=(\psi _{1}(Z_{ij}),...,\psi _{m_{n}}(Z_{ij}))^{T}$ be the $m_n\times1$ vector, $\boldsymbol{U}_{j}=(\boldsymbol{U}_{1j},...,\boldsymbol{U}_{nj})^{T}$ be the $n\times m_n$ design matrix for the $j$th instrument and $ \boldsymbol{U}=(\boldsymbol{U}_{1},...,\boldsymbol{U}_{L})$ be the corresponding $n\times m_{n}L$ design matrix. To select the significant instruments and estimate the component functions simultaneously, we consider the following penalized objective function with an adaptive group Lasso penalty

where the weights is defined by

and $\widetilde{\boldsymbol{\gamma}}_{n}=(\widetilde{\boldsymbol{\gamma}}_{1}^{T},\ldots, \widetilde{\boldsymbol{\gamma}}_{L}^{T})^{T}$ is obtained from group Lasso

Denote the $\widehat{\mathcal{A}}_{R}=\left\{j:\left\|\widehat{\boldsymbol{\gamma}}_j\right\|^{2}>0\right\}$, and the adaptive group Lasso estimators of $f_j$ in (ref) are

Therefore the endogenous variable $D_i$ can be estimated by

Denote $\widehat{\mathbf{D}}=(\widehat{D}_{1},\cdots,\widehat{D}_{n})^{T}$, then $\widehat{\mathbf{D}}$ is the optimal instrument similar to Belloni2012sparse and Fan2018Nonparametric when $\left|\mathcal{A}_{R}^{*}\right|$ is greater than 1, which is shown in the following Theorem (ref). Note that when the nonparametric additive reduced form model (ref) is indeed a linear model, this data-adaptive approach can usually select $m_n = 1$ to degenerate to a linear model of (ref). The EBIC Chen2008Extended and BIC Hansheng2007Tuning could be used to choose the tuning parameters $\lambda_{0n}$, $\lambda_{n}$, and $m_n$ adaptively in practice for the high-dimensional and low-dimensional cases, respectively.

Next, we illustrate how to select the valid instruments and estimate the true endogenous treatment effect $\beta^{*}$. By taking the conditional expectation of both sides of (ref) given instrumental variables $\mathbf{Z}_i$, we have

where $D_i^{*}=E(D_i|\mathbf{Z_{i.}})$. Denote $\nu_i = Y_i - E(Y_{i}|\mathbf{Z}_{i.})$, it is straightforward to show that $E(\nu_{i})=E[E(\nu_{i}|\mathbf{Z}_{i.})]=0$ and cov$(D_i^{*}\nu_{i})=E[D_i^{*}E(\nu_{i}|\mathbf{Z}_{i.})]=0$. Adding $\nu_i$ to both side of (ref), we have

Thus, $D_i^{*}$ is an exogenous variable in (ref). It is worth noting that the coefficient of the optimal instrument $D_i^{*}$ in the model (ref) remains the same $\beta^{*}$ as in the structural equation (ref). If $D_i^{*}$ is known, the model (ref) can be easily estimated by linear estimation method. In practice, we replace $D_i^{*}$ by its estimate $\widehat{D}_i$ to obtain the final IV estimator for $\beta^{*}$. Substituting $\widehat{D}_{i}$ from (ref) into (ref), we have

Here, we need to have at least one relevant and valid instrument so that the equation (ref) is not troubled by the problem of potential collinearity\footnote{This is the assumption needed for the inference of the endogenous treatment effect in our model. There is a rich literature on the inference using many weak IVs or invalid IVs. In recent studies, Hansen2014 consider IV estimation with many weak IVs in high-dimensional models where the consistent model selection in the first stage may not be possible. Bietal2020 discuss inferring treatment effects after testing instrument strength in linear models. Berkowitz2012 shows how valid inferences can be made when an instrumental variable does not perfectly satisfy the orthogonality condition. }. We consider a two step procedure to estimate the equation (ref). Notice that unlike $\beta^{*}$, we are essentially concerned with which element in $\boldsymbol{\alpha}^{*}$ is not equal to 0 (invalid IV) but to a lesser extent, the true value of $\boldsymbol{\alpha}^{*}$.

In summary, we present the following Algorithm 1 for the estimator.

It is important to consider the model selection in the asymptotic theory of the estimator as discussed in chernozhukov2018.

Theoretical Properties

We assume the following regularity conditions for theoretical study.

Here we only give the assumptions used in the proof of Theorem (ref). The assumptions of Lemma (ref) and (ref) are put in the appendix. The condition (C1) imposes mild restriction on the finite second moment of the endogenous treatment variable $D_i$. Condition (C2) restricts the boundedness of true treatment effect and the number of relevant instruments and invalid instruments. The number of invalid instruments is needed to be less than the half of instruments, which is the identification condition in Kang2015Instrumental. Condition (C3) clarifies the conditions about error terms. The distribution of the random errors $\xi_i$'s should not be too heavy-tailed, and it is satisfied for $\xi_i$'s that are bounded uniformly or normally distributed.

This lemma shows the selection and estimation consistency of the adaptive group Lasso for high-dimensional nonparametric additive reduced form model, which essentially follows the results of Theorem 3 in Huang2010. We give the proof of equation (ref) in the appendix.

This lemma shows the selection consistency of the adaptive Elastic-Net for high-dimensional linear structure model. That is, the true set of invalid instruments can be identified with probability tending to 1. The proof of this lemma is in the appendix.

Simulation

We conduct various simulation studies to evaluate the estimation performance for different methods. We consider a structural model with one endogenous variable,

where $\beta^{*}=0.75$, $\mathbf{Z}_{i .}=(Z_{i1},Z_{i2},\ldots,Z_{iL})^{T}$ is generated from a multivariate normal distribution $N(0,\mathbf{\Sigma)}$, and $\mathbf{\Sigma}=\left(\rho_{j_{1} j_{2}}\right)_{L \times L}$ with $\rho_{j_{1} j_{2}}=0.5^{\left|j_{1}-j_{2}\right|}$, for $j_{1}, j_{2}=1, \ldots, L,$ and for each $i=1, \ldots, n$. We set $\boldsymbol{\alpha}=(\mathbf{0}_{q},\boldsymbol{\iota}_{s_2},\mathbf{0}_{L-q-s_2})^{T}$, where $\boldsymbol{\iota}_{s_2}$ is a $1 \times s_2$ vector of ones, which means the first $q$ and the last $L-q-s_2$ instruments are valid, and $s_2$ is the number of invalid instruments. The endogenous variable is generated based on either of the following two reduced form models,

For the model 1, we consider a “cut-off at $s_1$" design, that is, $\boldsymbol{\gamma}^{*}=(2, 0.75, 1.5, 1, \dots , \boldsymbol{0}_{L-s_1})^{T}$, where $\mathbf{0}_{L-s_1}$ is a $1\times (L-s_1)$ vector of zeros and $s_1$ is the number of strong instruments. We fill in the values of non-zero elements in $\boldsymbol{\gamma}^{*}$ by replicating the non-zero elements of $(2,0.75,1.5,1)$ to the until its length is $s_1$. E.g, if $s_1=6$, the non-zero elements of $\boldsymbol{\gamma}^{*}$ are $(2,0.75,1.5,1, 2,0.75)$. For the model 2, the main setting is that the number of non-zero nonlinear components is fixed at 4 as shown above. And to evaluate the performance when the number of strong IVs increase, we also replicate the same functional forms for $Z_{i5},\dots , Z_{i12}$ in the form of $Z_{i1},\dots ,Z_{i4}$ in one simulation setting. We generate the error terms in both the structural model and reduced form models by $$\left(

\right) \stackrel{\mathrm{i.i.d.}}{\sim} N\left(\left[

\right],\left[

\right]\right)$$

In the simulations, we vary (i) the sample size $n$, (ii) the number of strong instruments $s_1$, (iii) the number of invalid instruments $s_2$ and (iv) the distribution among the four instruments sets $\mathcal{IV}_{1}-\mathcal{IV}_{1}$, which is affected by the value of $q$. The model settings are summarized in Table (ref).

We run each simulation $R = 1000$ times and compute the average of the estimation bias (denoted by "Bias"), $R^{-1} \sum_{r=1}^{R}\left(\widehat{\beta}^{r}-\beta^{*}\right)$, with its empirical standard deviation and the estimated mean squared errors (denoted by "MSE"), $R^{-1} \sum_{r=1}^{R}\left(\widehat{\beta}^{r}-\beta^{*}\right)^{2}$, where $\widehat{\beta}^{r}$ denotes an estimator of ${\beta}^{*}$ in the $r$th experiment. We compare our method with OLS, 2SLS, Oracle 2SLS (this is the 2SLS using the oracle $\mathcal{IV}_1$ set), NAIVE Fan2018Nonparametric and sisVIVE Kang2015Instrumental. For 2SLS and NAIVE, we report the estimation results using only the endogenous treatment variable in the structural equation. We also present results for the so-called Post sisVIVE estimator (sisVIVE.post in the following tables), which is 2SLS estimator that takes the set of nonzero estimated coefficients $\boldsymbol{\alpha}$ in sisVIVE as covariates in the structural equation. We report the model selection performance for the estimators that embedding variable selection. Specifically, we report average number ("mean") of instruments selected as strong (for NAIVE) or invalid (for sisVIVE, sisVIVE.post, and R2IVE) together with the minimum, median and maximum numbers of invalid IV selection, and the proportion of the instruments selected as strong or invalid to all strong or invalid instruments ("freq"), respectively. Notice all the model settings in Table (ref) are the "Stronger Valid" cases when the valid instruments are stronger than the invalid instruments. The readers could refer to Kang2015Instrumental for more discussion on this setting.

All simulation studies are conducted using the statistical software R. In particular, the R package naivereg Fan2018Nonparametric is used to estimate the optimal instruments and the R package sisVIVE Kang2015Instrumental is used to get the sisVIVE estimator. We use the R package gcdnet Yang2012An to select invalid instruments with (adaptive) Elastic-Net penalty. The EBIC Chen2008Extended and BIC Hansheng2007Tuning methods are employed to find the optimal tuning parameters for the high-dimensional and low-dimensional cases, respectively.

Linear reduced form equation

In this setting, we consider the linear reduced form. We first fix the sample size $n$, IV dimension $L$, the number of relevant instruments $s_1$ and the value of $q$ but change the number of invalid instruments $s_2$ to check the influence of the number of invalid instruments. Then we fix $n$, $L$ and $s_2$ but change the value of $s_1$ and $q$ to check the influence of the strength of instruments. Finally, we fix $L$, $s_1$ and $s_2$ but change the sample size.

In the first case, we fix $n=200$ and $L=100$. The number of relevant instruments $s_1=10$. We use the values of $s_2=0, 10, 30$ to check the influence of the number of invalid instruments on estimation results. For $s_2=0$, we have $|\mathcal{IV}_{2}|=|\mathcal{IV}_{4}|=0$. Set $|\mathcal{IV}_{1}|=10$ and $|\mathcal{IV}_{3}|=90$. For other nonzero values of $s_2$, we set $q=7$, which means $|\mathcal{IV}_{1}|=7$, $|\mathcal{IV}_{2}|=3$, $|\mathcal{IV}_{3}|=100-s_2-7$ and $|\mathcal{IV}_{4}|=s_2-3$. The specific numbers of each type of instrument variables are summarized in Table (ref). Table (ref) reports the estimation bias with the standard deviations and MSE, as well as the model selection results. Figure (ref) shows the box plots of bias for these estimators. In Figure (ref), we do not include the results of OLS estimators since they are always biased and have the largest MSE, which will enlarge the scale of the figure and make the differences between other IV estimators less distinguishable. When there is no invalid instruments, the sisVIVE is outperformed by the NAIVE due to the effect of many irrelevant instruments. When the invalid instruments exist, the 2SLS and NAIVE estimators perform similarly and are both severely biased due to the effects of invalid instruments. The sisVIVE estimators have smaller bias and MSE compared to 2SLS and NAIVE estimators. However, they are still substantially biased although the invalid instruments are always selected as invalid by sisVIVE and the sisVIVE.post does not help reducing bias. Furthermore, sisVIVE always select too many invalid instruments, which reduce the efficiency of estimators. Our R2IVE performs best (among the non-Oracle estimators) with the smallest bias and MSE. It is very close to oracle 2SLS estimator in linear reduced form models and is shown to be robust to both irrelevant and invalid instruments.

In the second case, we fix $n=200$, $L=100$ and $s_2=30$ but change the value of $s_1=4, 10, 20$ to check the influence of the strength of instruments on different estimators. We set $q=2,7,14$ for different $s_1$, respectively. $|\mathcal{IV}_{1}|=2,7,14$, $|\mathcal{IV}_{2}|=2,3,6$, $|\mathcal{IV}_{3}|=68,80,56$ and $|\mathcal{IV}_{4}|=28,27,24$ for different $s_1$ respectively. The results are shown in the Table (ref) and Figure (ref). We observe that the sisVIVE estimators are dependent of the number of relevant instruments, and they have diminishing bias and MSE when the number of relevant instruments $s_1$ increases. When $s_1=4$, the sisVIVE is even outperformed by NAIVE that takes all instruments as valid. The sisVIVE.post does not reduce the bias of the sisVIVE. This shows the importance of selecting strong IVs. Our R2IVE performs best and can estimate the casual effect precisely. The R2IVE also improves as the number of relevant instruments increases.

In the last case, we fix $L=100$, $s_1=20$, $s_2=20$ and $q=14$ while changing the sample size. The results are shown in the Table (ref) and Figure (ref). The increase of the sample size does not improve the estimated performance of 2SLS and NAIVE, as they are always biased due to the endogeneity of IVs. The sisVIVE estimators have diminishing bias and MSE when the sample size increases. Our R2IVE estimator always performs best with the smallest bias and MSE. Compared to sisVIVE, our method has much better finite sample performance. It is very close to oracle 2SLS in linear models. Note that the sisVIVE.post can reduce some bias from sisVIVE when $n$ is 1000 but is still outperformed by our estimator.

Nonlinear reduced form equation

In this subsection, we consider the nonlinear reduced form. The results are summarized in Table (ref) and Figure (ref). The influence of the invalid instruments is checked by comparing the top left and bottom left plots of Figure (ref). Similar to the linear case, the 2SLS and NAIVE estimators become biased due to the invalid instruments. When there is no invalid instruments, the sisVIVE is outperformed by NAIVE since it does not consider the irrelevant instruments and the nonlinear reduced form equation. When the invalid instruments exist, our R2IVE always performs best. Different from the linear case, the NAIVE always outperforms the 2SLS by capturing the nonlinear structure. We check the influence of the strength of instruments by comparing the bottom left and bottom right plots of Figure (ref). Both sisVIVE and R2IVE are improved when the number of relevant instruments increase and our estimators always outperform sisVIVE and sisVIVE.post. We check the influence of sample size by comparing the top right with bottom left plot of Figure (ref). Both sisVIVE and R2IVE improve as the sample size increases in the "stronger invalid" settings.

Applications to Trade and Economic Growth

Background

In this section, we illustrate the usefulness of our estimator by revisiting the classic question of trade and growth. The effect of trade on growth is a very important research topic in both theoretical and empirical economics, which has strong effect on trade policies.

One important issue in the empirical study of trade and growth is the endogeneity of trade variable due to the unobserved common driving forces that cause both trade and growth. Frankel1999Trade circumvented the endogeneity problem utilizing instrumental variable constructed using gravity model of trade Anderson1979. They showed that trade activities positively correlate with growth rate using cross-sectional data from 150 countries and economies using data from the mid-1980s. In FR99, they consider a linear structural equation, which include the log of GDP per worker (outcome variable), the share of international trade to GDP (explanatory variable of interests) and two exogenous variables representing the size of a country: population and land area. They also consider a linear bilateral trade reduced form equation, where the instruments include the distance between two countries, dummy variables for landlocked countries, common border between two countries and the interaction terms, and two exogenous variable aforementioned. The instrumental variable (called proxy for trade in FR99) is the sum of predicted bilateral trade shares for country $i$. Fan2018Nonparametric extend the study of FR99 by considering more potential instruments and considering a nonlinear reduced form. Besides the instrument used in FR99, they also include total water area, coastline, the arable land as percentage of total land, land boundaries, forest area as percentage of land area, the number of official and other commonly used languages in a country, and the interaction terms of constructed trade proxy with these variables (in total 15 instruments). The selected instruments include the proxy for trade (the original instrument in FR99), area of land, total population, and the interaction term of proxy for trade and number of languages. The NAIVE method provides stronger results regarding trade on growth than \emph{FR99}. In this study, we are concerned about the invalid IVs, which means some instruments might affect growth directly. The inclusion of invalid instruments leads to the inconsistency of $\beta$.

Data and Model Setting

Following FR99 and NAIVE, we use the cross-sectional data from 158 countries and economies and update the data to year 2017 to investigate the contemporary effect of trade on growth. We consider a linear structural equation

where $Y_i$ is GDP per worker in country $i$, $D_i$ is the share of international trade to GDP, $\mathbf{S}_i$ is the size of country: population and Land area (same as FR99), $\mathbf{Z_i}$ is the instruments. Besides the instruments used in Fan2018Nonparametric, we also also include a instrumental variable related to air pollution: the density of PM 2.5. Kukla-GryzEconomic found that international trade and per capita income lead to the increase in air pollution in developing countries. In order to reduce the negative impact of international trade on the environment, the state will gradually adopt new policies with more environmental friendly standards hence raising the costs of production, which means air pollution could in turn affect international trade. On the other hand, there is empirical evidence for the environmental Kuznets curve between economic growth and environmental pollution. Ali2018Pollution conclude the impact of pollution abatement on economic growth could turn into win-win policy options. Hence, there is some reasons to believe that PM2.5 may impact trade, but also affect economic growth through mechanisms other than trade. $\varepsilon_{i}$ is unobserved random disturbances in the growth function.

The reduced form model we consider is

where $f_{j}(\cdot)$ is the $j$th unknown smooth univariate functions, $z_{ij}$ is the $i$th observed value of the aforementioned $j$th instrument and $\xi_{i}$ is unobserved random disturbances, which is likely to be correlated with $\varepsilon_{i}$.

Note that we can replace the variables $Y_i$, $D_i$, and $\mathbf{Z}_{i.}$ with the residuals after regressing them on $\mathbf{S}_i$ (e.g., replace $\mathbf{Y}$ by $\widetilde{\mathbf{Y}}=(\mathbf{I}-\mathcal{P}_{\mathbf{S}})\mathbf{Y}$). The equation (ref) and (ref) becomes

The summary statistics of main data is presented in Table (ref). Figure (ref) is the scatter diagram of actual and constructed share of international trade. Their correlation coefficient is 0.36.

Empirical Results

To investigate the influence of invalid instruments on the estimation of $\beta$, we first conduct the estimation using the same instruments as Fan2018Nonparametric and compare our estimated value with FR99 and NAIVE. Then we add another instrumental variable PM 2.5. The results are summarized in Table (ref) and (ref). The first column is OLS estimator. The second estimator is the 2SLS estimator using the same instruments with FR99. The third column is the NAIVE estimator and the last column is our estimator.

When the variable PM2.5 is not included, the selected relevant instruments include the proxy for trade and the interaction term of proxy for trade and forest area as percentage of land area by the adaptive group Lasso with EBIC. The fitted functions of selected instruments by EBIC are plotted in Figure (ref). From Figure (ref), we see that proxy for trade and the interaction term of proxy for trade and forest area as percentage of land area instruments are likely to have nonlinear relationship with real trade share. In Table (ref), the OLS estimator has severe bias and is inconsistent because of the endogeneity issue. The t statistics for the NAIVE on trade is 3.85, compared to 2.99 for the FR99. As expected the each elements of $\boldsymbol{\alpha}$ is estimated to zero and our estimator is same as NAIVE.

When the variable PM 2.5 is included, the estimation results is summarized in Table (ref). If we use NAIVE, under the operating assumption that all the instruments are valid, the estimated causal effect is 1.34 (with a standard error of 0.26). Our estimator can throw out the irrelevant instrument, the arable land as percentage of total land, and select the invalid instrument, PM2.5. Our estimator estimates the causal effect to be 1.19 which is close to the results in Table (ref). This shows the R2IVE is robust to the potential invalid and irrelevant IVs. At last, compared with the original study of FR99, the causal effect of trade on growth in 2010s is found to be smaller in magnitude but even more significant.

Conclusion

In this paper, we develop a robust IV estimator to both the invalid and irrelevant instruments (R2IVE) for the estimation of endogenous treatment effect, which extends the sisVIVE Kang2015Instrumental by considering a true high-dimensional instrumental variable setting and a general nonlinear reduced form equation. The proposed R2IVE is shown to be root-$n$ consistent and asymptotically normal. Monte Carlo simulations demonstrate that the R2IVE performs better than the existing contemporary IV estimators (such as NAIVE and sisVIVE) in many cases. The empirical study revisits the classic question of trade and growth. It is demonstrated that the R2IVE can be applied to estimate the endogenous treatment effect with a large set of instruments without knowing which ones are relevant or valid and the reduced form is linear or nonlinear.