A Heteroskedasticity-Robust Overidentifying Restriction Test with High-Dimensional Covariates

A Heteroskedasticity-Robust Overidentifying Restriction Test with High-Dimensional Covariates

\def\spacingset#1{ {#1}} \spacingset{1}

\if00 \fi

\if10 {

} \fi

{\it Keywords:} overidentification test, maximum test, heteroskedasticity, power enhancement, data-rich environment

\spacingset{1.5}

Introduction

Instrumental variable (IV) regression is popular for inference of endogenous effects, whose validity relies on the IV exclusion restrictions. With increasing access to large-scale data, the model with high-dimensional covariates or instruments has drawn considerable attention from the theoretical and empirical literature. This paper develops a test for IV exclusion restrictions in a high-dimensional model. More precisely, consider the following instrumental variable model, for $i \in\{1, \ldots, n\}$,

where $Y_i\in \mathbb{R}$ is the dependent variable, $D_i\in \mathbb{R}$ is an endogenous variable, $X_{i\cdot}\in \mathbb{R}^{p_x}$ is a vector of exogenous covariates, $Z_{i\cdot}\in \mathbb{R}^{p_z}$ is a vector of instruments and $e_i$, $\varepsilon_{i,D}$ are random errors that may be correlated. In this paper, we allow both $p_x$ and $p_z$ to be larger than $n$ and assume the vectors $\varphi$, $\pi$, $\psi$ and $\gamma$ are sparse, which is specified by Assumption (ref)(i) in Section (ref). The paper develops a test of the null hypothesis,

against the alternative $\mathbb{H}_a: \pi \neq 0$. The IVs are valid if $\pi=0$. The classic Sargan test sargan1958estimation and J test hansen1982large consist of two steps: (1) Compute a two-stage least square (TSLS) estimator of $\beta$, denoted as $\widehat\beta_{\rm TSLS}$; (2) Regress $Y-D\widehat\beta_{\rm TSLS}$ on the covariates $X$ and IVs $Z$, and test the joint significance of the coefficients of IVs. Our new test follows similar ideas. We first construct a debiased Lasso-based estimator of the parameter $\beta$, denoted as $\widehat\beta_A$ in (ref). The estimator is $\sqrt{n}$-consistent and asymptotically normal under the null hypothesis (ref). We further run the Lasso regression of $Y_i-D_i \widehat{\beta}_A$ on $X_{i\cdot}$ and $Z_{i\cdot}$, and store the debiased estimators of the coefficients of $Z_{i\cdot}$ as $\widetilde \pi_A$. Under the null hypothesis (ref), $\widetilde \pi_A$ is asymptotically equal to the sample average of mean-zero random vectors. The test rejects the null hypothesis if the maximum norm of a scaled version of $\widetilde\pi_A$ exceeds the critical value obtained from a high-dimensional central limit theorem by chernozhukov2013gaussian.

Main Results and Contributions

We first design a maximum test (M test) based on the maximum norm of the coefficient vector $\widetilde \pi_A$ that may be high-dimensional. In closely related literature, some recent overidentification tests consider a model with a large number of IVs lee2012hahn,chao2014testing,carrasco2021testing,kolesar2018minimum, and we refer to these tests based on a limiting $\chi^2$ distribution as “$\chi^2$-type tests”. None of the $\chi^2$-type tests above allow $p:=p_x+p_z>n$ and $p_x \to \infty$, while our proposed M test covers this scenario. Under some commonly imposed sparsity assumptions Belloni2012sparse,belloni2014, the M test has the correct asymptotic size. Moreover, when $p$ grows with the sample size and $p<n$, the M test has better power than $\chi^2$-type tests under the sparse regime. We further propose an add-on asymptotically zero quadratic statistic (Q statistic) to improve the power when the model includes many “locally invalid" IVs, where the individual violation of IV validity is weak; see Section (ref) for details. The resulting test, called the power-enhanced M test (PM test), rejects the null hypothesis when either the M or the Q statistic is greater than the critical value of the significance level $\alpha$. Our paper extends the principle of power enhancement developed by fan2015power and kock2019power to the popular IV model and overidentification test. The PM test always has non-inferior power compared to the original M test. In simulations (see Section (ref) of the supplement), we show that the power of the PM test is at least as good as the M test and substantially improved when many IVs are locally invalid.

In the empirical study, we revisit the effect of trade on economic growth. We perform overidentification tests on an IV model with a large number of covariates. The set of instruments includes several possibly invalid instruments, such as energy usage and business environment. The PM test strongly rejects the null hypothesis under the 1% level. In contrast, the M test rejects the null hypothesis only under 5%. The modified Cragg-Donald (MCD) test by kolesar2018minimum, a representative $ \chi^2$-type test feasible for $p_x\to\infty$ with $p<n$, fails to reject at the 5% level, indicating the potential power gains of the PM test under high-dimensional IV models. We summarize the main contributions as follows:

Other Related Literature

Our test relates to the maximum test in a linear regression model chernozhukov2013gaussian, zhang2017simultaneous. The asymptotically zero Q statistic follows an inferential procedure for a quadratic form of high-dimensional parameters guo2019optimal,tony2020semisupervised,guo2021group. In terms of high-dimensional IV regression, Belloni2012sparse, belloni2014 and chernozhukov2015post proposed post-selection inference for the endogenous treatment effects. The post-selection method requires covariate and IV selection consistency. Nevertheless, IV selection often suffers from errors in finite samples guo2023causal. The treatment effect estimator used in our overidentifying restriction test adopts bias-corrected estimators of quadratic forms that are free from variable selection bias. Recently, belloni2022high and gold2020inference developed bias-corrected estimators for high-dimensional IV models. These estimators are asymptotically normal when all IVs are valid ($\pi=0$), and thus an overidentifying restriction test with a correct asymptotic size using these estimators is possible. Nevertheless, in the presence of high-dimensional covariates, it is unclear how to derive the limiting distributions of the abovementioned estimators under the alternative $\pi\neq 0$, which brings challenges to deducing the power. In contrast, we identify $\beta$ under the null $\pi=0$ based on quadratic forms of reduced-form coefficients, paving a more transparent way to power analysis. Another strand of literature has studied the estimation and inference of endogenous treatment effects with potentially invalid instruments kang2016instrumental,guo2018confidence,windmeijer2019use,fan2022endogenous,gautier22. In order to identify the treatment effect $\beta$, these methods required model identification conditions, e.g., the majority rule, which means more than half of the IVs are valid kang2016instrumental. On the other hand, our test does not require these identification conditions, such as the majority rule, since our primary goal is to test the IV validity. Therefore, our method is complementary to the above studies.

Other literature liao2013adaptive,cheng2015select,caner2018adaptive,chang2021culling has studied moment condition selection under the GMM framework, requiring prior knowledge of some valid moment conditions. Chang2021 considered the overidentification test in high-dimensional settings using marginal empirical likelihood ratios and a selective subset of moment conditions. mikusheva2022inference studied robust inference with many weak IVs.

Notations. We consider $p=p(n)$ as a function of $n$ and discuss the asymptotics where $n$ and $p$ jointly diverge to infinity. The phrase “with probability approaching one as $n\to\infty$" is abbreviated as “w.p.a.1". An absolute constant is a positive, finite constant that is invariant with the sample size. We use “$\stackrel{p}{\longrightarrow}$" and “$\stackrel{d}{\longrightarrow}$" to denote convergence in probability and distribution, respectively. For any positive sequences $a_n$ and $b_n$, “$a_n\lesssim b_n$” means there exists some absolute constant $C$ such that $a_n\leq Cb_n$, “$a_n\gtrsim b_n$” means $b_n\lesssim a_n$, and “$a_n\asymp b_n$” means $a_n\lesssim b_n$ and $b_n\lesssim a_n$. Correspondingly, “$\lesssim_p$”, “$\gtrsim_p$” and “$\asymp_p$” indicate that the aforementioned relations “$\lesssim$”, “$\gtrsim$” and “$\asymp$” hold w.p.a.1. “$a _n\gg b_n$” means $a_n/b_n\to\infty$ as $n\to\infty$. We use $[n]$ for some positive integer $n$ to denote the integer set $\{1,2,\cdots,n\}$. For a $p$-dimensional vector $x=(x_{1,}x_{2},\cdots,x_{p})^{\top}$, the number of nonzero entries is $\|x\|_0$, the $L_{2}$ norm is $\left\Vert x\right\Vert _{2}=\sqrt{\sum_{j=1}^{p}x_{j}^{2}}$, the $L_{1}$ norm is $\left\Vert x\right\Vert_{1}=\sum_{j=1}^{n}\left|x_{j}\right|$, and the maximum norm is $\|x\|_{\infty}=\max_{j\in[p]}|x_{j}|$. For a $p\times p$ matrix $A=(A_{ij})_{i,j\in[p]}$, we define the $L_1$ norm $\|A\|_1 = \max_{j\in[p]}\sum_{i\in[p]}|A_{ij}|$ and the maximum norm $\|A\|_\infty = \max_{i,j\in[p]}|A_{i,j}|$. “$A\succ 0$” means the matrix $A$ is positive definite. For any $p\times p$ matrix $A\succ 0$ with spectral decomposition $U\Lambda U^\top$, we define $\lambda_{\min}(A)$ and $\lambda_{\max}(A)$ as the minimum and maximum eigenvalues of $A$, and $A^{1/2}=U\Lambda^{1/2} U^\top$ with $\Lambda^{1/2}$ being the diagonal matrix composed of the square roots of the corresponding diagonal elements of $\Lambda$. We use ${\rm diag}(A)$ to denote the diagonal matrix composed of the diagonal elements of $A$. We define ${\rm I}_{A}(x,y)=x^\top Ay$ and ${\rm Q}_{A}(x)={\rm I}_{A}(x,x)$ for any vectors $x,y\in\mathbb{R}^{p}$. We use $0_p$ to denote the $p\times 1$ null vector, $1_p$ to denote the $p\times 1$ vector of ones, and $I_p$ to denote the $p$-dimensional identity matrix. The indicator function is ${\textbf 1}(\cdot)$. Finally, for any $a,b\in\mathbb{R}$, we use $a\vee b$ and $a\wedge b$ to denote $\max(a,b)$ and $\min(a,b)$, respectively. The remainder of the paper is organized as follows. In Section (ref), we introduce the model and a treatment effect estimator. Section (ref) discusses the M test and its power-enhanced version with their asymptotic properties. We present an empirical example in Section (ref). Section (ref) concludes the paper. Technical proofs, additional empirical study details, and Monte Carlo simulations are given in the supplement.

The Model and Treatment Effect Estimation

The test operates via a random sample $\{Y_i, D_i, X_{i\cdot}, Z_{i\cdot}\}_{1\leq i\leq n}$ in model (ref). Heteroskedastic errors are allowed so that ${{\rm Var}}(e_i|Z_{i\cdot},X_{i\cdot})$ and ${{\rm Var}}(\varepsilon_{i,D}|Z_{i\cdot},X_{i\cdot})$ could vary with $i$. To fix ideas, we assume high-dimensional covariates with $p_x\to\infty$ so that $p=p_x+p_z\to\infty$ with $p_z$ either fixed or growing. Our test, therefore, accommodates the studies with either high-dimensional or a fixed number of instruments.

In Section (ref), we present the treatment effect identification using equation (ref). This identification motivates the data-dependent treatment effect estimator in the following Section (ref). We further establish the asymptotic normality of this estimator in Section (ref).

Identification and Scale Invariance

Denote $Y=(Y_1,Y_2,\cdots,Y_n)^\top$, $D=(D_1,D_2,\cdots,D_n)^\top$, $X=(X_{1\cdot},X_{2\cdot},\cdots,X_{n\cdot})^\top$ and $Z=(Z_{1\cdot},Z_{2\cdot},\cdots,Z_{n\cdot})^\top$. The reduced form of model ((ref)) is

where $\Psi=\psi\beta+\varphi$, $\Gamma=\gamma\beta+\pi$ and $\varepsilon_Y=\varepsilon_D\beta+e=(\varepsilon_{1,Y},\varepsilon_{2,Y},\cdots,\varepsilon_{n,Y})^\top$ with $e=(e_1,e_2,\cdots,e_n)^\top$ and $\varepsilon_D=(\varepsilon_{1,D},\varepsilon_{2,D},\cdots,\varepsilon_{n,D})^\top$. We write $W_{i\cdot}=(X_{i\cdot}^\top,Z_{i\cdot}^\top)^\top$ for $i=1,2,\cdots,n$ and $W=(W_{1\cdot},W_{2\cdot},\cdots,W_{n\cdot})^\top$. Define the population Gram matrix $\Sigma := \mathbb{E}(W_{i\cdot}W_{i\cdot}^\top)$, and the precision matrix $\Omega := \Sigma^{-1}$. Furthermore, define $\sigma_{i,Y}^2:={\rm Var}(\varepsilon_{i,Y}|W_{i\cdot})$, $\sigma_{i,D}^2:={\rm Var}(\varepsilon_{i,D}|W_{i\cdot})$ and $\sigma_{i,YD}:=\text{cov}(\varepsilon_{i,Y},\varepsilon_{i,D}|W_{i\cdot})$. Let $\widehat{\Sigma} := n^{-1}\sum_{i=1}^n W_{i\cdot}W_{i\cdot}^\top = n^{-1}W^\top W$.

In the literature chao2014testing, it is common to use weighted norms of the unknown parameters for the construction of estimators and tests. From model (ref), we have $\Gamma = \gamma \beta+\pi$. Thus, for any $p$-dimensional square matrix $A$ such that ${\rm Q}_{A}(\gamma)=\gamma^\top A\gamma > 0$, we have

where ${\rm I}_{A}(\gamma,\Gamma)=\gamma^{\top} A\Gamma$ and ${\rm I}_{A}(\gamma,\pi)=\gamma^{\top} A\pi.$ Since $\Gamma = \gamma \beta+\pi$, ((ref)) holds when $A$ is either random or fixed. In order to achieve the scale-invariant property, we choose

where $\widehat\sigma_{jz}^2 := n^{-1}\sum_{i=1}^n Z_{ij}^2$, for $j=1,2,\dots, p_z$. When the $j$-th IV $Z_{ij}$ is scaled with some number $m>0$, the corresponding coefficient $\gamma_j$ is multiplied by $1/m$ since $Z_{ij}\gamma_j = (mZ_{ij})(\gamma_j/m)$; similar arguments apply to $\Gamma_j$ and $\pi_j$. Thus, with the weighting matrix $A$ in (ref), the quadratic forms and inner products in (ref) remain unchanged if we scale the instruments by some number $m>0$. It is easy to show that ${\rm Q}_{A}(\gamma)>0$ w.p.a.1 under the assumptions in Section (ref), and thus we assume ${\rm Q}_{A}(\gamma)>0$ throughout the theoretical discussions. We define the parameter

In Section (ref), we apply (ref) to derive a data-dependent estimator $\widehat{\beta}_A$ of $\beta_{A}$\footnote{We slightly abuse the terminology to say $\widehat{\beta}_A$ is an estimator of $\beta_A$ even when the matrix $A$ is random. The same applies to the notation $\widehat{\pi}_{A}$ in Section (ref).}. Since $\beta_{A}-\beta={{\rm I}_{A}(\gamma,\pi)}/{{\rm Q}_{A}(\gamma)}$, we have $\beta_{A}=\beta$ under the null hypothesis of $\pi=0,$

A Debiased Lasso-Based Estimator of $\beta$

We now introduce an estimator of $\beta_A$ defined in (ref), where $\beta_A = \beta$ when all IVs are valid ($\pi=0$). This estimator is useful to construct the test statistic in Section (ref). With the estimators $\widehat{\rm Q}_{A}(\gamma)$ and $\widehat{\rm I}_{A}(\gamma,\Gamma)$ specified later in (ref) and (ref), $\beta_A$ can be estimated by

In the following, we provide details for estimating $\widehat{\rm I}_{A}(\gamma,\Gamma)$ and $\widehat{\rm Q}_{A}(\gamma).$ We use Lasso tibshirani1996regression to get the initial estimates of $\Gamma$ and $\gamma$ in (ref):

where $\lambda_{1n},\lambda_{2n}$ are positive tuning parameters that are selected by cross-validation in practice. The plug-in estimator of $\beta_A$ given by ${\rm I}_{A}(\widehat\gamma,\widehat\Gamma)/{\rm Q}_{A}(\widehat\gamma)$ suffers from regularization bias and invalidates asymptotic normality. Therefore, we introduce a debiasing procedure for $\widehat \beta_{A}$ through constructing debiasing estimators of ${\rm I}_{A}(\gamma,\Gamma)$ and ${\rm Q}_{A}(\gamma)$. Here, we generalize the debiasing method for the quadratic form of high-dimensional parameters presented in recent literature guo2019optimal,guo2021group to heteroskedastic errors. We specify our bias correction procedure in the following. First, for ${\rm Q}_{A}(\gamma)$, the denominator of $\beta_A$, the estimation error of the plug-in estimator ${\rm Q}_{A}(\widehat{\gamma})$ is

The second term on the right-hand side (RHS) of (ref) is asymptotically negligible. The bias of the plug-in estimator ${\rm Q}_{A}(\widehat\gamma)$ is mainly induced by the first term on the RHS of (ref), specifically, the regularization bias in the initial LASSO estimator, $\widehat\gamma-\gamma$. Thus, we need a bias-corrected estimator of $\gamma$ for an asymptotically normal estimator of ${\rm Q}_{A}(\gamma)$. Following the idea of javanmard2014confidence, a bias-corrected estimator of $\gamma$ is given as

where $\widehat\Omega$ is the constrained $L_1$-minimization for inverse matrix estimation (CLIME, cai2011constrained) of the precision matrix $\Omega$. Specifically, let $\widehat\Omega^{(1)}$ be the solution of the problem

where $I_p$ is the $p$-dimensional identity matrix and $\mu_{\omega}$ is a positive tuning parameter. The CLIME estimator is defined as

The above definition (ref) guarantees that $\widehat{\Omega}$ is a symmetric matrix, even if $\widehat{\Omega}^{(1)}$ is not necessarily symmetric. Particularly, for two different values in $\{\widehat{\Omega}_{jk}^{(1)}, \widehat{\Omega}_{kj}^{(1)}\}$, we choose the one with a smaller absolute value, and assign $\widehat{\Omega}_{jk}$ as this particular value. This value assignment results in $\widehat\Omega_{jk}=\widehat\Omega_{kj}$ and thus $\widehat{\Omega}$ is symmetric. We use the fastclime R package pang2014fastclime for efficient computation of CLIME. The difference between (ref) and the analog in javanmard2014confidence is that we minimize the $L_1$-norm, instead of $L_2$-norm. The $L_1$ minization is also used in gold2020inference. Lemma (ref) in the supplement establishes convergence rates of the CLIME estimator in ((ref)).

A bias-corrected estimator of ${\rm Q}_{A}(\gamma)$ is then given as

where $\widehat\gamma$ and $\widetilde\gamma$ are respectively defined in (ref) and (ref). The estimation error of the debiased estimator $\widehat{\rm Q}_{A}(\gamma)$ is decomposed as

The first term on the RHS of (ref) is asymptotically normal since $\widetilde\gamma$ is debiased, and the second term is asymptotically negligible. Thus, we can deduce the asymptotic normality of the estimator $\widehat{\rm Q}_{A}(\gamma)$.

Similarly, the estimation error of the plug-in estimator ${\rm I}_{A}(\widehat\gamma,\widehat\Gamma)$ is decomposed as

With a similar motivation as (ref), we propose the following debiased estimator of ${\rm I}_{A}(\gamma,\Gamma)$,

where $\widetilde\Gamma$ is the debiased estimator of $\Gamma$ defined as

with $\widehat{\Omega}$ defined in (ref). We can then establish a bias-corrected estimator $\widehat\beta_A$ as (ref) using the estimators in (ref) and (ref).

Asymptotic Property of $\widehat{\beta}_A$

Under Assumptions (ref)-(ref) below, we can show that $\widehat{\rm Q}_{A}(\gamma)>0$ w.p.a.1, and thus the estimation error of $\widehat{\beta}_A$ in (ref) is decomposed as

We establish the asymptotic normality of $\sqrt{n}(\widehat{\beta}_A-\beta_{A})$ based on the decomposition (ref) and the asymptotic normality of $\sqrt{n}(\widehat{\rm I}_{A}(\gamma,\Gamma)-{\rm I}_{A}(\gamma,\Gamma))$ and $\sqrt{n}(\widehat{\rm Q}_{A}(\gamma)-{\rm Q}_{A}(\gamma))$. To state the theoretical results, we first recall the definition of sub-Gaussian norm vershynin2010introduction.

We impose the following assumptions to derive the asymptotic properties of $\widehat{\beta}_A$.

Assumption (ref) is a sub-Gaussianity condition for the covariates and IVs, with eigenvalue bounds for the population Gram matrix. Assumption (ref) imposes sub-Gaussianity and bounded conditional moment conditions on the error terms. We rule out the perfect correlation between error terms by bounding the correlation coefficient away from one.

Assumption (ref) assumes an approximately sparse precision matrix, which is required to establish the rate convergence of the CLIME estimator ((ref)). Such a sparse precision matrix assumption is widely used for inferential procedures in high-dimensional models van2014asymptotically,gold2020inference.

We now specify the sparsity assumption on model (ref) and its reduced form (ref). Define the sparsity index $s=\max\{\|\varphi\|_0+\|\pi\|_0,\|\psi\|_0+\|\gamma\|_0, \|\Psi\|_0+\|\Gamma\|_0\}$, and the probability limit of the weighting matrix $A$ as

where $\sigma_{jz}^2 := \mathbb{E}(Z_{ij}^2)$ for $j=1,2,\dots , p_z$.

Assumption (ref)(i) imposes the sparsity conditions by requiring an upper bound on $s$. Assumption (ref)(i) further implies $(\log p)^{7} = o(n^{c_\nu})$ with $c_\nu=7/(7+\nu)\in(0,1)$, which is required for the Gaussian approximation property used for the M test in the next section. Assumption (ref)(ii) provides an asymptotic lower bound for the global IV strength $\sqrt{{\rm Q}_{A^*}(\gamma)} \asymp \|\gamma\|_2$. In classical low-dimensional IV models, strong IVs satisfy $\|\gamma\|_2\gg n^{-1/2}$. Under an exact sparse precision matrix with $q=0$ and constant sparsity indices $s_\omega$, $m_\omega$, and $s$, Assumption (ref)(ii) becomes ${\rm Q}_{A^*}(\gamma) \gg (\log p)^{7+\nu}/n$ and is almost equivalent to the strong IV condition $\|\gamma\|_2^2 \gg 1/n$ under low dimensions up to a logarithmic term. Here, we only need global, not individual, strength for high-dimensional $\gamma$; the latter is required for post-selection inference guo2018confidence,guo2018testing.

Assumption (ref) specifies the theoretical rates for the tuning parameters. Similar restrictions are commonly used in Lasso-based estimation and inference methods bickel2009simultaneous,javanmard2014confidence,gold2020inference,belloni2022high. These rates are necessary for theoretical analysis and merely technical. We use the data-driven tuning parameter selection for practical implementation. Details are available in Section (ref) for simulations in the supplement.

The following theorem shows the asymptotic normality of $\widehat{\beta}_A$.

Theorem (ref) shows that we can use $\widehat{\beta}_A$ for inference on the treatment effect when all IVs are valid. Under the null hypothesis ((ref)) where all IVs are valid, the estimator $\widehat{\beta}_A$ is an alternative to the existing post-selection procedures belloni2014,chernozhukov2015post without depending on variable selection consistency. The suitability of $\widehat{\beta}_A$ is further demonstrated by the simulation results in Section (ref) of the supplement. In the next section, we use this initial estimator $\widehat{\beta}_A$ to construct the overidentification test for its convenience in deriving the asymptotic properties of the test statistic. In the proof of Theorem (ref) in Section (ref), we also deduce the asymptotic normality of $\widehat\beta_A-\beta_A$ for $\pi\neq0$ under the alternative set defined as ((ref)) below, which is useful in analyzing the power of our test.

Overidentifying Restriction Test

So far, we have developed an estimator $\widehat\beta_A$ in (ref). In this section, we develop testing procedures for the IV exclusion restriction (ref) using this estimator. Mainly, we test the weighted version of restriction $A^{1/2}\pi=0$ with $A = {\rm diag}(Z^\top Z/n)$. First, subtracting $D\beta_A$ from both sides of ((ref)) yields

where $\varphi_A=\varphi-\psi(\beta_A-\beta)$, $\pi_A=\pi-\gamma(\beta_A-\beta)$ and $e_A=\varepsilon_Y-\varepsilon_D\beta_A$. Note that we identify $\pi_A$, not the true $\pi$, from (ref). When $\gamma\neq 0$, $\pi = 0$ implies $\beta_A=\beta$ and hence $\pi_A=0$. Next, we derive the if and only if condition for equivalence between $\pi_A=0$ and $\pi=0$. To see this, we define the weighted quadratic forms of $\pi_A$ and $\pi$ as ${\rm Q}_{A}(\pi_A)=\pi_A^\top A\pi_A$ and ${\rm Q}_{A}(\pi)=\pi^\top A\pi$. Following from the definition of $\beta_A$ in (ref), we establish the following condition between ${\rm Q}_{A}(\pi_A)$ and ${\rm Q}_{A}(\pi)$

where ${\rm R}_A(\pi,\gamma)=\dfrac{{\rm I}_{A}(\pi,\gamma)}{\sqrt{{\rm Q}_{A}(\pi){\rm Q}_{A}(\gamma)}}\boldsymbol{1}\{{\rm Q}_{A}(\pi)>0,{\rm Q}_{A}(\gamma)>0\}$ is the relatedness between $A^{1/2}\pi$ and $A^{1/2}\gamma$. By ((ref)), if $|{\rm R}_A(\pi,\gamma)|\neq 1$, $\pi_A = 0$ if and only if $\pi = 0$ and hence it is equivalent to work with the following hypothesis for testing the null in (ref),

We interpret the condition $|{\rm R}_A(\pi,\gamma)|\neq1$ in the following Remark (ref).

In the following subsections, we propose the testing procedure for the null hypothesis in (ref). Section (ref) introduces a testing procedure for (ref) using the maximum norm $\|A^{1/2}\pi_A\|_\infty$. Intuitively, the maximum test is powerful when $\pi$ is sparse but with relatively large absolute value of $\pi_j$. However, when there are many locally invalid IVs, the maximum test might be less powerful than a quadratic form based test. Inspired by the principle of power enhancement fan2015power, kock2019power, in Section (ref), we construct an asymptotically zero quadratic statistic by an estimator of ${\rm Q}_{A}(\pi_A)$ and use it to enhance the power of the original M test.

The M Test

We start with constructing an estimator of $\pi_A$ and apply it to construct our proposed maximum test. Substituting $\beta_A$ by $\widehat{\beta}_A$ in equation (ref), we have

where\footnote{Throughout the paper, the subscript $A$ stands for a transformed variable or parameters using the unknown $\beta_A$. In addition, for generic notation $\theta$, $\widecheck \theta_A$ stands for the transformed variables or parameters using the estimator $\widehat\beta_A$, $\widehat \theta$ denotes Lasso estimators or residuals, and $\widetilde \theta$ represents debiased Lasso estimators. } $\widecheck{\varphi}_A=\varphi-\psi(\widehat{\beta}_A-\beta)$, $\widecheck{\pi}_A=\pi-\gamma(\widehat{\beta}_A-\beta)=\pi_A-\gamma(\widehat{\beta}_A-\beta_A)$ and $\widecheck{e}_A=\varepsilon_Y-\varepsilon_D\widehat{\beta}_A=e_A-\varepsilon_D(\beta_A-\widehat{\beta}_A)$. The left hand side, $Y-D\widehat{\beta}_A$, is analogous to the “residual" in the Sargan test. We apply Lasso to estimate $\widecheck \pi_A $ from (ref),

where $\lambda_{3n}$ is a positive tuning parameter selected by cross-validation in practice. The bias-corrected estimator for $({\varphi}_A^{\top}, \pi_A^{\top})^{\top}$ is given by

where $\widehat{\Omega}$ is defined by ((ref)). We use this bias-corrected $\widetilde{\pi}_A$ in the maximum test.

Next, we give the approximate distribution of $\widetilde{\pi}_A$. Let $\widehat{\Omega}_z$ be the $p_z\times p$ submatrix composed of the last $p_z$ rows of $\widehat{\Omega}$. We can deduce the following approximation under the null hypothesis $\pi=0$:

By the form of the RHS of ((ref)), the asymptotic covariance matrix of $ \sqrt{n} A^{1/2}\widetilde{\pi}_A $ can be approximated by

where $\widehat{A}_0 = A^{1/2}\left(I_{p_z}-\dfrac{\widehat{\gamma} \widehat{\gamma}^\top A}{\widehat{\rm Q}_{A}(\gamma)}\right)$ and $\widehat{e}_{iA}=Y_i-D_i\widehat{\beta}_A-X_{i\cdot}^\top\widehat{\varphi}_A-Z_{i\cdot}^\top\widehat{\pi}_A$. By chernozhukov2013gaussian, the distribution of $\sqrt{n}\|A^{1/2}\widetilde\pi_A\|_\infty$ can be well approximated by that of $\|\eta\|_\infty$, where $\eta\sim N(0, \widehat{\rm V}_A )$ conditionally on the observed data. The M statistic is defined as

Then, under any significance level $\alpha$, the M test rejects the null hypothesis when $M_n(A) > {\rm cv}_A(\alpha)$, where the critical value ${\rm cv}_A(\alpha)$ is given as

In practice, ${\rm cv}_A(\alpha)$ can be approximated by simulating independent draws $\eta\sim N(0,\widehat{\rm V}_A)$, following chernozhukov2013gaussian and zhang2017simultaneous.

We then define the alternative set of $\pi$ for theoretical justification of the M test. Recall $A^*$ defined in ((ref)) is the probability limit of the weighting matrix $A$. Define the relatedness between $A^{*1/2}\pi$ and $A^{*1/2}\gamma$ as

similar to the relatedness in ((ref)) with the weighting matrix $A$. Treating all other parameters such as $\beta$, $\gamma$ as given, we define the alternative set of $\pi$, for any $t>0$, as

for some absolute constant $c_r\in(0,1)$, where

and $\beta_{A^*}:={\rm I}_{A^*}(\gamma,\Gamma)/{\rm Q}_{A^*}(\gamma)$ are defined similarly to $\pi_A$ below ((ref)) and $\beta_A$ in ((ref)) with $A$ replaced by $A^*$. We have the following technical assumptions, which are important for the theoretical properties of the M test.

Recall that $\widehat{\rm V}_A$ defined in ((ref)) estimates the asymptotic variance of $\sqrt{n}A^{1/2}\widetilde\pi_A$, whose limiting form ${\rm V}_{A^*}$ is defined in ((ref)) in the supplement. The following assumption is needed to establish that the diagonal elements of ${\rm V}_{A^*}$ are lower-bounded away from zero, which is required for the theoretical justification of the maximum test.

Assumption (ref) can be interpreted as an overidentification condition: the global weighted IV strength $\sqrt{{\rm Q}_{A^*}(\gamma)}$ cannot be dominated by only one of the IVs. In other words, the model needs to be overidentified by two dominating IVs with the same order of strength.

Power Enhancement

As discussed earlier, the M test might not be powerful enough when there are many locally invalid IVs. In this case, a test statistic used to estimate the weighted quadratic form ${\rm Q}_{A}(\pi_A)=\pi_A^\top A\pi_A$ can be leveraged for power enhancement. Theorem (ref) shows that the M statistic $M_n(A)$ defined by ((ref)) satisfies $\Pr(M_n(A) > {\rm cv}_A(\alpha) ) \to \alpha$ as $n\to\infty$. Suppose that we have another statistic $q_n(A)\stackrel{p}{\longrightarrow} 0$ as $n\to\infty$ under the null hypothesis. Define $PM_n(A) := M_n(A)\vee q_n(A)$. Then, the PM test,

also has asymptotic size $\alpha$ with power at least the same as that of the M test $\boldsymbol{1}\{M_n(A) > {\rm cv}(\alpha) \}$. We then construct an asymptotically zero statistic $q_n(A)$ named as the Q test statistic in ((ref)) that measures the magnitude of ${\rm Q}_{A}(\pi_A)$. This Q test is only for power enhancement and we do not perform this test individually. Following the same idea about the debiased estimators of ${\rm Q}_{A}(\gamma)$ and ${\rm I}_{A}(\gamma,\Gamma)$ in ((ref)) and ((ref)), we construct the following bias-corrected estimator of ${\rm Q}_{A}(\pi_A)$:

where $\widehat\pi_A$ and $\widetilde\pi_A$ are defined in ((ref)) and ((ref)) respectively. We then define the Q statistic as

For ease of discussion, we define a new alternative set

with $|{\rm R}_{A^*}(\pi,\gamma)|$ defined by ((ref)) and the absolute constant $c_r\in(0,1)$ used in ((ref)). We have the following results in favor of the asymptotically zero Q statistic $q_n(A)$.

Under the alternative with $\pi\neq 0$, we still need the sparsity condition Assumption (ref)(i) to prove the consistency of $\widehat\pi_A$ in ((ref)), which is required to establish the asymptotic properties of the test statistic $q_n(A)$. Under this particular sparsity condition, the conditions required for Q statistic $q_n(A)$ achieving asymptotic power 1 are not weaker than those required for the M test. Thus, the power enhancement by Q statistic $q_n(A)$ compared to the original M test is not visible from the theoretical point of view. Nevertheless, the power enhancement procedure is still favorable in practice. As mentioned in the paragraph right after Remark (ref), in practice, there can be many locally invalid IVs with small $|\pi_j|$. Our numerical studies in Section (ref) show that power enhancement is evident for many locally invalid IVs, with the type I error almost unaffected.

Practitioners can easily implement our test with a high-dimensional dataset\footnote{The R code for implementing the above method is available at \url{https://github.com/ZiweiMEI/PMtest}.}. The steps for the PM test are summarized in Algorithm (ref).

Empirical Example

To illustrate the usefulness of the proposed test with high-dimensional data, we revisit the empirical analysis of the effect of trade on economic growth (frankel1999does, FR99 hereafter). fan2018nonparametric searched for instruments from all geographical variables following the celebrated gravity theory of trade. In this paper, we update all data to 2018 and expand the set of IVs from fan2018nonparametric to include potentially invalid IVs from World Bank economic data. Following the literature, the dependent variable $Y$ is the logarithm of GDP. There are $n=159$ countries, and $p=p_z+p_x=58$, which includes (1) the constructed trade $\widehat{T}$ proposed by FR99 under the guidance of the gravity theory of trade, (2) the logarithms of population $X_1$ and land area describing the sizes of the countries $X_2$ and (3) other covariates and candidate IVs concerning geographical characteristics, energy, the environment and natural resources, and business activities\footnote{$\widehat{T}$, $X_1$ and $X_2$ are instruments and covariates that have been widely recognized in the literature since FR99. To make better comparisons to the literature, we do not penalize them in the Lasso problems, following the suggestions of belloni2014.}. The dependent variable, the endogenous variable, the original FR99 covariates, and a subset of the baseline instruments used in fan2018nonparametric, together with three additional and possibly invalid IVs, are summarized in Table (ref) of the supplement's Section (ref). We perform overidentification tests using this (sub)set of IVs.

We standardize the data so that all variables have zero sample mean and unit standard deviation, under which the weighting matrix $A$ is the identity matrix. Table (ref) shows the p-values of different tests performed on the real data. We first test the correct specifications of all 16 instruments in Table (ref) and expect the null hypothesis to be rejected since at least some of the instruments, namely $Z_{14}$ (air pollution), $Z_{15}$ (access to electricity) and $Z_{16}$ (business environment), are likely to have a direct effect on economic growth. The variable dimensions in this case are $p_x=42$ and $p_z=16$. We can see that the M test and PM test reject the null hypothesis at the 5% and 1% levels, respectively, while MCD fails to reject the validity of IVs at the 5% level. Next, we test whether a previously studied subset of IVs is valid. This application shows that empirical researchers can also use our method to test whether a subset of IVs is valid. Here, we select the subset of IVs used in fan2018nonparametric, including $Z_1,Z_2,\cdots,Z_{13}$, as displayed in Table (ref), and treat the other three instruments as covariates. Therefore, the variable dimensions are now $p_x=45$ and $p_z=13$. All the considered tests do not reject the null hypothesis, meaning there is no evidence that this subset of instruments is invalid. The takeaway from this empirical exercise is that practitioners should be cautious in the interpretation of a failure to reject the null hypothesis by existing overidentification tests when many covariates and/or instruments are present. Using tests with low power would result in further difficulty in the estimation and inference of the endogenous treatment effect. Our proposed test improves the power in the high-dimensional IV model with potentially invalid instruments; hence, it is recommended in a data-rich environment to detect invalid instruments.

Conclusion

In this paper, we develop a new test on overidentifying restriction for linear IV models with high-dimensional covariates and/or IVs. This test {allows for $p>n$ and $p_x\to\infty$}, and is robust to heteroskedasticity. We show that, by utilizing a sparse model structure, our PM test has better power than the $\chi^2$-type tests even when {$p<n$ and $p\to\infty$, under which all tests under discussion are feasible.} As high-dimensional data become more common in observational studies, the PM test should have many applications in detecting instrument misspecifications. From a technical perspective, this paper extends the inference of maximum and $L_2$ norms to heteroskedastic errors, and shows its applicability to triangular systems such as the linear IV regression model.

\setcounter{section}{0} \setcounter{page}{1} \setcounter{footnote}{0}

The Appendices include the following parts: Section (ref) provides additional discussions complementary to the theory and the empirical example in the main text. Section (ref) collects simulation results. Section (ref) contains all technical proofs. \newcounter{counter}[section] \setcounter{table}{0} \setcounter{equation}{0} \setcounter{figure}{0} \setcounter{theorem}{0} \setcounter{assumption}{0} \setcounter{lemma}{0} \setcounter{remark}{0} \setcounter{corollary}{0} \setcounter{proposition}{0}

\setcounter{subsection}{0} \setcounter{table}{0} \setcounter{equation}{0} \setcounter{figure}{0} \setcounter{theorem}{0} \setcounter{assumption}{0} \setcounter{lemma}{0} \setcounter{remark}{0} \setcounter{corollary}{0} \setcounter{proposition}{0}

Additional Discussions and Details

Relation between $\pi$ and $\pi_{A}$

For simplicity of discussion, we do not distinguish $A$ from $A^*$ in this section, and we use $A^*$ to be consistent with the alternative set ((ref)). As discussed in Section (ref) of the paper, the true $\pi$ is of our interest while we work with the data scale-invariant version of $\pi_{A^*}$. It is thus helpful to look into the relation between $\pi$ and the identified $\pi_{A^*}$ for a clearer picture of the alternative set $\mathcal{H}_{M}(t)$ defined as ((ref)). Below are two illustrative examples. Example (ref) shows that perfectly parallel $A^{*1/2}\pi$ and $A^{*1/2}\gamma$ cause a zero $\pi_{A^*}$ even if $\pi\neq 0$, and hence the M test has no power to detect invalid IVs. Other overidentifying restriction tests also have no power under similar conditions. Example (ref) shows that when $A^{*1/2}\pi$ and $A^{*1/2}\gamma$ are far away from perfectly parallel, the alternative set defined by $\|A^{*1/2}\pi_{A^*}\|_\infty$ is similar to that defined by $\|{A}^{*1/2}\pi\|_\infty$ up to a square root term of sparsity indices.

In summary, the alternative set induced by the data scale-invariant version of $\pi_{A^*}$ is appropriate for power analysis of the M test.

Descriptive Statistics of the Empirical Example in Section (ref)

Simulations

Setup

The simulation DGP follows Model ((ref)) in the main text. We focus on high-dimensional covariates where \((n,p_x)\in\{(150,50),(150,100),(300,150),(300,250),(500,350),(500,450)\}\). For each pair $(n,p_x)$, we set $p_z\in\{10,100\}$ to consider both low- and high-dimensional instrumental variables. The exogenous variables $W_{i\cdot}$ are independently generated by a multivariate Gaussian distribution with mean zero and covariance matrix $\Sigma=(|0.5|^{|i-j|})_{i,j\in[p]}$. We construct the error terms as follows: \[

\] where $e_i^1|W_{i\cdot}\sim N(0,Z_{i1}^2)$, $\varepsilon_{i,D}^0$ and $e_{i}^0$ are i.i.d. $N(0,1)$ variables. We set $a_0 = 0$ for homoskedasticity and $a_0 = 2^{-1/4}$ for heteroskedasticity so that the R-square\footnote{According to Footnote 11 of bekker2015jackknife, $R^2(e^2|Z) = \dfrac{{\rm Var}[\mathbb{E}(e^2|Z)]}{{\rm Var}[\mathbb{E}(e^2|Z)] + \mathbb{E}[{\rm Var}(e^2|Z)]}$. } for the regression of $e_{i}^2$ on the IVs equals 0.2. We fix $\beta=1$. For each combination $(n,p_x,p_z)$, we set $\varphi=(1,0.5,\cdots,0.5^{s_\varphi-1}, 0^\top_{p_x-s_\varphi})^\top$ and $\psi=(1,0.6,\cdots,0.6^{s_\psi-1}, 0^\top_{p_x-s_\psi})^\top$. We consider two sparse settings of $\gamma:$

Throughout the simulation study, we set $s_\varphi=s_\psi=10$ and $s_\gamma = 7$. For IV validity, we first consider \[\pi=\pi^{(1)}=(\rho_\pi,0_{p_z-1}^\top)^\top,\] where only the first IV is invalid. To demonstrate the necessity of power enhancement, we also consider another setting of $\pi$, given as \[\pi^{(2)}:=

\] When $p_z=100$, the vector $\pi^{(2)}$ induces a much larger number of invalid instruments with a smaller maximum norm compared to $\pi^{(1)}$. In this case, the Q test applying the $L_2$ norm is expected to be more powerful than the M test. We will see the benefit of power enhancement in the simulation results. We vary $\rho_\pi$ from -1 to 1. The Lasso problems are solved by the glmnet R package. The tuning parameter is selected by cross-validation with the one-standard-error rule that is also favored in the current literature windmeijer2019use. We use the fastclime pang2014fastclime package with the built-in parameters to obtain the CLIME estimator ((ref)) and ((ref)). The package efficiently solves the problem using the parametric simplex method. In addition to the M test and PM test, we report the simulation results of the MCD test proposed by kolesar2018minimum as a representative of $\chi^2$-type tests, which allows many covariates with the restriction $(p_x+p_z)/n \to c_p\in(0,1)$ as $n\to\infty$.

Summary of Simulation Results

Tables and fugures of the empirical size and power from the simulation studies are available in Section (ref). Table (ref) shows the empirical type I errors of different tests under $\rho_\pi=0$. The MCD test controls the type I error below or close to the nominal size. However, it is infeasible when $p_x+p_z>n$. In comparison, our M test and PM test are robust to high-dimensional covariates and instruments even when $p>n$. The most severe over-rejection occurs under $(n,p_x,p_z)=(300,250,10)$, which is no more than 0.03 off from the target rejection rate 5%. In most cases, the rejection rate is close to the nominal size. The slight bias in Type I error is offset by substantial power gains compared to the MCD test, as in the figures shown below. In addition, the empirical type I errors are similar between the M test and PM test, indicating that the power enhancement for the M test has almost no effect on the empirical size. In Section (ref), we also show the simulation results of our proposed IQ estimator ((ref)) under the null hypothesis $\pi=0$. The IQ estimator has satisfactory performance in estimation and inference for $\beta$. We then discuss the power. To fix ideas, we focus on the power curves from $(n,p_x)\in\{(150,50),(500,450)\}$ shown in Figures (ref)-(ref) in the discussions. Other power curves are also available in Section (ref). Figures (ref) and (ref) show the results when $p_z=10$. With this small number of IVs, the M test and PM test have almost the same power. In addition, both tests are more powerful than the MCD test. The power improvement is more evident when $n=500$ and $p_x=450$, where $p$ is very close to the sample size $n$. Figures (ref) and (ref) show the results when $p_z=100$. Given $p\geq n$, the $\chi^2$-type MCD test becomes infeasible; hence, the results of the MCD test are unavailable in these two figures. Again, the power curves of the M test and PM test are close when there is only one invalid IV ($\pi=\pi^{(1)}$), as shown in the first and third rows of the two figures. However, with 30 locally invalid instruments ($\pi=\pi^{(2)}$, the second and fourth rows), the M test is outperformed by the PM test. This result shows that our power enhancement procedure makes the test more powerful in some extreme cases with many locally invalid instruments without significant impacts on type I errors. Finally, the results are robust to the settings of $\gamma$ and heteroskedastic errors.

Tables and Figures of Size and Power

Simulation Results of Estimation and Inference for $\beta$

Tables (ref) and (ref) show the simulation results of our proposed IQ estimator ((ref)) and its confidence interval under the null hypothesis $\pi=0$. The $100(1-\alpha)\%$ confidence interval is given by \[\left[\widehat\beta_A-z_{\alpha/2}\sqrt{\dfrac{\widehat{\rm V}_\beta}{n}},\widehat\beta_A+z_{\alpha/2}\sqrt{\dfrac{\widehat{\rm V}_\beta}{n}}\right]\] where $z_{\alpha/2}$ is the $1-\alpha/2$ quantile of the standard normal distribution, and $\widehat{\rm V}_\beta$ is defined in Theorem (ref). The IQ estimator has satisfactory performance in estimation and inference for $\beta$ when all IVs are valid.

Proofs

Throughout the proof, we use $C$ and $c$ to denote generic absolute constants that may vary from place to place. We first present some useful preliminary lemmas in Section (ref). Section (ref) includes the proofs of the theoretical results in Section (ref) of the main text. Firstly, some essential propositions about the initial Lasso estimators and test statistics are summarized in Section (ref). Secondly, we give the proof of Theorem (ref) in Section (ref). Section (ref) includes the proofs of the main theoretical results of the proposed tests in Section (ref) of the main text. Firstly, some essential propositions are given in Section (ref). Secondly, we give the proofs of Theorems (ref) and (ref) in Sections (ref) and (ref), respectively.

Preliminary Lemmas

This subsection provides useful lemmas implied by (or directly from) other literature. Define the restricted eigenvalue of the empirical Gram matrix $\widehat{\Sigma} = W^\top W /n $, given as

where the restricted set $\mathcal{R}(s):=\{\theta\in\mathbb{R}^p:\|\theta_{\mathcal{M}^c}\|_1\leq 3\|\theta_\mathcal{M}\|_1\text{ for all }\mathcal{M}\subset\mathbb{R}^p \text{ and } |\mathcal{M}|\leq s\}$. Lemma (ref) provides the Lasso convergence rate. This is a direct result of Lemma 1 in mei2022lasso and Theorem 6.1 of buhlmann2011statistics.

Lemma (ref) shows the probability bounds for the maximum norm of some sub-Gaussian and sub-exponential variables, and a lower bound of the restricted eigenvalue useful in the proofs.

Lemma (ref) shows that under certain conditions, linear transformations of sub-Gaussian vectors are still sub-Gaussian.

Lemma (ref) shows the asymptotic properties of the inverse covariance estimator CLIME (ref).

Lemma (ref) shows a more convenient asymptotic regime used in the proofs.

Lemma (ref) shows the probability bounds for the maximum norms that are useful to bound the estimation errors of asymptotic variance.

Proofs of the Initial Estimator in Section (ref)

Essential Propositions

Proposition (ref) provides probability upper bounds of the Lasso estimators of the reduced form estimators.

Proposition (ref) provides probability upper bounds of the weighting matrix $A$.

Proposition (ref) provides some error bounds that are useful in deriving estimation error of the asymptotic variance. Define \[\sigma_{iA}^2 = \sigma_{i,Y}^2 - 2\beta_A \sigma_{i,YD} + \beta_A^2\sigma_{i,D}^2.\] Similarly, define \[\sigma_{iA^*}^2 = \sigma_{i,Y}^2 - 2\beta_{A^*} \sigma_{i,YD} + \beta_{A^*}^2\sigma_{i,D}^2\] where $\beta_{A^*}=\dfrac{{\rm I}_{A^*}(\gamma,\Gamma)}{{\rm Q}_{A^*}(\gamma)}$ is defined below ((ref)).

Proof of Theorem (ref)

Define $\widehat{\rm U}_\beta := n^{-1}\sum_{i=1}^n(W_{i\cdot}^\top\widehat{u}_{\gamma})^2(\widehat{\varepsilon}_{i,Y}-\widehat{\beta}_A\widehat{\varepsilon}_{i,D})^2$ where $\widehat{u}_{\gamma}$ is defined in Theorem (ref). Thus, $\widehat{\rm V}_\beta^{-1/2}=\widehat{\rm Q}_{A}(\gamma)^{-1}\widehat{\rm U}_\beta^{1/2}$. We will show a stronger result that is useful for power analysis: when $\pi\in\mathcal{H}_{A^*}(t)$ for any absolute constant $t$, the following asymptotic normality holds

under the conditions in Theorem (ref). Define $u_\gamma=\Omega(0_{p_x}^\top,(A\gamma)^\top)^\top$. The estimation error of $\widehat{\rm Q}_{A}(\gamma)$ can be decomposed as

By Proposition (ref), Proposition (ref). \[{\rm Q}_{A}(\widehat{\gamma}-\gamma) \lesssim_p \|\widehat{\gamma}-\gamma\|_2^2 \lesssim_p \dfrac{s\log p}{n}.\] Additionally, by Lemma (ref), \[

\] where the last inequality applies $s_\omega m_{\omega}^{1-2q} s\log p^{(1-q/2)} = o(n^{(1-q)/2})$ and $s\sqrt{\log p} = o(\sqrt{n})$ implied by Assumption (ref) and Lemma (ref). Recall that $\widehat{u}_{\gamma} = \widehat{\Omega}(0_{p_x}^\top,\widehat{\gamma}^\top A)^\top$ and $u_{\gamma} = \Omega(0_{p_x}^\top,\gamma^\top A)^\top$. Thus,

where the last inequality applies Proposition (ref) and Lemma (ref). Then by ((ref)) and ((ref)) \[

\] Thus,

The probability bound of the first term implied by ((ref)) is given as \[

\] which, together with ((ref)), implies that

Thus, $\widehat{\rm Q}_{A}(\gamma)/{\rm Q}_{A}(\gamma)\stackrel{p}{\longrightarrow}1$ by ((ref)), Assumption (ref), and the fact that $\sqrt{{\rm Q}_{A^*}(\gamma)}\asymp \|\gamma\|_2$. Besides, define $\widehat u_\Gamma = \widehat\Omega(0_{p_x}^\top, (A\widehat\Gamma)^\top )^\top$, $u_\Gamma = \Omega(0_{p_x}^\top, (A\Gamma)^\top )^\top$ and $u_{\pi_A} = \Omega(0_{p_x}^\top, (A\pi_A)^\top)^\top$ where $\pi_A=\pi-\gamma(\beta_A-\beta)=\Gamma-\gamma\beta_A$. The estimation error of $\widehat{\rm I}_{A}(\gamma,\Gamma)$ can be decomposed as

and following the same procedures to derive ((ref)), we deduce that

where the last step applies $\|\Gamma\|_2\lesssim \|\pi\|_2 + |\beta|\cdot\|\gamma\|_2\lesssim\|\pi\|_2 + \|\gamma\|_2$. Then by ((ref)), ((ref)) and ((ref)) we deduce that

and by ((ref)),

Note that when $\pi\in\mathcal{H}_{A^*}(t)$, $\|\pi\|_2 \lesssim \sqrt{s\log p/n}$, which implies $\left|u_{\pi_A}^\top \dfrac{W^\top \varepsilon_D}{\sqrt{n}}\right|\lesssim_p \dfrac{m_\omega s\log p}{\sqrt{n}}$ together with ((ref)). Thus,

Define the asymptotic variance of the first term on the RHS of ((ref))

The remaining of this proof will show that

Step 1. Show that ${\rm U}_\beta \asymp_p \|\gamma\|_2^2$. Recall that $\sigma_{iA^*}^2 = \mathbb{E}(e_{iA^*}^2|W)$ where $e_{iA^*} = \varepsilon_{i,Y}-\beta_{A^*}\varepsilon_{i,D}$. By the upper and lower bounds of conditional variances and covariances in Assumption (ref), we deduce that \[

\] where $\rho_\sigma$ is specified in Assumption (ref). By ((ref)),

and hence by the bound of the second term on the LHS of ((ref)), $\sigma_{iA}^2\asymp_p \sigma_{iA^*}^2 \asymp 1$ uniformly for all $i\in[n]$. In addition, \[ \left|\dfrac{u_\gamma^\top \widehat{\Sigma} u_\gamma}{u_\gamma^\top \Sigma u_\gamma} - 1 \right| \leq \dfrac{\|u_\gamma\|_1^2\cdot\|\widehat{\Sigma} - \Sigma\|_\infty}{u_\gamma^\top \Sigma u_\gamma} \lesssim_p \dfrac{\|\gamma\|_2^2\sqrt{\dfrac{m_\omega^2 s^2 \log p}{n}}}{u_\gamma^\top \Sigma u_\gamma} \lesssim m_\omega\sqrt{\dfrac{ s^2 \log p}{n}} = o(1), \] under Assumption (ref), implying that

Consequently, by the definition of ${\rm U}_\beta$ in ((ref)),

Step 2. Define $\chi_i = \dfrac{ u_\gamma^\top W_{i\cdot} e_{iA}}{\sqrt{n{\rm U}_\beta}}$ where $e_{iA}$ is the $i$-th element in the $n$-dimensional vector $e_A$. Thus we have $\mathbb{E}(\chi_i|W) = 0$ and $\sum_{i=1}^n \mathbb{E}(\chi_i^2|W) = 1$. By Corollary 3.1 of hall1980martingale, it suffices to show the following conditional Lindeberg condition

for any fixed $\chi > 0$. Following the same arguments in the proof of Lemma 24 in javanmard2014confidence, each element of the matrix $\Omega W^\top$ is sub-Gaussian. Consequently, $\|\Omega W^\top\|_\infty \lesssim_p \sqrt{\log n + \log p} \leq n^{1/4}$ when $\log p= o(n^{1/3})$ as implied by Assumption (ref). Thus by Proposition (ref) and ((ref)), \[

\] w.p.a.1 for some absolute constant $C_\chi>0$. Also, by ((ref)),

where the absolute constants $c_0$ and $C_0$ are defined in Assumption (ref). In addition, ((ref)) and the definition of ${\rm U}_\beta$ in ((ref)) imply

Therefore, for any $\chi>0$, w.p.a.1,

where the fourth row applies ((ref)) and the fifth row applies ((ref)). The upper bound $\left(\dfrac{C_\chi\cdot s^{1/2}}{\chi \cdot n^{1/4}}\right)^{c_0} \to 0$ as $n\to\infty$, as $s^{1/2} = o(n^{1/4})$ implied by Assumption (ref). Then the Lindeberg condition ((ref)) holds. We have completed Step 2. Step 3. Show that $\widehat{\rm U}_\beta / {\rm U}_\beta \stackrel{p}{\longrightarrow} 1$. We decompose the estimation error of the asymptotic variance $\widehat{\rm U}_\beta - {\rm U}_\beta$ as \[

\] where \[\Delta_{1\beta} = \dfrac{1}{n}\sum_{i=1}^n\left(\widehat{u}_{\gamma}^\top W_{i\cdot}\right)^2\left( (\widehat\varepsilon_{i,Y} - \widehat{\beta}_A\widehat\varepsilon_{i,D})^2 - \sigma_{iA}^2\right),\] and \[\Delta_{2\beta} = \dfrac{1}{n}\sum_{i=1}^n\left((\widehat{u}_{\gamma}-u_{\gamma})^\top W_{i\cdot}\right)^2\sigma_{iA}^2 + (\widehat{u}_{\gamma}-u_{\gamma})^\top \dfrac{2}{n}\sum_{i=1}^n W_{i\cdot}W_{i\cdot}^\top \sigma_{iA}^2 u_\gamma. \] We first bound $\Delta_{1\beta}.$ Note that by ((ref)) and ((ref)),

Then by Lemma (ref) $\widehat\beta_A - \beta_A = o_p(1)$ and hence by ((ref)) $\widehat\beta_A\lesssim_p 1$ and

Then by Proposition (ref), ((ref)), ((ref)), ((ref)), ((ref)) and the fact that $\sigma_{iA}^2 = \sigma_{i,Y}^2 + \beta_A^2\sigma_{i,D}^2 - 2\beta_A\sigma_{i,YD}$, we deduce that \[

\] where the last inequality applies ((ref)) and ((ref)). In addition, by Lemma (ref) we know the entries of \[\widetilde{W}_{i\cdot} := \Omega W_{i\cdot}\] are sub-Gaussian with uniformly bounded sub-Gaussian norms. Then similar upper bounds as Proposition (ref) still hold with $W_{i\cdot}$ replaced by $\widetilde{W}_{i\cdot}$, which implies \[\left\|\dfrac{1}{n}\sum_{i=1}^n \widetilde{W}_{i\cdot}\widetilde{W}_{i\cdot}^\top\left[(\widehat\varepsilon_{i,Y} - \widehat{\beta}_A\widehat\varepsilon_{i,D})^2-\sigma_{i A}^2\right]\right\|_\infty \lesssim_p \dfrac{s^2\log p}{n} + \left(1+\dfrac{1}{\|\gamma\|_2}\right)\sqrt{\dfrac{\log p}{n}}, \] and \[\left\|\dfrac{1}{n}\sum_{i=1}^n {W}_{i\cdot}\widetilde{W}_{i\cdot}^\top\left[(\widehat\varepsilon_{i,Y} - \widehat{\beta}_A\widehat\varepsilon_{i,D})^2-\sigma_{i A}^2\right]\right\|_\infty \lesssim_p \dfrac{s^2\log p}{n} + \left(1+\dfrac{1}{\|\gamma\|_2}\right)\sqrt{\dfrac{\log p}{n}}.\] Then, by Assumption (ref), Lemma (ref) and ((ref)), \[

\] where the last equality applies ((ref)). We next bound $\Delta_{2\beta}$. By Assumption (ref), Lemma (ref) and Proposition (ref), \[

\] The probability upper bounds of $\Delta_{1\beta}$ and $\Delta_{2\beta}$ imply that \[\dfrac{\widehat{\rm U}_\beta}{{\rm U}_\beta} - 1 = \dfrac{\Delta_{1\beta} + \Delta_{2\beta}}{{\rm U}_\beta} \stackrel{p}{\longrightarrow} 0,\] or equivalently, $\dfrac{\widehat{\rm U}_\beta}{{\rm U}_\beta} \stackrel{p}{\longrightarrow} 1.$ This completes the proof of Theorem (ref).

Proofs of Theorems in Section (ref)

Essential Propositions

Proposition (ref) provides Lasso estimation errors of the identified parameters $\pi_A$ that measure IV validity.

Proposition (ref) provides an intermediate result for lower bounded individual variances of the test statistic for the M test.

Proposition (ref) shows the Gaussian Approximation property for the key component in the test statistic, which is the key for the asymptotic size and power of the M test. Define

and $e_{A^*} = ( e_{1A^*}, e_{2A^*},\cdots, e_{nA^*})^\top$.

Proposition (ref) provides a decomposition of the debiased Lasso estimator $\widetilde\pi_A$ of the target vector $\pi_{A^*}$.

Proposition (ref) provides a probability upper bound for the estimation error of ${\rm V}_A$. Define

with $A^*$ defined as ((ref)). Recall that $\Omega_z$ is the $p_z\times p $ submatrix composed of the last $p_z$ rows of $\Omega := \Sigma^{-1}$.

Proof of Theorem (ref)

This proof follows the procedure in the proof of Theorem 2.2 in zhang2017simultaneous. Conditional on the observed data, the normal vector $\eta \in [p_z]$ is equal in distribution to \[\eta \stackrel{d}{=} \dfrac{1}{\sqrt{n}}\sum_{i=1}^n \widehat{A}_0 \widehat{\Omega}_z W_{i\cdot}\widehat{e}_{iA}\cdot w_{i}, \] where $\{w_i\}_{i=1}^n$ are i.i.d. standard normal variables. Define \[\mathcal{T} = \max_{j\in[p_z]}\sqrt{n}A_j^{1/2\top}(\widetilde\pi_A-\pi_A),\ \mathcal{T}_0 = \max_{j\in[p_z]}\dfrac{1}{\sqrt{n}}\sum_{i=1}^n \xi_{ij},\] where $A_j^{1/2\top}$ denotes the $j$-th row of the matrix $A^{1/2}$, and \[\mathcal{W} = \max_{j\in[p_z]} \eta_j. \] Here $\mathcal{T}$ and $\mathcal{T}_0$ are analogs of “$T_0$” and “$T$” in (14) of chernozhukov2013gaussian, and $\mathcal{W}$ is “$W$” and “$W_0$” in (15) of the same paper. Proposition (ref) shows that $|\mathcal{T}-\mathcal{T}_0| = o_p\left(\dfrac{1}{\log p}\right)$ and hence,

where $\zeta_1 = \dfrac{1}{\log p}$ and $\zeta_2 = o(1)$. Furthermore, define $\varpi(\vartheta):=C_\varpi \vartheta^{1/3}(1\vee \log (p_z/\vartheta))^{2/3}$ with $C_\varphi>0$ large enough and \[\Delta_{\rm V} := \|\widehat{\rm V}_A - {\rm V}_{A^*}\|_\infty. \] Finally, define the critical value of $\mathcal{W}$ \[{\rm cv}_{\mathcal{W}}(\alpha) := \inf\{x\in\mathbb{R}:{\rm Pr}_\eta(\mathcal{W}\leq x) \geq 1-\alpha\}.\] Following the same path to verify Theorem 3.2 of chernozhukov2013gaussian, we can deduce that \[\sup_{\alpha^*\in(0,1)}\left|\Pr\left(\mathcal{T}_0 > {\rm cv}_{\mathcal{W}}(\alpha^*)\right)-\alpha^*\right| \lesssim \varpi(\vartheta) + \Pr(\Delta_{\rm V} > \vartheta) + \zeta_1\sqrt{1\vee \log (p/\zeta_1)} + \zeta_2. \] where the $n^{-c}$ comes from Proposition (ref). By ((ref)) \[\zeta_1\sqrt{1\vee \log (p/\zeta_1)} + \zeta_2 = o(1).\] Take $\vartheta = 1/(\log p)^3$. By ((ref)) and the definition of $\varpi(\vartheta)$ below ((ref)), \[\varpi(\vartheta) + \Pr(\Delta_{\rm V} > \vartheta) = o(1).\] Thus

as $n \to \infty$. Prove ((ref)). Recall that $\pi_A = 0$ when $\pi=0$. Then ((ref)) is a direct corollary of ((ref)). Prove ((ref)). Let $a_{ij}$ be the normal variable with covariance matrix ${\rm V}_{A^*}$ as defined in Proposition (ref). By Step 1 in the proof of the same lemma, we have $\min_{j\in[p_z]}({\rm V}_{A^*})_{jj})\leq C$ for some absolute constant $C$. By Lemma 6 of tony2014two, for any $x\in\mathbb{R}$, \[\Pr\left(\max_{j\in[p_z]}\dfrac{(\sum_{i=1}^n a_{ij})^2}{n({\rm V}_{A^*})_{jj}} - 2\log p_z + \log \log p_z \leq x\right)\to F(x) := \exp\left[\dfrac{1}{\sqrt{\pi}}\exp\left(-\dfrac{x}{2}\right)\right],\] as $p_z \to\infty$, which implies \[\Pr\left(\max_{j\in[p_z]}\dfrac{(\sum_{i=1}^n a_{ij})^2}{n({\rm V}_{A^*})_{jj}} < 2\log p_z - 0.5\log \log p_z \right)\to1.\] By the bounds of $({\rm V}_{A^*})_{jj}$, we deduce for some absolute constant $C$,

The Gaussian approximation result from Proposition (ref) implies that \[

\]

Then ((ref)) implies

Recall that conditional on the observed data, $\eta\sim N(0,{\rm V}_A)$. By Proposition (ref) and Lemma 3.1 of chernozhukov2013gaussian, taking $t=2C\log p_z - 0.5C\log\log p_z$ for the same Lemma, we deduce that the distribution of $\sqrt{n}\|\eta\|_\infty$ can be well approximated by $\max_{j\in[p_z]}n^{-1/2}|\sum_{i=1}^n a_{ij}|$ so that \[

\] Consequently, w.p.a.1,

Furthermore, since $\|A^{1/2}-A^{*1/2}\|_1 \lesssim_p \sqrt{\log p/n}$ by Proposition (ref) and \[

\] We then deduce that

Take $C_\pi=\sqrt{5C}$. Combining ((ref)) and ((ref)), whenever $\pi\in \mathcal{H}_{A^*}(C_\pi + \epsilon)$ for any $\epsilon>0$, \[

\]

Proof of Theorem (ref)

We have the following decomposition of $\widehat Q_A$

where

and

Recall that ${u}_{\pi_A} = (0_{p_x}^\top, \pi_A^\top A)^\top$ and ${\widehat u}_{\pi_A} = (0_{p_x}^\top, \widehat\pi_A^\top \widehat A)^\top$. Define $\epsilon_n:=n^{1/4}\|\pi_{A^*}\|_2$. Note that by Proposition (ref) we can deduce $Q_A \asymp_p n^{-1/2}\epsilon_n^2$. Suppose that $|\Delta_{1Q} + \Delta_{2Q}| = o_p\left(\dfrac{1+\epsilon_n^2}{\sqrt{n}\log p}\right)$.

where the last step applies Lemma (ref). Thus, w.p.a.1, \[

\] for any $c>0$. Consequently, it suffices to show that $|\Delta_Q| = |\Delta_{1Q} + \Delta_{2Q}| = o_p(n^{-1/2}(\log p)^{-1}(1+\epsilon_n^2)). $ Show $|\Delta_{1Q}| = o_p(n^{-1/2}(\log p)^{-1})(1+\epsilon_n^2))$. By ((ref)), the definition $\epsilon_n = n^{1/4}\|\pi_{A^*}\|_2$ and ((ref)),

In addition, by ((ref)), Proposition (ref), Lemma (ref), Proposition (ref) and Proposition (ref), \[

\] and hence,

With ((ref)), ((ref)) and ${\rm Q}_{A}(\widehat\pi_A-\widecheck\pi_A)\lesssim_p (1+\|\pi\|_2/\|\gamma\|_2)^2 s\log p/n$ from Proposition (ref), we can deduce that \[

\] Below we show the last step to derive the $o_p\left(\dfrac{1+\epsilon_n^2}{\sqrt{n}\log p}\right)$ term by term. By Lemma (ref) and ((ref)), \[

\] and \[

\] where the last equality applies $\epsilon_n\leq (1+\epsilon_n^2)/2$, and \[ \dfrac{\epsilon_n^2 m_\omega^2 s\log p}{n^{3/2}{\rm Q}_{A}(\gamma)} = \dfrac{m_\omega^2 s(\log p)^2}{n{\rm Q}_{A}(\gamma)}\cdot \dfrac{\epsilon_n^2}{\sqrt{n}\log p} = o_p\left(\dfrac{1+\epsilon_n^2}{\sqrt{n} \log p}\right).\] This completes the proof of $\Delta_{1Q} = o_p(n^{-1/2}(\log p)^{-1}(1+\epsilon_n^2))$. Show $\Delta_{2Q} = o_p(n^{-1/2}(\log p)^{-1}(1+\epsilon_n^2))$. Note that by Propositions (ref) and (ref), Equation (ref) and Lemma (ref), \[

\] where the last equality applies that $\dfrac{m_\omega^2 s\log p}{n^{1/2}\|\gamma\|_2} = o_p(1)$. In addition, \[\|u_{\pi_A}\|_1 \leq \|\Omega\|_1\|A\|_1\|\pi_A\|_1\lesssim_p m_\omega\sqrt{s}\|\pi\|_2 \lesssim \dfrac{m_\omega \sqrt{s} \epsilon_n}{n^{1/4}}. \] Then applying the upper bounds of $\left\|\widehat{u}_{\pi_A}-u_{\pi_A}\right\|_1 $ and $\|u_{\pi_A}\|_1$ derived above, together with Proposition (ref), ((ref)), ((ref)) and ((ref)), the first term of $\Delta_{2Q}$ is bounded by \[

\] where the last two steps apply Lemma (ref). Besides, using the same set of probability upper bounds, the second term of $\Delta_{2Q}$ is bounded by \[

\] This completes the proof of Theorem (ref).