A Ridge-Regularised Jackknifed Anderson-Rubin Test

A Ridge-Regularised Jackknifed Anderson-Rubin Test

\def\spacingset#1{ {#1}} \spacingset{1}

\if00 \fi

\if10 {

} \fi

{\it Keywords:} instrumental variables, weak identification, high dimensional models, ridge regression. \spacingset{1.8}

Introduction

Instrumental variables (IVs) are commonly employed in economics and related fields to estimate causal effects from observational data. The use of a large number of IVs has gained popularity due to weak identification, where researchers aim to capture limited exogenous variation in endogenous covariates and obtain more precise inference. One example is Mendelian randomisation in biology, where weakly associated genetic mutations are used as IVs, and the number of IVs can exceed the number of observations Davies:2015,burgess2021mendelian. Other prominent examples are the granular IV approach Gabaix:2020ws and the saturation approach to identify treatment effects Blandhol2022.\footnote{Recent examples of applications with many IVs include Muller, who uses judge dummies interacted with individual covariates as IVs to identify causal effects of incarceration; vanDuijn, who derive an equation for residential sorting that produces many more IVs than observations; and Lonn, who derive structural asset-pricing models that lead to a vast number of moment conditions.} However, it is crucial to have reliable inference methods that account for the potential lack of joint informativeness of a large number of weak IVs, especially when pretesting the strength of IVs is impractical or not available due to the excessive number of IVs compared to observations. The present paper contributes to the literature on the development of such methods.

We propose a ridge-regularised jackknifed Anderson-Rubin (RJAR) test to construct confidence sets for the coefficients of endogenous variables in weakly-identified and heteroskedastic IV models when the number of IVs is large. Jackknife-based methods have recently been used in this context because they are applicable in an asymptotic framework where the number of IVs diverges with the number of observations. However, by relying on existing central limit theorems developed for standard projection matrices, these methods require that the number of IVs be less than the number of observations, and often perform poorly when the number of IVs is close to (but still less than) the number of observations. Recently-proposed regularisation approaches for inference under many IVs require strong identification or a sparse relationship between the endogenous variables and the IVs to work well. By combining jackknifing with ridge regularisation, we provide a test that has the desired asymptotic size under heteroskedasticity, arbitrarily weak identification, and more IVs than observations, all while achieving good power both when the relationship between the endogenous variables and the IVs is sparse and when it is dense.

Throughout this paper, we focus on Anderson:1949tx tests. This is because, in addition to being robust to weak IVs, they are also robust to arbitrary relationships between the endogenous variables and the IVs, since they make no assumption on the first-stage projection of the endogenous variables on the IVs. In the context of AR tests with an increasing number of IVs, it is helpful to distinguish three different asymptotic regimes.

The first `moderately many' IVs regime allows the number of IVs, $k_n$, to grow with the sample size, $n$, but still requires it to be asymptotically negligible with respect to the sample size. Examples include Andrews:2007bl, who require $k_n/n^{1/3} \to 0$, and Phillips:2017bz, who require $k_n / n \to 0$.

The second `many' IVs regime allows the number of IVs to be of the same order of magnitude as the number of observations. Anatolyev:2010kk provide an AR test that remains valid for $k_n/n \to \tau$, $0 \leq \tau < 1$, $n \to \infty$, provided that the error terms are homoskedastic and the IVs satisfy a restrictive balanced-design assumption.\footnote{See Anatolyev:2017ee and Crudu:2020bk for a discussion of the restrictiveness of the balanced-design assumption of Anatolyev:2010kk.}$^,$\footnote{See Kaffo:2017gp for a bootstrapped version of the Anatolyev:2010kk AR test.} Bun:2021en provide analogous results for the GMM version of the AR statistic under the assumption of independently and identically distributed (i.i.d.) data. These results were recently extended in the form of a jackknifed AR statistic by Crudu:2020bk and Mikusheva:2020uo to the case where errors are allowed to display arbitrary heteroskedasticity, and the only assumption on the IVs is that the diagonal entries of the projection matrix of the IVs are bounded away from unity from above.\footnote{ Boot, Matushita:2022,Lim propose weak-identification robust jackknife-based procedures for inference with many IVs. However, these methods make additional assumptions on the first-stage projection of the endogenous variables on the IVs, and hence fall outside of the scope of this paper.}

The third `very many' IVs regime allows the number of IVs to grow with $n$, and further allows the number of IVs to be greater than the number of observations. Belloni:2012kw propose a Sup Score test that remains valid under mild conditions, and allows the number of IVs to increase exponentially with the sample size. Carrasco:2016un propose a ridge-regularised AR statistic that allows for more IVs than observations under the assumption of i.i.d. data and homoskedasticity. Kapetanios:2015fp extend Bai:2010ds by proposing weak-identification robust factor-based tests that in principle allow for the number of IVs to be larger than the number of observations, provided that the factor structure of the IVs is sufficiently strong.

Using the notation of the model introduced in the next section, Table (ref) provides a schematic overview of the main assumptions and results in the literature on inference that is robust to many weak IVs.

Our test provides a twofold extension of the existing literature. First, it allows for valid inference under many IVs and heteroskedastic errors while further weakening the assumptions of similar tests proposed by CMS and MS. This is made possible by deriving the limiting behaviour of the RJAR statistic from the bottom up, without relying on the existing results in Chao:2012iz or Hansen:2014ie. Second, this test allows for more IVs than observations under arbitrary heteroskedasticity of the error terms. The only other approach currently available in the literature that is robust to heteroskedastic error terms and more IVs than observations is the Sup Score test of BCCH. Simulations show that the RJAR test has power comparable to the Sup Score test of BCCH whenever the signal in the first stage is sparse (i.e., only a few of the IVs are informative), and substantially more power when the signal in the first stage is dense (i.e., not sparse). Simulations also show that the RJAR test is as powerful as, and in some cases more powerful than, the AR test of CT, the validity of which has been established only under homoskedastic error terms. Finally, we provide a comparison of the most recently proposed approaches to conducting inference in possibly heteroskedastic linear IV models when the number of IVs is not negligible with respect to the sample size. Indeed, using extensive simulation evidence and an empirical application based on Card:2009hj, we provide a comparison between these existing `state-of-the-art' methods in a controlled and comparable setting.

In the rest of the paper, we use the following notation. $I_p$ denotes the $p \times p$ identity matrix. For an $m\times 1$ vector $a = [a_1, \dots, a_m]'$, $||a||_2 := \sqrt{\sum_{j = 1}^{m}a_j^2}$. The entry $(i, j)$ for an $m\times p$ matrix $A$ is denoted as $A_{ij}$ for $1\leq i \leq m$, $1 \leq j \leq p$. $\text{tr}(A):= \sum_{i=1}^mA_{ii}$ for any $m\times m$ matrix $A$ with entries given by $A_{ij}$ for $1 \leq i, j \leq m$, and $||A||_F\coloneqq \sqrt{\text{tr}(A'A)}$ denotes the Frobenius norm of $A$. The remaining notation follows standard conventions.

The structure of the paper is as follows. Section (ref) specifies the linear IV model considered throughout. Section (ref) introduces the RJAR test, and provides the main asymptotic results. Section (ref) provides simulation evidence on the size and power of our RJAR test and compares it with the tests proposed by BCCH, CT, CMS and MS. Section (ref) considers an empirical application based on Card:2009hj. Section (ref) concludes. All proofs are given in the Supplementary Material.

Model

We consider the heteroskedastic linear IV model given by

where $y$ is an $n\times 1$ vector containing the dependent variable, $X$ is an $n\times g$ matrix containing the endogenous variables, $\beta$ is a $g\times 1$ coefficient vector, $\varepsilon$ is an $n\times 1$ vector containing the structural error terms, $Z$ is an $n \times k_n$ matrix containing the IVs, $\Pi$ is a $k_n\times g$ coefficient matrix, and $V$ is an $n\times g$ matrix containing the first-stage errors. $k_n$ can diverge with $n$, but $g$ is fixed. Also, let $y_i$, $X_i$, $\varepsilon_i$, $Z_i$, and $V_i$ denote the $i^{\text{th}}$ row of $y, X, \varepsilon, Z$, and $V$, respectively. As in BCCH, CMS and MS, we treat $Z$ as fixed (non-stochastic),\footnote{The treatment of IVs as non-stochastic is an assumption that has been commonly used in several of the papers that are most closely related to ours, see Table 1. With stochastic IVs, one can interpret the results as holding conditional on $Z$, see also CMS Footnote 1.} and assume that any exogenous covariates have been partialled out (see the discussion of Assumption (ref) below). We exclusively consider methods that allow for arbitrarily weak identification, i.e., methods that control asymptotic size irrespective of the value of $\Pi$.

Inference is conducted on the coefficient vector $\beta$ by testing hypotheses of the following type for a prespecified $\beta_0 \in \Re^g$:

For a given non-randomised test of asymptotic size $\alpha \in (0, 1)$, a confidence set of asymptotic coverage $1-\alpha$ can be constructed as the collection of those $\beta_0$ for which $H_0$ in Equation (ref) is not rejected. For convenience, define

which we refer to as the structural error under the null hypothesis.

It will become apparent below that so long as the error term, $\varepsilon$, remains additive, the linearity in the structural equation (ref) and the first-stage equation (ref) can be relaxed to allow for any (known) real-valued function, without affecting the asymptotic size of the RJAR test.

The RJAR test

This section introduces our proposed RJAR test, derives its large sample properties under the null hypothesis in Equation (ref), and discusses its relationship to the most closely related tests in the literature: the ridge-regularised AR test of CT, the jackknifed AR tests of CMS and MS, and the Sup Score test of BCCH.

Definition of the RJAR test

The original AR test of the null hypothesis in Equation (ref) can be thought of as a test that the IVs are exogenous using the implied structural errors in Equation (ref). More specifically, the AR test is a Wald test of the significance of the IVs in the auxiliary regression of the structural errors under the null, $e_i(\beta_0)$, on the IVs, $Z_i$.\footnote{The AR statistic is defined as $e(\beta_0)'Z \widehat{\Sigma}^{-1} Z'e(\beta_0)/n$, where $\widehat{\Sigma}$ is a consistent estimator of $\text{Var}[Z'e(\beta_0)/\sqrt{n}]$.} The weak-IV-robust AR test of the null hypothesis in Equation (ref) with asymptotic size $\alpha$ rejects the null if and only if the AR statistic exceeds the $1-\alpha$ quantile of a $\chi^2$ distribution with $k_n$ degrees of freedom. When the number of IVs $k_n$ grows with $n$, i.e., when there are many (potentially weak) IVs, the $\chi^2$ approximation of the original AR statistic breaks down. The recently proposed AR test of CT, which we shall briefly review in Section (ref), uses ridge-regularisation to allow for a growing number of IVs that may even exceed the sample size, but is not robust to arbitrary heteroskedasticity in the error terms. The recent papers by CMS and MS, which we shall also briefly review in Section (ref), propose jackknifed versions of the AR test that remain valid when $k_n$ grows with $n$, but require $k_n<n$. Our proposed method combines ridge regularisation with jackknifing of the auxiliary regression of $e_i(\beta_0)$ on $Z_i$ to allow for a weak-identification robust test capable of dealing with both more IVs than observations and arbitrary heteroskedasticity in the error terms.

We first standardise the IVs in-sample (after partialling out any covariates) as in BCCH (p. 2393) so that

The RJAR test for the testing problem in Equation (ref) is then based on the following statistic:

where $r_n\coloneqq \text{rank}(Z)$ (assumed to be positive without loss of generality),

and $P^{\gamma_n} := Z(Z'Z + \gamma_n I_{k_n})^{-1}Z'$ is the ridge-regularised projection matrix with $\gamma_n \geq 0$ if $r_n = k_n$ and $\gamma_n>0$ if $r_n < k_n$. $\gamma_n$ is a (sequence of) regularisation parameter(s) whose choice we discuss below. We note that setting $\gamma_n = 0$ when $r_n = k_n$ corresponds to the case where a standard least-squares projection matrix $Z(Z'Z)^{-1}Z'$ is used to construct the RJAR statistic. This is the standard jackknife statistic. We also note that the $r_n$-scaling in the denominator of the RJAR statistic cancels with the $r_n$ in the denominator of $\hat{\Phi}_{\gamma_n}(\beta_0)$.

We set the regularisation parameter $\gamma_n$ to

where $\Gamma_n \coloneqq \{\gamma_n\in \Re : \gamma_n \geq 0 \text{ if } r_n = k_n, \text{ and } \gamma_n \geq \gamma_{-} > 0 \text{ if } r_n<k_n\}$ for some constant $\gamma_{-}$ not depending on $n$. The existence of $\gamma^*_n$ is shown in Lemma (ref) in the Supplementary Material. We let $\gamma_n^*$ be an element of $\underset{\gamma_n \in \Gamma_n}{\arg\max}\sum_{i = 1}^n\sum_{j\neq i}(P^{\gamma_n}_{ij})^2$ out of conservativeness, i.e., to make Assumption (ref) below as plausible as possible given the IVs. Furthermore, we let $\gamma^*_n$ be the maximal element of this set because the maximiser is not necessarily unique without imposing additional assumptions on the singular values and left-singular vectors of the IVs (although in practice we only found unique maximisers). We choose to take the maximum of the maximisers to make the smallest eigenvalues of the ridge-regularised Gram matrix, $Z'Z + \gamma_nI_{k_n}$, as far away from zero as possible when $r_n < k_n$. To see that $\gamma_n^*$ maximises the smallest eigenvalue of $Z'Z + \gamma_nI_{k_n}$ among the elements of $\underset{\gamma_n \in \Gamma_n}{\arg\max}\sum_{i = 1}^n\sum_{j\neq i}(P^{\gamma_n}_{ij})^2$, notice that due to the symmetry of $Z'Z + \gamma_nI_{k_n}$, its smallest eigenvalue can be expressed as

where the first equality used that $a'a = 1$, and the second equality used that the smallest eigenvalue of $Z'Z$ is 0 when the rank $r_n$ of $Z$ is less than $k_n$.

We now have all the ingredients to define our new test.

Asymptotic properties of the RJAR test

We make the following assumptions to derive the limiting distribution of $RJAR_{\gamma^*_n}(\beta_0)$ under the null hypothesis in Equation (ref).

Assumption (ref) is a mild condition on the structural error terms, and allows for conditional heteroskedasticity. It is the same as in CMS. It is slightly less restrictive than the one in MS (who require finite sixth moments on the structural error terms), and slightly more restrictive than the one in BCCH (who require finite third moments).\footnote{It is difficult to allow for autocorrelated errors in our framework, since our proof relies on casting the RJAR statistic as a degenerate U-statistic, which requires that $\varepsilon_i$ be independent of $\varepsilon_j$ for $i\neq j$.}

Assumption (ref) is a weak technical assumption. It implies that both $k_n$ and $n$ diverge. It also allows for the sum of the number of IVs and the number of exogenous covariates to be larger than the number of observations, provided that the number of exogenous covariates that have been partialled out be sufficiently small (so that the rank of the matrix of partialled IVs continues to diverge).\footnote{{More formally, suppose that $W$ is an $n\times l_n$ matrix of covariates with rank $l_n$, and define $M_W := I_{n} - W(W'W)^{-1}W'$. Furthermore, let $\tilde{Z}$ be the $n\times k_n$ matrix of IVs prior to partialling out the controls $W$, and let $\tilde{r}_n := \text{rank}(\tilde{Z})$, such that $Z = M_W\tilde{Z}$ and $r_n := \text{rank}(Z) \leq \text{min}(n-l_n, \tilde{r}_n)$. Therefore, $l_n$ can diverge as $n\to \infty$ without contradicting $r_n\to \infty$, which means that a growing number of exogenous controls can be accommodated.}} This assumption is weaker than the restriction on the dimensionality in MS and CMS, who require $r_n = k_n$, and $k_n < n$ for each $n\in \mathbb N$, as well as $k_n\to \infty$ as $n\to \infty$. BCCH prove asymptotic size control of their Sup Score test under the assumption that $\log k_n = o(n^{1/3})$.

Assumption (ref) when $k_n<n$

Assumption (ref) is a high-level assumption on the number of IVs and their correlation structure. Assumption (ref) implies that $\gamma^*_n$ satisfies $\underset{n\to \infty}{\liminf} \frac{1}{r_n}\sum_{i = 1}^n\sum_{j\neq i}(P^{\gamma^*_n}_{ij})^2 > 0$. In the absence of exogenous covariates and when $k_n < n$ (as in CMS and MS), Assumption (ref) is weaker than the balanced-design assumption in CMS and MS which requires for $P := Z(Z'Z)^{-1}Z'$ that

for some $0 < \delta < 1$. This is stated formally in part 1 of the following Proposition.

Part 2. of Proposition (ref) implies that Assumption (ref) can be satisfied --- even when the balanced-design assumption is not --- as long as the number of diagonal elements of $P$ “close to 1” increases slower than $k_n$. Note also that if for some $\delta'\in(0,1)$ it holds that $\underset{1 \leq i \leq n}{\text{ max }}P_{ii} \leq 1 - \delta'$ for all $n\in\mathbb{N}$, then $\mathcal{A}_n(\delta')=\emptyset$. Thus, part 1. of Proposition (ref) actually follows from Part 2. However, we have chosen to keep part 1. as a separate statement to explicitly state that the balanced-design assumption implies Assumption (ref).

Assumption (ref) when $k_n\geq n$

In case $k_n\geq n$ the following proposition provides sufficient conditions for Assumption (ref) to be satisfied with probability one in case $Z$ is random.

Note that Assumption (ref) is satisfied in particular when the entries of $Z$ are i.i.d Gaussian. The latter was used in Hansen:2014ie to provide a set of sufficient conditions for their assumptions to be satisfied. The proof of Proposition (ref) also sheds further light on the exact value of $\eta$. To summarize, Propositions (ref) and (ref) show that Assumption (ref) is likely to be satisfied; this can be the case even in settings in which the assumptions underlying previous methods are violated.

If very small values of $\frac{1}{r_n}\sum_{i = 1}^n\sum_{j\neq i}(P^{\gamma_n}_{ij})^2$ are observed, then Assumption (ref) may be questionable. If this is the case, practitioners could make Assumption (ref) more plausible by suitably modifying the matrix of IVs, $Z$. This could include the dropping of IVs (either randomly or motivated by economic reasoning, but not informed by the in-sample correlation of the IVs with the endogenous variable), or the (iterative) removal of certain observations. If sufficient data is available to split the sample, a data-driven selection of IVs on one split of the sample for use on the other is also possible.

Asymptotic normality of the RJAR test statistic

We are now in a position to state the asymptotic distribution of the RJAR test under the null hypothesis.

Notice that we did not impose any assumption on the coefficients of the instruments $\Pi$ in the first-stage regression in Equation (ref). Therefore, the RJAR test is robust to arbitrarily weak identification.

Closest alternatives in the literature

The RJAR test combines two existing approaches in the literature. It uses ridge-regularisation as in CT to allow for $r_n < k_n$, and jackknifing as in CMS and MS to allow for arbitrary heteroskedasticty in the error terms.

The RJAR test is similar to the ridge-regularised AR test proposed by CT given by

where $P^{\theta} = Z(Z'Z + \theta I_k)^{-1}$ for some fixed scalar $\theta$ that does not depend on $n$, where $\theta \geq 0$ if $r_n = k_n$, and $\theta > 0$ if $r_n < k_n$. CT show that under the assumption of homoskedastic error terms, $AR_{CT}$ converges to an infinite sum of weighted $\chi^2_1$ distributions. Since the limiting distribution depends on an infinite number of unobserved weights, CT propose a bootstrap procedure to derive the critical values for $AR_{CT}$. The distinguishing features of the RJAR test are the data-driven and unique penalty parameter used to regularise the projection matrix (we find substantial sensitivity of the performance of $AR_{CT}$ to different values of $\theta$), the robustness to arbitrary heteroskedasticiy in the error terms, and its computational speed (it uses standard Normal asymptotic critical values instead of the bootstrap).

Robustness to arbitrary heteroskedasticity in the error terms is achieved through jackknifing, as in CMS and MS. The distinguishing feature of the RJAR test to the tests proposed in CMS and MS is the use of a ridge-regularised projection matrix $P^{\gamma_n}$, which makes the RJAR test applicable also when $k_n > r_n$, unlike the aforementioned two tests that use the standard least-squares projection matrix $Z'(Z'Z)^{-1}Z'$, and cannot be computed when $k_n > r_n$.

CMS assume $r_n = k_n < n$, and propose the jackknifed AR statistic given by

where $C := A - B$, $A := P + \Delta$, $B := (I_n - P)D(I_n - D)^{-1}(I_n - P)$, $\Delta := PD(I_n - D)^{-1}P - \frac 1 2 PD(I_n-D)^{-1} - \frac 1 2 D(I_n-D)^{-1}P$, and $D$ is the diagonal matrix containing the diagonal elements of $P$. $\hat{\Phi}_{CMS}(\beta_0) := \frac{2}{k_n}\sum_{i = 1}^n\*\sum_{j \neq i}C_{ij}^2\*e_i^2(\beta_0)\*e_j^2(\beta_0)$. Under the null hypothesis in Equation (ref), CMS show that $AR_{CMS}(\beta_0)$ converges to a Standard Normal distribution. The CLT underlying this result is a modified version of Lemma A2 of Chao:2012iz proposed in Bekker:2015jm.

MS also assume that $r_n = k_n < n$, and propose a different jackknifed AR statistic than CMS that can be obtained from $RJAR_{\gamma_n}(\beta_0)$ in Equation (ref) by setting $\gamma_n = 0$, and replacing $\hat{\Phi}_{\gamma_n}(\beta_0)$ with

where $M = I_n - P$, and $M_i$ is the $i^{\text{th}}$ row of $M$. The reason why the unregularised jackknifed AR test in MS uses $\hat{\Phi}_{MS}(\beta_0)$ instead of the variance estimator given in Equation (ref) evaluated at $\gamma_n = 0$, is because, according to their Theorem 4, and the discussion in Section 4.2.1, the former yields higher power than the latter. It follows that the unregularised version of our RJAR test, which arises when $\gamma^*_n = 0$ in Equation (ref), will be dominated by the jackknifed AR test of MS in terms of power, and practitioners may prefer the latter when $r_n = k_n$ and the balanced-design assumption is satisfied. This can have implications for the finite-sample performance of the jackknifed AR test of MS compared to the RJAR, which we investigate in Section (ref).

The only other existing test that allows for $r_n < k_n$ and arbitrary heteroskedasticty is the Sup Score test of BCCH. BCCH first standardise the IVs as in Equation (ref), and then propose the Sup Score statistic given by

BCCH propose to use the critical value $\nu_\alpha = c_{BCCH} \*\mathcal{Q}\left(1-\alpha/(2k_n)\right)$, $c_{BCCH} > 1$ and $\alpha \in (0, 1)$. They show that comparing their Sup Score statistic to this critical value yields a test of the null hypothesis in Equation (ref) that has asymptotic size less than or equal to $\alpha$. Being a supremum-norm test suggests that the BCCH Sup Score test will work well with a sparse first stage (i.e. where only a few elements of $\Pi$ are zero), but may have lower power than the RJAR test when the first stage is dense. This is verified in the simulations in Section (ref).

Simulations

We now investigate the size and power properties of the RJAR test and compare them to those of the tests proposed in CT, CMS, MS, and BCCH. We take our simulation setup from Hansen:2014ie who in turn take theirs from BCCH. The DGP is given by

for $i = 1, \dots, n = 100$ . The IVs $Z_i$ are independent and identically Gaussian with mean 0 and $\text{Var}\left[Z_{il} \right] = 0.3 $ and $\text{Corr}\left[ Z_{il}, Z_{im}\right] = 0.5^{|l-m|}$.\footnote{Simulations in Appendix (ref) show that the results remain qualitatively unchanged if uncorrelated Gaussian IVs are considered instead.} The error terms are given by

with $\sigma_\varepsilon^2=2$, $\sigma_v^2=1$ and $\sigma_{\varepsilon v} = 0.6\sigma_\varepsilon\sigma_v$.

$\pi = \zeta\*\kappa$, where $\kappa$ is a vector of zeros and ones that varies with the type of DGP considered (sparse or dense, as modelled below), and $\zeta$ is some scalar that ensures that for a given concentration parameter, $\mu^2$, the following relationship is satisfied:

This implies that

To illustrate how the sparsity structure of the first stage in Equation (ref) can affect the size and power of the studied tests, we consider both a sparse first stage and a dense first stage. Sparsity in the first stage is modelled by setting $\kappa = [\iota_5', 0_{k_n-5}' ]'$, where $\iota_q$ is a $q\times 1$ vector of ones, and $0_q$ is a $q\times 1$ vector of zeros. Denseness in the first stage is modelled by setting $\kappa = [\iota_{0.4k_n}', 0_{0.6k_n}']'$.

We consider $k_n = 30, 90, 190$. In the context of Assumption (ref), we search over values greater than 1 when choosing $\gamma^*_n$ in case $r_n<k_n$. For the case of 30 IVs, the RJAR test does not impose any regularisation ($\gamma^*_n = 0$). For the case of 90 IVs, $\gamma^*_n = 12.048$. We note that in the latter case $\sum_{i = 1}^n\sum_{j\neq i}(P^{\gamma^*_n}_{ij})^2 = 11.123 > 8.845 = \sum_{i = 1}^n\sum_{j\neq i}(P_{ij})^2$. This shows that even in the case where $r_n < n$, ridge regularisation can make Assumption (ref) strictly more plausible. For the case of 190 IVs, $\gamma^*_n = 109.187$.

The variance estimator of MS occasionally yields a negative value. These cases are conservatively interpreted as a failure to reject the null hypothesis. As recommended by BCCH, $c_{BCCH} = 1.1$. As in the simulation section in CT, we set $\theta = 0.05$. The number of Monte Carlo replications is 10,000.

Appendix (ref) provides additional simulation evidence on inference with heteroskedastic error terms. Appendix (ref) provides additional simulation evidence on projection-based inference.

Size

Figure (ref) shows the simulation results with a sparse first stage for tests of size 0.01 to 0.99, that is the rejection frequency under $H_0:\beta_0 = 1$. As far as the illustration of the tests' size properties is concerned, the dense first stage yields virtually the same results, and is hence omitted. Since all tests are robust to weak IVs, the rejection frequencies of the tests are not affected by the strength of identification. For the case of 30 IVs, the AR tests of CT, CMS, and MS and the RJAR test have correct size, while the BCCH Sup Score test is undersized.

For the case of 90 IVs, the AR test of CT and the RJAR test control size. The AR test of CMS appears to control size for common small nominal test sizes (e.g., 0.05 or 0.1). The AR test of MS appears to be generally oversized. For example, at nominal level 0.05, the rejection frequency of the test is 0.189.\footnote{The simulations in Appendix (ref) show that if $n$ and $k_n$ are increased to about 1,000 and 900, respectively, there is virtually no more size distortion.} The BCCH Sup Score test continues to be undersized.

For the case of 190 IVs, only the AR test of CT, the BCCH Sup Score test, and the RJAR test are feasible. As before, the BCCH Sup Score test is undersized, while the AR test of CT and the RJAR test have correct size.

Power

Figures (ref)--(ref) show the power of the tests when the number of IVs and the sparsity pattern of the first stage is varied. It is still the case that $H_0:\beta_0 = 1$.

For the case of 30 IVs (Figure (ref)), the AR tests of CT and CMS have similar power to the RJAR test, while MS is slightly more powerful than the RJAR test. The BCCH Sup Score test is less powerful than all other tests.

For the case of 90 sparse IVs (Figure (ref)), the RJAR test is slightly more powerful than the BCCH Sup Score test. The AR test of MS fails to control the size, while the AR tests of CT and CMS exhibit power properties substantially worse than those of the BCCH Sup Score test and the RJAR test. For the case of 90 dense IVs (Figure (ref)), the RJAR test is substantially more powerful than all other alternatives. For the case of 190 sparse and dense IVs (Figure (ref)), the RJAR test is more powerful than the BCCH Sup Score test.\footnote{The poor power of the BCCH Sup Score is likely a consequence of the Union Bound underlying their Lemma 5. We note that the poor power properties of the BCCH Sup Score are acknowledged in BCCH, and are what leads them to recommend using their main estimator (Post-LASSO two-stage least squares) whenever weak identification is not a concern, and the first stage is sufficiently sparse.} The RJAR test is also more powerful than the AR test of CT. Thus, for all the DGPs that are considered here, the RJAR test is as powerful as existing methods whenever these are applicable, and sometimes much more powerful.

Empirical Application

We consider an empirical application based on Card:2009hj. The coefficient of interest is given by $\beta_s$ in the following model:

where $y_{is}$ is the difference between residual log wages for immigrant and native men in skill group $s$ in city $i$,\footnote{As discussed in Card:2009hj, residual wages are wages once observed characteristics of the entire US workforce are controlled for.} $X_{is}$ is the log ratio of immigrant to native hours worked in skill group $s$ of both men and women in city $i$, and $W_i$ is a vector of city-level controls with coefficient vector $\delta_s$, and $\varepsilon_{is}$ is the structural error. In the context of the production function specified in Card:2009hj, $\beta_s$ can be interpreted as the (negative) inverse elasticity of substitution between immigrants and natives in the US in their respective skill group. As in Card:2009hj, we consider two skill groups $s = h, c$ (high school or college equivalent) separately.

Card:2009hj raises the concern that unobserved factors in a city may lead to both higher wages and higher employment levels of immigrants relative to natives, causing $X_{is}$ to be endogenous. Card:2009hj proposes to use the ratio of the number of immigrants from country $l$ in city $i$ to the total number of immigrants from foreign country $l$ in the US as an IV. The rationale for these IVs is that existing immigrant enclaves are likely to attract additional immigrant labour through social and cultural channels unrelated to labour market outcomes. We consider two sets of IVs. First, we consider the original setup of Card:2009hj, using as IVs the $k_n = 38$ different countries of origin of the immigrants. Second, motivated by the saturation approach of Blandhol2022, we consider the setup where these 38 original IVs are interacted with the $q = 9$ available controls (including a constant). This yields $k_n = 342$ IVs. In both cases, the number of observations (i.e., the number of cities) is $n = 124$.\footnote{We note that the controls and the IVs are at the city level, and hence the same for the applications to high-school workers and college workers. Therefore, the projection matrix and the ridge-regularised projection matrix will also be the same across these two skill groups.}

We construct (weak-identification robust) confidence sets for $\beta_s$ by inverting the AR tests of CT, CMS and MS, the Sup Score test, and the RJAR test. Thus, the 95% confidence set for any test is obtained as the collection of $\beta_{s,0}$ for which that test does not reject the null at 5% level of significance. As in the simulation exercise in Section (ref), we search over values greater than 1 when choosing $\gamma^*_n$ in case $r_n<k_n$. Again $c_{BCCH} = 1.1$ and $\theta = 0.05$. The number of boostrap replications for the AR test of CT is set to 2,500. A grid of 100 values for $\beta_{s,0}$ is used for $s = h, c$. Data is taken from a single cross section, as made available by GoldsmithPinkham:2020hm.

Figure (ref) shows the confidence sets when $k_n = 38$ for high-school workers and college workers. We find that $\gamma^*_n = 0$, implying that no regularisation is needed. This is in line with our simulations in Section (ref), where we found that regularisation was not needed to maximise the sum in Assumption (ref) when $k_n/n = 0.3$. Furthermore, $\underset{1 \leq i \leq n}{\text{ max }}P_{ii} = 0.944$, and $r^{-1}_n\sum_{i = 1}^n\sum_{j\neq i}(P^{\gamma^*_n}_{ij})^2 = 0.513$. We also point out that only three diagonal entries of $P$ are larger than $0.9$, which suggests that Assumption (ref) is reasonably satisfied by part 2. of Proposition (ref). The 95% confidence sets for each test are given by all the points below the grey horizontal line. The confidence sets for both skill groups broadly confirm the results in Card:2009hj. We find that the confidence sets for high-school workers is smallest for the jackknifed AR statistic of CMS, whereas the BCCH Sup Score test yields the smallest confidence interval for the application to college workers. For both cases, the AR test of CT yields the largest confidence interval. Based on the power results in Section (ref), this suggests a very sparse first stage for college workers, i.e., a few nationalities being highly predictive of inflows of immigrant labour.

Figure (ref) shows the confidence sets when $k_n = 342$ for high-school workers and college workers, respectively. Since $r_n = n - q = 124-9 = 115 < 342 = k_n$, the jackknifed AR statistics of CMS and MS are not applicable. We find that $\gamma^*_n = 5.299$ and $r^{-1}_n\sum_{i = 1}^n\sum_{j\neq i}(P^{\gamma^*_n}_{ij})^2 = 0.106$. The 95% confidence sets obtained by inverting the AR test of CT are empty for both $\beta_{h}$ and $\beta_{c}$. This could be due to heteroskedasticity in the error terms. In line with the simulation results on power reported in Section (ref), the RJAR test yields smaller confidence intervals than the BCCH Sup Score test. The qualitative conclusions with respect to the case of 38 IVs remain unchanged.

Conclusion

We contributed to the literature on (very) many IVs in the cross-sectional linear IV model by proposing a new, ridge-regularised jackknifed AR test. Our test compares favourably with existing methods in the literature both theoretically, by allowing for high-dimensional IVs and weakening a common assumption on the IVs' projection matrix, and practically, by having correct asymptotic size and displaying favourable power properties even when the number of IVs approaches or exceeds the number of observations.

\numberwithin{equation}{section} \setcounter{section}{0}

Proofs

Throughout this appendix, $C>0$ denotes a universal constant that can change across lines. CSHNW refers to Chao:2012iz from where we also borrow our summation conventions. We also define $\Phi_{\gamma_n} := \frac{2}{r_n}\sum_{i = 1}^n\sum_{j \neq i}(P_{ij}^{\gamma_n})^2\mathbb{E}[\varepsilon_i^2]\mathbb{E}[\varepsilon_j^2]$. Since all the proofs in this appendix are under the null hypothesis, we write $\varepsilon_i$ and $\hat{\Phi}_{\gamma_n}$ for $e_i(\beta_0)$ and $\hat{\Phi}_{\gamma_n}(\beta_0)$, respectively.

Furthermore, we note that Assumption (ref) implies that

for some $c > 0$ and $n$ sufficiently large.

Our strategy for deriving the limiting distribution of our RJAR test is as follows. Since $P^{\gamma_n}$ is generally not idempotent for $\gamma_n \geq 0$, we cannot rely on standard properties of idempotent matrices. Therefore, we first derive a series of properties for the ridge-regularised projection matrix $P^{\gamma_n}$ that hold for any $\gamma_n\geq 0$ when $r_n = k_n$ and $\gamma_n >0$ when $r_n < k_n$. These properties are derived using a singular value decomposition of the matrix of IVs $Z$, and are collected in Lemmas (ref) and (ref). These Lemmas play a similar role to Lemma B1 and Lemma B2 in Chao:2014. The proof of Theorem (ref) derives the asymptotic distribution of our RJAR statistic under the assumption that $\Phi_{\gamma_n}$ is known. This is achieved by casting the RJAR statistic as a degenerate U statistic with variable kernel, and verifying the conditions of a Martingale CLT. A benefit of deriving our proofs from the bottom up (and not relying on the augmentation approach in Hansen:2014ie) is that we do not require that all diagonal elements of $P^{\gamma_n}$ be bounded away from 1. Lemma (ref) shows that our proposed estimator of $\Phi_{\gamma_n}$ is consistent. Taken together with Theorem (ref) and Assumption (ref), Lemma (ref) implies that our RJAR test is asymptotically valid. Finally, Lemma (ref) proves the existence of $\gamma^*_n$, and the proof of Proposition (ref) shows that Assumption 3 is weaker than the balanced-design assumption in CMS and MS.

The following singular value decomposition of the $n\times k_n$ matrix of instruments $Z$ of rank $r_n$ will be used frequently (see, e.g., Lutkepohl:1996uz):

where $U$ is an $n\times n$ matrix such that $U'U=UU'=I_n$, $Q$ is a $k_n\times k_n$ matrix such that $Q'Q = QQ' = I_{k_n}$, and $S$ is the $n\times k_n$ matrix given by

where $D$ is the diagonal $r_n\times r_n$ matrix containing the singular values of $Z$. Hence, one can write

where $\tilde D =S(S'S + \gamma_n I_{k_n})^{-1}S'$ is the diagonal $n\times n$ matrix with diagonal entries given by given by $\tilde{D}_{ll} = \frac{D_{ll}^2}{D_{ll}^2 + \gamma_n }\leq 1$ for $l = 1, \dots, r_n$, and zero otherwise. Note that the diagonal entries of $\tilde D$ are also the the eigenvalues of $P^{\gamma_n}$.

Lemmas

The following lemma collects some properties of $P^{\gamma_n}$. Recall that $Z$ has rank $r_n$.

Proof of Proposition (ref)

As observed after the statement of Proposition (ref), it suffices to prove its part 2. Thus, assume that there exists a $\delta' \in (0, 1)$ such that $\frac{1}{k_n}|\mathcal{A}_n(\delta')| \to 0$. Notice that the sum in Assumption (ref) with $\gamma_n = 0$ can be written as

where the first equality uses the idempotency and symmetry of $P$, and the second equality uses the fact that the trace of $P$ is equal to its rank $k_n$. Thus, if $\frac{1}{k_n}|\mathcal{A}_n(\delta')| \to 0$,

and hence Assumption (ref) is satisfied.

\qed

Proof of Proposition (ref)

The following preparatory lemma is likely well-known but we state it here for ease of reference. Its proof is a simple consequence of the Marchenko-Pastur theorem.\footnote{We thank Alexei Onatski for suggesting this.} Denote by $\lambda_{n,1}\geq \hdots\geq \lambda_{n,n}\geq 0$ the eigenvalues of $\frac{ZZ'}{k_n}$, where $\lambda_{n,i}=\frac{D^2_{ii}}{k_n}$ in our earlier notation on p. (ref), and by $F_n$ the empirical distribution of these.

Proof of Theorem (ref)

We begin by showing that

converges in distribution to a standard normal {for all sequences of $\gamma_n$ that satisfy Assumption (ref)}.\footnote{We do not directly invoke Lemma 2 in Hansen:2014ie because it would require strengthening the assumptions used in this paper.} To this end, define $\mathcal{U}_n := 2\sum_{i = 2}^n\sum_{j = 1}^{i-1}P^{\gamma_n}_{ij}\varepsilon_i\varepsilon_j$, $s_n^2: = \mathbb{E}[\mathcal{U}_n^2]$ and note that by the symmetry of $P^{\gamma_n}$, $RJAR^*_{\gamma_n}$ can be written as

We proceed by establishing that $s_n^{-1}\mathcal{U}_n \overset{d}{\to} \mathcal{N}[0, 1]$ as $n\to\infty$. This, in turn, follows from Hall:1980uh) upon verifying that i) for all $\epsilon>0$

where $Y_{ni} = 2\sum_{j = 1}^{i-1}P^{\gamma_n}_{ij}\varepsilon_i\varepsilon_j$, and ii)

where $\mathcal{V}_n^2 = \sum_{i = 2}^n\mathbb{E}[Y^2_{ni}|\varepsilon_1, \dots, \varepsilon_{i-1}]$. Consider first the condition in Equation (ref) and write

so that, upon using that $\mathcal{U}_n=\sum_{i=2}^n Y_{ni}$ and $\mathbb{E}[Y_{ni}Y_{nj}]=0$ for $i\neq j$, one gets

Furthermore,

so that

It follows that

where the first inequality follows from Assumption (ref). The second inequality follows from Lemma (ref) (ref). The third inequality follows from Lemma (ref) (ref). The fourth inequality follows from Equation (ref). The fifth inequality follows from Lemma (ref) (ref). The limit holds by Assumption (ref) and Assumption (ref). The condition in Equation (ref) follows from Equation (ref).

In order to verify the convergence in Equation (ref) it suffices to show that

Since $\mathcal{V}_n^2 = \sum_{i = 2}^n\mathbb{E}[Y^2_{ni}|\varepsilon_1, \dots, \varepsilon_{i-1}]$ it follows that

the last equality following from Equation (ref). It remains to be verified that

To this end, observe that

such that

For $i \leq j$, tedious but straightforward calculations included for completeness yield

where $\text{Var}[\varepsilon_h^2] \coloneqq \mathbb{E}[\varepsilon_h^4] - \mathbb{E}[\varepsilon_h^2]\mathbb{E}[\varepsilon_h^2]$.

Defining $q \coloneqq \text{min}\{i,j\}$, the above display in turn implies

In order to establish the convergence in Equation (ref), it is sufficient to show that $\frac{\mathfrak{A}_n + \mathfrak{B}_n}{s_n^4} \to 0$. To this end, note that

The first inequality follows from Assumption (ref). The second inequality follows from Lemma (ref) (ref). The third inequality follows from the symmetry of $P^{\gamma_n}$. The fifth inequality follows from Lemma (ref). Next,

the first inequality following from Assumption (ref) and the second from Lemma (ref) (ref). Thus,

where the second inequality follows from Equation (ref) and the convergence from Assumption (ref) and Assumption (ref). Equation (ref) verifies the condition in Equation (ref), which in turn implies that the condition condition in Equation (ref) holds.

Having verified the conditions in Equation (ref) and Equation (ref) we conclude that $RJAR^*_{\gamma_n} \overset{d}{\to}\mathcal{N}[0, 1]$. By Lemma (ref) $|\hat{\Phi}_{\gamma_n} - \Phi_{\gamma_n}|\overset{p}{\to} 0$ and hence also $\hat{\Phi}_{\gamma_n}/\Phi_{\gamma_n} \overset{p}{\to} 1$ under Assumptions (ref) and (ref). Hence, under the null hypothesis, the continuous mapping theorem implies that $RJAR_{\gamma_n}(\beta_0) \overset{d}{\to} \mathcal{N}[0, 1]$.

\qed