Enhanced power enhancements for testing many moment equalities: Beyond the $2$- and $\infty$-norm

Enhanced power enhancements for testing many moment equalities: Beyond the $2$- and $$-norm

\and

}

Introduction

In the era of big data, target parameters in statistical models are often (partially) identified by a large number of moment equalities. Then, one frequently wishes to test whether a candidate (structural) parameter $\bm{\beta}^*_n$ is an element of the identified set, i.e., whether $\bm{\beta}^*_n$ satisfies $\mathbb{E} h_n(\bm{X},\bm{\beta}^*_n)=\bm{0}_d$.\footnote{Here $\bm{\beta}^*_n$ may or may not be point-identified, cf., e.g., molinari2020microeconometrics for many examples of how lack of point-identification may arise.} An important example which can be cast within this framework and that has received much recent attention is testing restrictions on coefficients in regression models in the presence of endogenous regressors and many (weak) instruments; recent references include mikusheva2020inference, matsushita2022jackknife, boot2023identification, and dovi2022ridge. Indeed, this is our running example and further details and references can be found in Section (ref). Testing whether a treatment has an effect on one of many outcome variables or one of several groups of individuals is another example of high practical relevance, cf. Section (ref). More generally, true parameter(s) in $M$-, $Z$-, and GMM-estimation problems satisfy such moment restrictions, and hypotheses on the mean vector(s) or covariance matrices in classic one- and two-sample testing problems can be accommodated too.

Given a sample $\bm{X}_{1,n}, \hdots, \bm{X}_{n,n}$ of random variables with the same marginal distribution as $\bm{X}$, denote the (scaled) empirical counterpart of the population moments $\mathbb{E} h_n(\bm{X},\bm{\beta}^*_n)$ by

and let $\hat{\bm{\Sigma}}_n(\bm{\beta}^*_n)$ be an estimator of the covariance matrix $\bm{\Sigma}_n(\bm{\beta}^*_n)$ of $\bm{H}_n(\bm{\beta}^*_n)$. It is then natural to base a test of $\mathbb{E} h_n(\bm{X},\bm{\beta}^*_n)=\bm{0}_d$ on the distance of $\hat{\bm{\Sigma}}^{-1/2}_n(\bm{\beta}^*_n)\bm{H}_{n}(\bm{\beta}^*_n)$ from the origin. To measure this distance, one most commonly makes use of a $p$-norm, which then requires the choice of the exponent $p$. Most tests used in practice are based on $p=2$ or $p=\infty$ and it is well-understood that tests based on the former are relatively powerful against “dense” alternatives whereas tests based on the latter are relatively powerful against “sparse” alternatives, cf., e.g., ingster2003nonparametric. To construct a test that is simultaneously powerful against dense as well as many sparse alternatives, fan2015power combined tests based on the $2$- and $\infty$-norm via their power enhancement principle. The idea of combining the $2$- and $\infty$-norm based tests or a finite number of $p$-norm based tests in order to construct a more powerful test has since gained considerable popularity, cf. xu2016adaptive, yang2017weighted, tang2018testing, kp, liu2019statistical, jammalamadaka2020sobolev, he2021asymptotically, feng2020max, juodis2022incidental, zhang2021adaptive, yu2023power, enhanceIV, ge2022dynamic, li2024power, yu2020fisher, yu2024power, li2024detection.

Despite the success of the power enhancement principle and related combination procedures, sparse and dense alternatives are merely two (conceptually useful) “endpoints" between which a continuum of “semi-sparse” alternatives exist. This continuum of structures is mirrored by a continuum of $p$-norms, $p\in[2,\infty]$, between the two extremes $p=2$ and $p=\infty$ that most tests are based on. In the clean but restrictive testbed of the Gaussian sequence model, kp2021consistency showed that there exist semi-sparse alternatives against which tests based on the $2$- and $\infty$-norm are inconsistent, but against which tests based on any $p\in(2,\infty)$ are consistent. Thus, it is important to harness the power from all $p$-norms, $p\in[2,\infty]$, which kp2021consistency exploited to construct a test that is “dominant” --- in the Gaussian sequence model --- in the following sense: If there exists some $p$ for which the corresponding $p$-norm based test is consistent against a given alternative, then so is their test. In particular, as Figure (ref) illustrates for a slight variation $\psi_d$ (which is more convenient for our purposes) of their test, $\psi_d$ is

In the present paper, we leverage the findings of kp2021consistency to construct a test simultaneously dominating all $p$-norm based tests with $p\in[2,\infty]$ in terms of consistency for testing $H_{0,n}:\mathbb{E} h_n(\bm{X},\bm{\beta}^*_n)=\bm{0}_d$.\footnote{The results in kp2021consistency show that already in the Gaussian sequence model tests based on $p$-norms with $p \leq 2$ are dominated (in terms of consistency) by the $2$-norm based test. For simplicity, we therefore focus on $p \in [2, \infty]$ in the present paper.} This requires rigorous high-dimensional Gaussian approximations to justify that results obtained in the stylized Gaussian sequence model carry over to more complex empirically relevant models. Indeed, a large and vibrant literature on high-dimensional Gaussian approximations over hyperrectangles was sparked by the work of chernozhukov2013gaussian and is surveyed in chernozhukov2023high. Such approximations over hyperrectangles are suitable for studying tests based on $p=\infty$. However, since we study the full family of $p$-norm based tests and construct a test dominating all of its members, we cannot rely on these approximations, because the rejection regions of $p$-norm based tests, albeit convex, are no hyperrectangles. Similarly, central limit theorems for quadratic forms (related to 2-norm type tests) do not suffice for our purpose. Gaussian approximations over convex sets have received considerable attention, cf. nagaev2006estimate, senatov1981uniform, gotze1991rate, bentkus2003dependence, bentkus2005lyapunov, fang2020large.

Summary of results

We now discuss our contributions in more detail. In Theorem (ref) we first exhibit high-level conditions under which the characterization of the consistency properties of $p$-norm based tests in the Gaussian sequence model from kp2021consistency carries over to the setting considered in the present paper. We then establish in Theorem (ref) that these high-level conditions are satisfied under

In particular, although it is not our focus, dependent data can be accommodated. We also provide a catalogue of primitive sufficient conditions for (i) and (ii) in the i.i.d. case and discuss their implication on the allowed growth rate of $d$ in dependence of $n$. To this end, because it is often not realistic to impose that the entries of $h_{n}(\bm{X}_{i,n},\bm{\beta}^*_n)$ have light tails, we focus on a setting wherein (essentially) only four moments exist. Here, we need $d=o(n^{2/5})$, cf. Corollary (ref), and we emphasize that this rate cannot be improved by much, even if one restricts attention to tests based on $p=\infty$ only, see also kock2023moment. Clearly, $d$ larger than $n$ is ruled out. Nevertheless, many empirically relevant settings involving a large $d$ that is still smaller than $n$ are covered. For example, angrist2022machine based on the data from angrist1991does consider a setting with $n=329{,}509$ and up to $d=1{,}530$ instruments.\footnote{Similarly, register-based data sets, such as those provided by Statistics Denmark, allow one to access data on entire populations yielding very large $n$ (in the millions) and thus allowing $d$ to be large as well.} Alternatively, one can verify (i) and (ii) under stronger moment assumptions or by imposing structure on $\bm{\Sigma}_n(\bm{\beta}^*_n)$ to allow $d$ to grow faster than $n^{2/5}$. Furthermore, in Section (ref), we introduce “projection-based” tests based on a sample-splitting device that overcome dimensionality constraints completely (without sacrificing asymptotic size).

On a technical level, the conditions (i) and (ii) allow us to establish a Gaussian approximation for $\hat{\bm{\Sigma}}^{-1/2}_n(\bm{\beta}^*_n)\bm{H}_{n}(\bm{\beta}^*_n)$ over convex sets when the “noncentrality term”

is sufficiently close to $\bm{0}_d$, cf. Lemma (ref). This enables us to deduce the power properties of tests based on the statistics $\enVert[1]{\hat{\bm{\Sigma}}^{-1/2}_n\bm{H}_{n}(\bm{\beta}^*_n)}_p$ for any $p\in[2,\infty]$ from those of $p$-norm based tests in the Gaussian sequence model, which was analyzed in kp2021consistency. However, the case of “large” $\bm{\theta}_n(\bm{\beta}^*_n)$ must be handled by a separate proof strategy tailored to this regime.

We find that a $p$-norm based test is consistent for $p\in[2,\infty)$ if and only if $$d^{-1/2}\sum_{j=1}^d \sbr[1]{\theta_{j,n}^2(\bm{\beta}^*_n)\vee |\theta_{j,n}(\bm{\beta}^*_n)|^p}\to\infty,$$ and obtain a structurally similar, but somewhat more complex, necessary and sufficient condition for consistency of the $\infty$-norm based test. Thus, apart from the case of $p=2$, consistency of a $p$-norm based test is not solely determined by the $p$-norm of $\bm{\theta}_n(\bm{\beta}^*_n)$. Furthermore, the characterization reveals the following monotonicity: For $2\leq p<q<\infty$ tests based on the $q$-norm are consistent against more alternatives than tests based on the $p$-norm. This monotonicity does not extend to $p=\infty$.\footnote{Although sufficient, yet not necessary, conditions are known for consistency of tests based on $p=\infty$ (based on maximin rates of testing, cf. ingster2003nonparametric), we are not aware of any full characterization providing necessary and sufficient conditions beyond the Gaussian setting considered in kp2021consistency. Thus, our characterization also sheds new light on tests based on $p=\infty$, which are frequently employed.}

Based on the monotonicity property, we then construct a test $\psi_n(\bm{\beta}^*_n)$, say, that harnesses the strength of all $p$-norms with exponents in $[2, \infty]$ in the sense that it has asymptotic size $\alpha \in (0, 1)$ and is consistent against a sequence of alternatives whenever some $p$-norm based test with $p \in [2, \infty]$ and asymptotic size in $(0, 1)$ is consistent against that sequence of alternatives. Thus, $\psi_n(\bm{\beta}^*_n)$ simultaneously dominates all $p$-norm based tests in terms of consistency. In particular, it goes beyond current implementations of the power enhancement principle, which typically target consistency against alternatives that either tests based on the $2$- or $\infty$-norm are consistent against, i.e., only two norms are targeted.

Finally, we illustrate the scope of our results in the running example of the linear instrumental variable model with many (weak) instruments. Here our general construction yields a test that is powerful irrespectively of whether the first-stage is sparse, dense, or semi-sparse.

The problem

We observe realizations of random variables $\bm{X}_{1,n},\hdots,\bm{X}_{n,n}$, each of which is defined on an underlying probability space $(\Omega,\mathcal{F},\mathbb{P})$ and takes its values in the measurable space $(\mathcal{X}_{n},\mathcal{A}_n)$. We wish to test whether a candidate structural parameter $\bm{\beta}^*_n \in \mathbf{B}_n \neq \emptyset$ satisfies the moment equalities

for $h_{j,n}:\mathcal{X}_{n}\times \mathbf{B}_n\to\mathbb{R}$ with $h_{j,n}(\cdot,\bm{\beta}^*_n)$ being $\mathbb{P}$-integrable and $\mathbb{E}$ denoting the expectation with respect to $\mathbb{P}$. The $\bm{X}_{i,n}$ are assumed to be identically distributed throughout, but are allowed to be dependent (although our detailed examples are for independent variables). In this paper we are interested in a setting wherein $d(n)\to\infty$ as the sample size $n\to\infty$. That is, there are “many" moment equalities. To conserve on notation, we write $d=d(n)$ whenever this causes no confusion. For our general results, no structure needs to be imposed on the parameter space $\mathbf{B}_n$. Of course, the complexity of constructing confidence sets for the parameters via test inversion depends on the size and structure of $\mathbf{B}_n$.

For $h_{n} = (h_{1,n},\hdots,h_{d,n})'$ and $\bm{0}_{d}$ denoting the $d\times 1$ vector of zeros, the requirement in (ref) is more conveniently expressed as

We thus consider the testing problem

and are interested in asymptotic size and power properties of tests in situations in which $d$ increases with $n$. Finally, we stress that since the tests to be constructed build on plugging in a candidate $\bm{\beta}^*_n$, no assumptions need to be made on (the degree of) identification of the parameter(s) satisfying (ref). Thus, the tests are trivially robust to (weak) identification problems.

Examples

As pointed out already in hansen1982generalized, moment conditions as in (ref) frequently arise in economic models as first-order conditions to an agent's optimization problem. Furthermore, as mentiond in the introduction, the true population parameters in $M$- and $Z$-estimation problems, such as (non-linear) least squares and maximum likelihood satisfy first-order conditions of the form (ref). Our running example will be inference in the presence of many (weak) instruments, and since the 2-norm based test there is just the Anderson-Rubin test, we begin by recalling how this fits into our general framework. We also provide details for treatment effect testing examples. For clarity, we assume i.i.d. sampling in the examples below.

Inference in the presence of many (weak) instruments

Consider the classic linear instrumental variable (IV) setting in which $\bm{X}_{i,n}=(y_{i,n},\bm{Y}'_{i,n},\bm{z}'_{i,n})'$ for $y_{i,n}\in\mathbb{R}$ an outcome of interest, $\bm{Y}_{i,n}\in\mathbb{R}^k$ a vector of endogenous explanatory variables and $\bm{z}_{i,n}\in\mathbb{R}^{d}$ a vector of instruments. Thus, $\mathcal{X}_n=\mathbb{R}^{1+k+d}$, $\mathbf{B}_n=\mathbb{R}^k$ and $h_{n}(\bm{X}_{1,n},\bm{\beta}^*_n)=(y_{1,n}-\bm{Y}'_{1,n} \bm{\beta}^*_n)\bm{z}_{1,n}$. In this context, we wish to test whether a candidate parameter vector $\bm{\beta}_n^*$ lies in the identified set, i.e., whether

Note that the number of moment equalities equals the number of instruments $d$. This can be large even for $k$ fixed. Inference on $\bm{\beta}^*_n$ in the presence of many (weak) instruments has received considerable recent attention as witnessed by, e.g., the works of andrews2007testing, anatolyev2011specification, belloni2012sparse, tchuente2016regularization, kaffo2017bootstrap, crudu2021inference, mikusheva2021many, matsushita2021second, mikusheva2020inference, matsushita2022jackknife, dovi2022ridge, boot2023identification. There are several important practical reasons for this interest. First, when identification is weak, researchers may seek to obtain more precise inference by using a large number of instruments in order to capture more of the exogenous variation in the endogenous covariates. Second, approaches such as the granular IV approach of gabaix2020granular, the saturation approach of blandhol2022tsls or Mendelian Randomization (davey2003mendelian), the latter using genetic variation as instruments, can lead to situations where the number of instruments is large compared to the sample size. The same is true for technical instruments such as transformations or interactions or empirical strategies such as “judge designs,” cf. miksun23. However, as there is no guarantee that even a large number of instruments is jointly informative, it is important to develop tests that remain valid under no or weak identification.

Our general results allow us to construct a test that strictly dominates the Anderson-Rubin test, the sup-score test, and power enhancement combinations of these in terms of consistency properties when $d\to\infty$, cf. Section (ref).

The examples in the following two subsections revisit Examples 1 and 2 in belloni2018high and both consider testing for treatment effects in a randomized controlled trial.

Randomized controlled trial with many outcomes

Consider a randomized controlled trial in which $\bm{X}_{i,n}=(D_{i,n},\bm{Y}'_{i,n}(0),\bm{Y}'_{i,n}(1))'$ for $D_{i,n}\in\cbr[0]{0,1}$ indicating whether individual $i$ is assigned to the treatment ($D_{i,n}=1$) or not ($D_{i,n}=0$). For each individual we observe a large number, $d$, of outcome variables and denote this by $\bm{Y}_{i,n}(1) = (Y_{i1,n}(1), \hdots, Y_{id,n}(1))' \in \mathbb{R}^{d}$ in case individual $i$ is treated and $\bm{Y}_{i,n}(0) = (Y_{i1,n}(0), \hdots, Y_{id,n}(0))' \in\mathbb{R}^{d}$ if not. This framework of many outcome variables is relevant, because a researcher may, for example, be interested in whether a medication has an effect on any of a large number of health outcomes (blood pressure, weight, heart rate, ...) rather than only monitoring the effect on a single of these.\footnote{Although a treatment may be targeted to affect a single outcome of interest, it may nevertheless be important to scan for potential side effects on other outcome variables.}

Suppose that $\mathbb{P}(D_{1,n}=1)=\pi$ for $\pi\in(0,1)$ a known probability of being assigned to the treatment and that the treatments are assigned independently of the potential outcomes, that is $D_{1,n} \perp \!\!\! \perp (\bm{Y}_{1,n}(0),\bm{Y}_{1,n}(1))$. Furthermore, $\mathcal{X}_n=\mathbb{R}^{1+2d}$. The treatment does not have an effect on any of the outcomes being measured if $\mathbb{E} \bm{Y}_{1,n}(1)=\mathbb{E} \bm{Y}_{1,n}(0)$. More generally, for $\bm{\beta}^*_n = (\beta^*_{n,1}, \hdots, \beta^*_{n,d})' \in \mathbb{R}^d$ a vector of coordinate-wise effects, we are interested in testing whether $\mathbb{E} \bm{Y}_{1,n}(1) - \mathbb{E} \bm{Y}_{1,n}(0)=\bm{\beta}^*_n$. Because $D_{1,n} \perp \!\!\! \perp (\bm{Y}_{1,n}(0),\bm{Y}_{1,n}(1))$ this is equivalent to

where $\bm{Y}_{1,n}=D_{1,n}\bm{Y}_{1,n}(1)+(1-D_{1,n})\bm{Y}_{1,n}(0)$ is the observed outcome. This is a testing problem of the form in (ref) with $h_{n}(\bm{X}_{1,n},\bm{\beta}^*_n)=\frac{D_{1,n}\bm{Y}_{1,n}}{\pi}-\frac{(1-D_{1,n})\bm{Y}_{1,n}}{1-\pi} - \bm{\beta}^*_n$.

Randomized controlled trials with many groups

Similar to the previous example of Section (ref), consider a randomized controlled trial. However, let there be only one outcome variable for each individual. Suppose, however, that each subject falls into one of $d$ groups and one wants to test whether a treatment has the hypothesized effect (e.g., no effect) in every one of these groups. Assuming for simplicity that there are $n$ observations available in each group, one can thus again write the problem in the form of two $d$-dimensional outcome vectors $\bm{Y}_{i,n}(0)$ and $\bm{Y}_{i,n}(1)$ as in the previous example ($\bm{Y}_{i,n}(0)$ is the vector of the stacked outcomes of the $i$-th subjects in each group under the no treatment condition, whereas $\bm{Y}_{i,n}(1)$ is the vector of the stacked outcomes of the $i$-th subjects in each group under the treatment condition). The null hypotheses of the treatment having effect $\beta_{n,j}^*$ in group $j$ for $j = 1, \hdots, d$ then again amounts to (ref).

As pointed out in, e.g., belloni2018high, the project STAR investigating the effect of class size on student learning falls within the framework of the present example as many background characteristics were collected on pupils and teachers. For example gender or race. For pupils, their month of birth was also recorded. Identifying each group with a particular combination of covariates, this results in a large number of groups.

Characterization of consistency properties of $p$-norm based tests

Recall from the introduction that given $\bm{\beta}_n^* \in \mathbf{B}_n$, we write

and let $\hat{\bm{\Sigma}}_n(\bm{\beta}^*_n)$ be a positive semidefinite and symmetric estimator of the covariance matrix $\bm{\Sigma}_n(\bm{\beta}^*_n)$ of $\bm{H}_n(\bm{\beta}^*_n)$, which we assume to exist.

A common way of measuring the empirical evidence against the null of $\mathbb{E} h_{n}(\bm{X}_{1,n},\bm{\beta}^*_n)=\bm{0}_{d}$ is based on the the Euclidean norm of $\hat{\bm{\Sigma}}^{-1/2}_n(\bm{\beta}^*_n)\bm{H}_{n}(\bm{\beta}^*_n)$, that is, one rejects $H_{0,n}$ null whenever

exceeds a critical value $\kappa_{n,2}$ chosen to ensure that the resulting test has a desired (asymptotic) size $\alpha\in(0,1)$, and where $\enVert[0]{\bm{x}}_2=\sqrt{\sum_{i=1}^dx_i^2}$ for $\bm{x}\in\mathbb{R}^d$.\footnote{In order to avoid taking a stance on $\hat{\bm{\Sigma}}_n(\bm{\beta}^*_n)$ potentially not being invertible, we denote by $\bm{A}^{-1}$ the Moore-Penrose pseudoinverse for any matrix $\bm{A}$ and define $\bm{A}^{-1/2}:=\del[1]{\bm{A}^{1/2}}^{-1}$ in case $\bm{A}$ is symmetric and positive semidefinite, and where $\bm{A}^{1/2}$ denotes the unique symmetric positive semidefinite square root of $\bm{A}$. Recall that the Moore-Penrose inverse is identical to the regular matrix inverse whenever the latter exists.} For $\bm{x} = (x_1, \hdots, x_d)' \in \mathbb{R}^d$ and $p \in [2, \infty]$, define the $p$-norm

and introduce, analogously to $S_{n,2}(\bm{\beta}^*_n)$ in (ref) but based on the $p$-norm, the family of test statistics

For sequences of critical values $(\kappa_{n,p})_{n\in\mathbb{N}}$ that guarantee a desired asymptotic size, we shall now study consistency properties of the $p$-norm based tests

and coverage properties of the associated confidence sets

when $d\to\infty$ as $n \to \infty$. Define $$\mathbf{B}^* := \bigtimes_{n = 1}^{\infty} \mathbf{B}_n,$$ i.e., the set of possible sequences of parameters $\bm{\beta}_n$ for $n \in \mathbb{N}$, along which power or coverage properties of tests for the sequence of testing problems in (ref) can be studied asymptotically in the current setup. In particular, we write $\bm{\beta}^*=(\bm{\beta}_1^*,\bm{\beta}_2^*,\hdots)$. Let $$\mathbf{B}^{(0)}=\cbr[1]{\bm{\beta} = (\bm{\beta}_1, \bm{\beta}_2, \hdots) \in\mathbf{B}^*:\mathbb{E} h_{n}(\bm{X}_{1,n},\bm{\beta}_n)=\bm{0}_{d}\text{ for every }n\in\mathbb{N}} \subseteq \mathbf{B}^*$$ be the set of sequences of parameters satisfying $H_{0,n}$ in (ref) for every $n \in \mathbb{N}$.

As we shall see, the scaled deviations from the null hypothesis

play the role of a noncentrality parameter in characterizing the consistency properties of tests based on $S_{n,p}(\bm{\beta}^*_n)$. We also let $\sigma_p^2 := \mathbb{V}ar(|Z|^p)$ with $Z\sim\mathsf{N}_1(0,1)$ and define the functions

Power properties of $p$-norm based tests: $p\in[2,\infty)$

We first consider the case of $p\in[2,\infty)$ as the characterization of their power has a common structure. The case of $p=\infty$ is then covered in Section (ref).

Our most general characterization of the power properties of $p$-norm based tests is carried out under the following high-level conditions for which primitive conditions are provided in Section (ref).

Note that $d \lambda_p(0)$ appearing in Assumption (ref) is not the expected value of $S_{n,p}^p(\bm{\beta}_n^*)$, which, in fact, may not even exist. Section (ref) provides conditions under which Assumption (ref) is satisfied for all powers $p\in[2,\infty)$ even when the $h_{j,n}(\bm{X}_{1,n},\bm{\beta}_n^*)$ have only four moments.

The following theorem characterizes asymptotic size and power properties of $p$-norm based tests for $p \in [2, \infty)$. We denote the cdf of a standard normal distribution by $\Phi$.

Part 1 of Theorem (ref) characterizes sequences of critical values that yield asymptotic size control. For any $\alpha\in(0,1)$, a canonical choice is $$\kappa_{n,p}=\sbr[0]{\Phi^{-1}(1-\alpha)d^{1/2}\sigma_p+d\lambda_p(0)}^{1/p}.$$ Part 2 provides results on local asymptotic power and Part 3 provides a complete characterization of the alternatives that a $p$-norm based test is consistent against. Apart from when $p=2$, neither of these can be expressed in terms of only the $p$-norm of the (scaled) deviation from the null $\|\bm{\theta}_n(\bm{\beta}_n^*)\|_p$; instead they depend on the asymptotic behaviour of $$d^{-1/2}\sum_{i = 1}^{d} [\lambda_p(\theta_{i,n}(\bm{\beta}_n^*))-\lambda_p(0)].$$ Concerning consistency, the divergence of the latter is equivalent to divergence of $$\frac{\sum_{i=1}^dg_p(\theta_{i,n}(\bm{\beta}_n^*))}{\sqrt{d}}=\frac{\sum_{i=1}^d\sbr[1]{\theta_{i,n}^2(\bm{\beta}_n^*)\vee |\theta_{i,n}(\bm{\beta}_n^*)|^p}}{\sqrt{d}},$$ which is somewhat easier to interpret.

Concerning the consistency of a $p$-norm based test (with asymptotic size in $(0, 1)$), (ref) implies that a sufficient condition is that $d^{-1/2}\|\bm{\theta}_n(\bm{\beta}_n^*)\|_2^2\to\infty$ or $d^{-1/2}\|\bm{\theta}_n(\bm{\beta}_n^*)\|_p^p\to\infty$, cf. also the previous display. What is more, observing that

Part 3 of Theorem (ref) shows that

when $(\kappa_{n,p})_{n\in\mathbb{N}}$ and $(\kappa_{n,q})_{n\in\mathbb{N}}$ are chosen such that the corresponding tests have asymptotic sizes in $(0,1)$. In words, any violation of the moment conditions $\mathbb{E} h_{n}(\bm{X}_{1,n},\bm{\beta}_n^*)=\bm{0}_{d}$ that a test based on the $p$-norm is consistent against, a test based on the $q$-norm will also be consistent against, $2\leq p<q<\infty$. Thus, the $q$-norm based test always weakly dominates the $p$-norm based test in terms of consistency. This domination is strict if and only if there exists a $\bm{\beta}^*\in\mathbf{B}^*$ such that $\liminf_{n\to\infty}\mathbb{P}\del[1]{S_{n,p}(\bm{\beta}_n^*)\geq \kappa_{n,p}}<1$ but $\mathbb{P}\del[1]{S_{n,q}(\bm{\beta}_n^*)\geq \kappa_{n,q}}\to 1$, i.e., if and only if (cf. (ref))

Summarizing, given the conditions imposed in Theorem (ref), the characterization and monotonicity properties of the consistency of $p$-norm based tests established in kp2021consistency in the Gaussian sequence model continue to hold in the current setting.

By Theorem 3.4 in kp2021consistency the $\bm{\theta}_n(\bm{\beta}_n^*)$ that satisfy (ref) are necessarily approximately sparse and unbalanced. Furthermore, if the limit inferior to the left in (ref) is zero, then there exists a subsequence along which the $p$-norm based test has asymptotic power equal to its size, while the $q$-norm based test is consistent --- a substantial difference in power.

Primitive conditions for Assumptions (ref) and (ref)

We next provide conditions that will be shown to be sufficient for Assumptions (ref) and (ref) to be satisfied and hence for Theorem (ref) to apply. To this end, let $\bm{Z}_d\sim\mathsf{N}_d(\bm{0}_d,\mathbf{I}_d)$, $$\mathcal{C}_n=\cbr[0]{C\subseteq \mathbb{R}^{d(n)}: C\text{ is convex and Borel measurable}},$$ and $\|\bm{A}\|_2$ be the spectral norm of the matrix $\bm{A}$.

Note that these conditions do not impose independence.

Theorem (ref) shows that for Theorem (ref) to apply it suffices to upper bound the estimation error of the covariance matrix, $\|\hat{\bm{\Sigma}}_n(\bm{\beta}_n^*)-\bm{\Sigma}_n(\bm{\beta}_n^*)\|_2$, and a Gaussian approximation to hold.\footnote{We impose the Gaussian approximation to hold over convex sets as we study all $p$-norm based tests simultaneously. If one is only interested in a single $p$, it may be enough for the approximation to hold over $p$-norm balls with arbitrary centres and radii for this given $p$ --- potentially allowing for $d$ to grow faster.} As discussed below, for both of these requirements catalogues of sufficient conditions exist in the literature.

To prove Theorem (ref) we show in Lemma (ref) that when $d^{-1}\sum_{i=1}^dg_2(\theta_{i,n}(\bm{\beta}_n^*))$ is bounded, a Gaussian approximation property holds even when $\bm{\Sigma}_n(\bm{\beta}_n^*)$ is replaced by $\hat{\bm{\Sigma}}_n(\bm{\beta}_n^*)$:

This essentially allows us to deduce the desired properties of $S_{n,p}(\bm{\beta}_n^*)=\|\hat{\bm{\Sigma}}_n^{-1/2}(\bm{\beta}_n^*)\bm{H}_n(\bm{\beta}_n^*)\|_p$ in Assumptions (ref) and (ref) from the ones of $p$-norms of the high-dimensional Gaussian mean shift $\bm{Z}_d+\bm{\theta}_n(\bm{\beta}_n^*)$ in the “limit experiment”, the latter of which was analyzed in kp2021consistency.\footnote{This is done when $d^{-1}\sum_{i=1}^dg_2(\theta_{i,n}(\bm{\beta}_n^*))=d^{-1}\|\bm{\theta}_n(\bm{\beta}_n^*)\|_2^2$ is bounded. In regimes wherein this is not the case, arguments tailored to those settings are used to verify (ref) of Assumption (ref).} More broadly, (ref) may be of independent interest beyond the scope of this paper as it allows one to deduce probabilistic properties of the scaled “central statistic” $\hat{\bm{\Sigma}}_n^{-1/2}(\bm{\beta}_n^*)\bm{H}_n(\bm{\beta}_n^*)$ from those of $\bm{Z}_d+\bm{\theta}_n(\bm{\beta}_n^*)$ even when $d\to\infty$. We next discuss Parts 1 and 2 of Assumption (ref) in further detail.

Concerning 1 of Assumption (ref), it is well-known that if the vectors $h_n(\bm{X}_{i,n},\bm{\beta}_n^*)$ are i.i.d. and appropriately sub-Gaussian, then the empirical covariance matrix $\hat{\bm{\Sigma}}_{n,\text{emp}}(\bm{\beta}_n^*)$ satisfies $\|\hat{\bm{\Sigma}}_{n,\text{emp}}(\bm{\beta}_n^*)-\bm{\Sigma}_n(\bm{\beta}_n^*)\|_2=O_\mathbb{P}(\sqrt{d/n})$, cf., e.g., vershynin2012close or koltchinskiilounici2017. If, in addition, $\bm{\Sigma}_n(\bm{\beta}_n^*)$ possesses further structure, this can be utilised to construct estimators with even better performance guarantees. For example, bickel2008covariance showed that if $\bm{\Sigma}_n(\bm{\beta}_n^*)$ is sparse, $\hat{\bm{\Sigma}}_{n,\text{emp}}(\bm{\beta}_n^*)$ can be thresholded to yield an estimator satisfying $\|\hat{\bm{\Sigma}}_n(\bm{\beta}_n^*)-\bm{\Sigma}_n(\bm{\beta}_n^*)\|_2=O_\mathbb{P}(\sqrt{\log(d)/n})$. The overviews in fan2016overview and cai2016estimating list further estimators utilising structural properties of $\bm{\Sigma}_n(\bm{\beta}_n^*)$ and their performance guarantees.

One is often not willing to assume that the vectors $h_n(\bm{X}_{i,n},\bm{\beta}_n^*)$ are sub-Gaussian. In particular, much effort is currently being devoted to the construction of covariance matrix estimators with the sub-Gaussian performance guarantee $\|\hat{\bm{\Sigma}}_n(\bm{\beta}_n^*)-\bm{\Sigma}_n(\bm{\beta}_n^*)\|_2=O_\mathbb{P}(\sqrt{d/n})$ even when only four moments exist. Although the sample covariance matrix does not work well under such heavy tails,\footnote{This is in analogy to the sub-optimality of the sample average under heavy tails in the one-dimensional mean estimation problem, cf. the overview in lugosi2019mean.} the existence of estimators satisfying $\|\hat{\bm{\Sigma}}_{n}(\bm{\beta}_n^*)-\bm{\Sigma}_n(\bm{\beta}_n^*)\|_2=O_\mathbb{P}(\sqrt{d/n})$ --- without imposing structure on $\bm{\Sigma}_n(\bm{\beta}_n^*)$ --- has been established in abdalla2022covariance and oliveira2022improved, cf. also mendelson2020robust for bounds containing an extra factor $\sqrt{\log(d)}$. These results are proven under the assumption that the kurtosis of all one-dimensional marginals of $h_n(\bm{X}_{1,n}, \bm{\beta}_n^*)$ is bounded (in particular only four moments need to exist):

As pointed out in mendelson2020robust, Assumption (ref) is satisfied if, e.g., $h_n(\bm{X}_{1,n},\bm{\beta}_n^*)-\bm{\mu}_n(\bm{\beta}_n^*)$ follows a multivariate $t$-distribution with $\nu>4$ degrees of freedom. This example is one of rather heavy tails as only moments strictly lower than $\nu$ exist.

Concerning Part 2 of Assumption (ref), note that by translation invariance of convex sets this is equivalent to

that is (scaled) partial sums in $\mathbb{R}^d$ with mean $\bm{0}_d$ and identity covariance matrix $\mathbf{I}_d$ should obey a Gaussian approximation. We show that by fang2020large this holds under i.i.d. sampling and Assumption (ref) (or, slightly weaker, bounded fourth moments) if, up to logarithmic factors in $n$, $d/n^{2/5}\to 0$.

Summarizing the above discussion results in the following statement.

Under the assumptions of Corollary (ref), Theorem (ref) provides a characterization of the consistency properties of $p$-norm based tests for all $p\in[2,\infty)$. In particular, even though the $h_{j,n}(\cdot,\bm{\beta}_n^*)$ only possess four moments, the characterization also holds for $p$-norm based tests with $p>4$; that is the consistency properties of $p$-norm based tests are characterized even when the $p$th moments of the entries of $h_{n}(\bm{X}_{1,n},\bm{\beta}_n^*)$ need not exist.

The size of confidence sets based on $p$-norms for $p\in[2,\infty)$

Theorem (ref) also has implications for the coverage properties and size of confidence sets on the form

Under the assumptions of Part 1 of Theorem (ref), these confidence sets have the desired (uniform) asymptotic coverage guarantee. That is, for any $\bm{\beta}^*\in\mathbf{B}^{(0)}$, it holds that

Furthermore, under the assumptions of Part 3 of the same theorem and $2\leq p<q<\infty$, one has for any $\bm{\beta}^*$ (not satisfying the moment conditions) that

that is any $\bm{\beta}^*$ that is excluded from $\mathsf{CS}_{n,p,1-\alpha_p}$ with probability tending to one will also be excluded from $\mathsf{CS}_{n,q,1-\alpha_q}$ with probability tending to one. Also, if $\bm{\beta}^*$ satisfies the two conditions in (ref), with the limit inferior being zero, then

In words, $\bm{\beta}^*$ then does not satisfy the moment conditions, yet it is included in the confidence set $\mathsf{CS}_{n,p,1-\alpha_p}$ with the same asymptotic probability as those parameters that do, whereas $\mathsf{CS}_{n,q,1-\alpha_q}$ correctly excludes this $\bm{\beta}^*$ asymptotically. Thus, (ref) and (ref) give a sense in which basing confidence sets on larger $p\in[2,\infty)$ results in “smaller” confidence sets. However, using results in pinelis2010asymptotic, it is not difficult to show that already in the high-dimensional Gaussian location model one can have that $\mathbb{E} h_{n}(\bm{X}_{1,n},\bm{\beta}_n^*)\neq\bm{0}_{d}$ and

Thus, it is not generally the case that eventually $\mathsf{CS}_{n,q,1-\alpha_q}\subseteq \mathsf{CS}_{n,p,1-\alpha_p}$.

Tests based on the supremum-norm: $p=\infty$

Tests based on the supremum-norm have received considerable attention due to the work on high-dimensional Gaussian approximations over hyperrectangles

The following theorem exactly characterizes which alternatives tests based on the supremum-norm are consistent against under the same assumptions as those of Theorem (ref).

Well-known sufficient, yet not necessary, conditions for consistency of the supremum-norm based test, such as $\|\bm{\theta}_n(\bm{\beta}_n^*)\|_\infty-\sqrt{2\log(d)}\to\infty$, are trivial special cases of Theorem (ref). In particular, the $\infty$-norm based test is consistent against $(\sqrt{3\log(d)},0,\hdots,0)$ whereas by (ref) of Theorem (ref) no $p$-norm based test with $p\in [2,\infty)$ is consistent.

Concerning dense alternatives of the form $\bm{\theta}_n(\bm{\beta}_n^*)=(c_n,\hdots,c_n)$, it follows that the supremum-norm based test is consistent if and only if $\sqrt{\log(d)}|c_n|\to\infty$, cf. Appendix A.4 of kp2021consistency. Thus, the supremum-norm based test is not consistent if $c_n=1/\sqrt{\log(d)}$ whereas by (ref) of Theorem (ref) every $p$-norm based test with $p\in [2,\infty)$ is consistent. Hence, i) the monotonicity in (ref) does not extend to $q=\infty$ and ii) no single $p$-norm based test dominates all others in terms of its consistency properties.

The exact characterization of consistency in Theorem (ref) sheds new light on the frequently used supremum-norm based test for testing moment equalities. It is more informative than the currently available power analysis of the supremum-norm based test through a maximin-lens. The latter provides the sufficient condition for consistency $\|\bm{\theta}_n(\bm{\beta}_n^*)\|_\infty-\sqrt{2\log(d)}\to\infty$, but is silent about the alternatives not satisfying this condition that the supremum-norm based test is nevertheless consistent against. For example, if $\bm{\theta}_n(\bm{\beta}_n^*)=(1,\hdots,1)'$, then by the observation on dense alternatives in the previous paragraph the supremum-norm based test is consistent, yet $\|\bm{\theta}_n(\bm{\beta}_n^*)\|_\infty-\sqrt{2\log(d)}\not\to\infty$.

Concerning confidence sets, Theorem (ref) shows that for a given $\alpha\in(0,1)$, $\kappa_{n,\infty}(\alpha)$ as in (ref), and $\bm{\beta}^* \in\mathbf{B}^{(0)}$, the sets $\mathsf{CS}_{n,\infty,1-\alpha}:=\cbr[1]{\bm{\beta}\in\mathbf{B}_n:S_{n,\infty}(\bm{\beta})\leq \kappa_{n,\infty}(\alpha)}$ satisfy $\lim_{n\to\infty}\mathbb{P}\del[1]{\bm{\beta}_n^*\in \mathsf{CS}_{n,\infty,1-\alpha}}= 1-\alpha$.

A test dominating all $p$-norm based test for $p\in[2,\infty]$

The discussion following Theorem (ref) revealed that no single $p$-norm based test is “best” in terms of the amount of alternatives it is consistent against. This makes choosing a $p$ to base a test on very difficult. Even if one knows that the deviation from the null $\mathbb{E} h_n(\bm{X}_{1,n},\bm{\beta}_n^*)$ is sparse, which could suggest using a test based on the supremum-norm, sparsity need not be inherited by the noncentrality parameter

due to the presence of $\bm{\Sigma}^{-1/2}_{n}(\bm{\beta}_n^*)$. Therefore, one often does not know anything about the structure of $\bm{\theta}_n(\bm{\beta}_n^*)$, which is the actual quantity entering the characterization of the power of $p$-norm based tests in Theorems (ref) and (ref). This underscores the importance of having a test that is powerful irrespective of the unknown structure of $\bm{\theta}_n(\bm{\beta}_n^*)$.

One could hope that combining tests based on the two “endpoints” $p=2$ and $p=\infty$, e.g., by the power enhancement principle of fan2015power, of the interval $[2,\infty]$ results in a test that is consistent whenever some $p$-norm based test for $p\in[2,\infty]$ is consistent. However, Corollary 4.2 in kp2021consistency shows that already in the Gaussian sequence model this is not the case: There exist (semi-sparse) alternatives against which tests based on $p=2$ and $p=\infty$ are inconsistent, but against which a test based on any single $p\in(2,\infty)$ is consistent. To construct a test that is consistent whenever a test based on some $p\in[2,\infty]$ is consistent, we use a construction similar in spirit to the one employed in the Gaussian sequence model in kp2021consistency. A crucial difference is that the present construction explicitly includes the supremum-norm based test rather than using a $p_n$-norm based test with sufficiently quickly increasing $p_n$ to dominate the former. The present construction is more convenient as it does not require the analysis of tests for moving $p_n$, which allows us to reduce the number of approximation steps in the analysis.

The dominant test

Let $\alpha\in(0,1)$ and $\alpha_2,\alpha_I$ and $\alpha_\infty$ be non-negative with $\alpha_2+\alpha_I+\alpha_\infty=\alpha$ and let $\kappa_{n,2}$ and $\kappa_{n,\infty}$ satisfy

Furthermore, let $p_n$ be a strictly increasing and unbounded sequence in $(2, \infty)$ and let $m_n$ be a non-decreasing and unbounded sequence in $\mathbb{N}$. Fix an array

such that

where the conditions implicitly impose the existence of the respective limits. For every $n \in \mathbb{N}$ and every $j = 1, \hdots, m_n$, choose $\kappa_{n,p_j}>0$ and $c_n\in(0,1]$ such that

where $\mathfrak{P}_n=\cbr[0]{2,p_1,\hdots,p_{m_n},\infty}$. Define the test $\psi_n(\bm{\beta}_n^*)$ as

Thus, $\psi_n(\bm{\beta}_n^*)$ rejects $H_{0,n}:\mathbb{E} h_{n}(\bm{X}_{1,n},\bm{\beta}_n^*)=\bm{0}_{d}$ if for any $p$-norm based tests with $p\in\mathfrak{P}_n$ it is the case that $S_{n,p}(\bm{\beta}_n^*)$ exceeds $c_n\kappa_{n,p}$. The tests based on $p=2$ and $p=\infty$ are included in the construction of $\psi_n(\bm{\beta}_n^*)$ to make it powerful against dense and sparse alternatives, respectively. The tests based on $p\in\cbr[0]{p_1,\hdots,p_{m_n}}$ are included to cover the (semi-sparse) alternatives that neither tests based on the 2- or supremum-norm are consistent against. The quantities $\alpha_2,~\alpha_I,$ and $\alpha_\infty$ control how much size is distributed to the components of $\psi_n(\bm{\beta}_n^*)$ targeting dense, semi-sparse and sparse alternatives, respectively while the array $\mathcal{A}$ distributes the size $\alpha_I$ to the $p$-norm based tests with $p\in\cbr[0]{p_1,\hdots,p_{m_n}}$. The role of $c_n$ is to ensure that $\psi_n(\bm{\beta}_n^*)$ has asymptotic size $\alpha$, cf. Part 1 of Theorem (ref) below. Choosing $c_n=1$ will generally result in $\psi_n(\bm{\beta}_n^*)$ having asymptotic size not larger but potentially smaller than $\alpha$, i.e., a conservative asymptotic level $\alpha$ test.

Part 1 of Theorem (ref) shows that the critical values $\kappa_{n,p}$, $p\in\mathfrak{P}_n$, and the $c_n$ that guarantee exact size $\alpha\in(0,1)$ under Gaussianity in (ref) also guarantee that $\psi_n(\bm{\beta}_n^*)$ has asymptotic size $\alpha$. This may not be obvious ex-ante as the construction of $\psi_n(\bm{\beta}_n^*)$ is based on combining $|\mathfrak{P}_n|=(m_n+2)\to\infty$ tests. However, having established (ref), this becomes a trivial consequence of writing non-rejection of $\psi_n(\bm{\beta}_n^*)$ as the event

where for $r\in(0,\infty)$ we set $\mathbb{B}_p(r)=\cbr[0]{\bm{x}\in\mathbb{R}^d:\|\bm{x}\|_p<r}$ and observe that $\bigcap_{p\in\mathfrak{P}_n}\mathbb{B}_p(c_n\kappa_{n,p})$ is convex.

Part 2 shows that $\psi_n(\bm{\beta}_n^*)$ is consistent against every deviation from the null hypothesis that some $p$-norm based test with $p\in[2,\infty]$ is consistent against (irrespective of the asymptotic size of the latter, as long as it is a number in $(0, 1)$). In particular, $\psi_n(\bm{\beta}_n^*)$ is consistent whenever tests based on the $2$- or $\infty$-norm with asymptotic size in $(0, 1)$ are consistent. Recall that the (dense and sparse) alternatives that tests based on these norms are consistent against are the ones targeted by current applications of the power enhancement principle of fan2015power, cf. also kp. In addition, $\psi_n(\bm{\beta}_n^*)$ is consistent against further alternatives, as it is also consistent as soon as there exists a $p\in(2,\infty)$ such that a test based on this $p$ is consistent (a property that tests based on the power enhancement principle that combine $2$- and $\infty$-norm based tests do not share, cf. kp2021consistency).

Furthermore, as a consequence of Theorems (ref) and (ref), it is not possible to choose a single best $p$-norm to base a test on: Our results show that no matter which single $p\in[2,\infty]$ one chooses, there exists another $p$-norm based test that is consistent against alternatives that the test based on the chosen $p$ is inconsistent against. Theorem (ref) suggests a way to overcome this impossibility of choosing a single “best” norm to base a tests on as $\psi_n(\bm{\beta}_n^*)$ simultaneously dominates all $p$-norm based tests for $p\in[2,\infty]$.

Theorem (ref) is silent about the power properties of $\psi_n(\bm{\beta}_n^*)$ when no $p$-norm based test is consistent. Thus, there could, in principle, exist alternatives against which a $p$-norm based test has substantially higher asymptotic power than $\psi_n(\bm{\beta}_n^*)$ without the former converging to one. The following theorem shows that this possibility can be ruled out by allocating sufficient asymptotic size $\alpha_p$ to a given $p$-norm in the construction of $\psi_n(\bm{\beta}_n^*)$.

Theorem (ref) shows that compared to the $p$-norm based test of asymptotic size $\alpha$ and with exponent $p$, there are no alternatives for which much is lost in terms of asymptotic power by using a test $\psi_n(\bm{\beta}_n^*)$ with $\alpha_{p}$ close to $\alpha$. On the other hand: i) $\psi_n(\bm{\beta}_n^*)$ is consistent against any alternative that some $p$-norm based test is consistent against and ii) alternatives can exist against which $\psi_n(\bm{\beta}_n^*)$ is consistent yet the $p$-norm based test has asymptotic power equal to its asymptotic size. Thus, there can be much to gain from using $\psi_n(\bm{\beta}_n^*)$. Note, however, that the above construction of a test necessitates the choice of $p$ to target. That is, $\psi_n(\bm{\beta}_n^*)$ with asymptotic size $\alpha$ cannot have power arbitrarily close to, e.g., that of the $2$-norm and $\infty$-norm based tests (each with asymptotic size $\alpha$) simultaneously.

Many IVs and dominating the Anderson-Rubin test

Recall that in the context of the linear IV model in Section (ref) one has

The theory developed in the previous sections has the following consequences for testing whether a given $\bm{\beta}_n^*$ satisfies $\mathbb{E} (y_{1,n}-\bm{Y}_{1,n}'\bm{\beta}_n^*)\bm{z}_{1,n}=\bm{0}_d$:

Rapidly increasing dimension via sample splitting

The results presented so far generally restrict the growth rate of $d$. Suppose, however, that one wishes to test whether $\bm{\beta}_n^*$ satisfies $D=D(n)$ moment conditions

with $D$ unrestricted. Under independent sampling one can use a two-step procedure to test the validity of (ref) by means of the tests developed in the previous sections: Based on a partition of the sample into two disjoint subsamples $N_1=N_1(n)\subset\cbr[0]{1,\hdots, n}$ and $N_2=N_2(n)\subset\cbr[0]{1,\hdots, n}$ of sizes $n_1=n_1(n)$ and $n_2=n_2(n)$, respectively, one first uses the subsample $N_1$ to select a subset $S=S(n_1,n_2,\cbr[0]{\bm{X}_{i,n}}_{i\in N_1})\subset \cbr[0]{1,\hdots,D}$ of moment functions. In a second step, using the subsample $N_2$, one tests whether $\bm{\beta}_n^*$ satisfies

using only the data $\cbr[0]{\bm{X}_{i,n}}_{i\in N_2}$. In Theorem (ref) in Section (ref) in the appendix we show that if a predetermined fixed number of selected moments $|S|=d(n_2)$ satisfies the growth conditions of the theory developed in the previous sections relative to the second step “sample size” $n_2$, then all asymptotic size guarantees established so far remain valid. The concrete moment selection mechanism $S$ does not affect the asymptotic size of the subsequent tests as independent sampling guarantees that no selection bias is introduced. However, it is clear that the power of a test depends on $S$ including moments that are violated. Some ways to select the subset of moments $S$ to be tested in order to obtain powerful tests are discussed in Appendix (ref), where the size-control result informally sketched above is also formally established in Theorem (ref).

Numerical results

In Section (ref) we investigate numerically the properties of our test $\psi_n(\bm{\beta}_n^*)$ for hypothesis testing on a structural parameter in the presence of many instruments, cf. Sections (ref) and (ref). Then, to investigate the effect of even larger $d$ without unduly increasing the computational burden, Section (ref) considers directly the limiting Gaussian testing problem which by (ref) approximates the rejection frequencies of $\psi_n(\bm{\beta}_n^*)$ and $p$-norm based tests in many moment testing problems.

Testing in the presence of many instruments

We generate data from the following standard two-equation model with one endogenous regressor ($\mathbf{B}_n=\mathbb{R}$), but many instruments $d$:

The variables are named and interpreted as in Section (ref). All variables are i.i.d. across $i=1,\hdots,n$ and we consider a setting where the instruments $\bm{z}_{i,n}$ and error terms have heavy tails in the sense that the $d$ independent entries of $\bm{z}_{1,n}$ follow a $t(5)$-distribution and

In addition, $\bm{z}_{1,n}\perp \!\!\! \perp(u_{1,n},\nu_{1,n})$. Thus, the “true” parameter $\bm{\beta}_n$ satisfies $\mathbb{E} (y_{1,n}-\bm{Y}_{1,n}'\bm{\beta}_n)\bm{z}_{1,n}=\mathbb{E} u_{1,n}\bm{z}_{1,n}=\bm{0}_d$ and we test whether the candidate structural parameter $\bm{\beta}_n^*$ satisfies

which falls within our general testing framework. Throughout we use $\bm{\beta}_n^*=0$ and gauge the size and power of the tests considered by generating data for a range of $\bm{\beta}_n$.

We consider $(n,d)\in\cbr[1]{(5{,}000,100),(25{,}000,500),(100{,}000,1{,}000)}$. Although one cannot always obtain $n$ of this order, these settings reflect many practically relevant situations, cf. Footnote (ref).

The number of relevant instruments is governed by the number of non-zero entries of $\bm{\pi}_n\in\mathbb{R}^d$, i.e., the sparsity/denseness of the “first stage”. For each pair $(n,d)$ we consider the sparsest and densest possible first stage, namely $\bm{\pi}_n=(1,0,\hdots,0)$ and $\bm{\pi}_n=(1,\hdots,1)$. As the latter first stage is clearly more informative, we adjust the magnitude of the deviation of $\bm{\beta}_n$ from $\bm{\beta}_n^*=0$ according to the “strength” of $\bm{\pi}_n$ to get non-trivial power curves.\footnote{Alternatively, one can adjust the magnitude of the entries of $\bm{\pi}_n$, i.e. change the strength of the individual instruments, with the number of non-zero entries of $\bm{\pi}_n$.} Finally, we consider a setting of a semi-sparse first stage where a moderate number of instruments is relevant. Here $\bm{\pi}_n=(\bm{\iota}_{d_{R,n}},0,\hdots,0)$ where $\iota_{d_{R,n}}$ is a vector of $d_{R,n}$ ones and $d_{R,n}=4,7$, $12$, for $n=5{,}000,25{,}000$ and $100{,}000$, respectively. We also note that a simple calculation reveals that in the present setting the number of non-zero entries of $\bm{\theta}_n(\bm{\beta}^*_n)=\sqrt{n}\bm{\Sigma}^{-1/2}_{n}(\bm{\beta}^*_n)\mathbb{E} h_{n}(\bm{X}_{1,n},\bm{\beta}^*_n)$ equals that of $\bm{\pi}_n$ as $\bm{\Sigma}^{-1/2}_{n}(\bm{\beta}^*_n)$ is block-diagonal. Thus, when we vary the sparsity of $\bm{\pi}_n$, we identically vary that of $\bm{\theta}_n(\bm{\beta}^*_n)$.

Throughout, we estimate $\bm{\Sigma}_n(\bm{\beta}_n^*)$ by the sample covariance matrix of the vectors $(y_{i,n}-\bm{Y}_{i,n}'\bm{\beta}_n^*)\bm{z}_{i,n},\ i=1,\hdots,n$. We also experimented with the median-of-means estimator in Section 3.4 of ke2019user, but (although being more precise for heavy-tailed distributions and allowing for larger $d$) this did not improve the performance of the resulting tests in the settings considered.

We implement our test $\psi_n(\bm{\beta}_n^*)$ from Section (ref) with $\alpha=0.05$, $\mathfrak{P}_n=\cbr[0]{2,3,5,10,\infty}$, $\alpha_2=\alpha_\infty=\alpha_{n,p}=\alpha/5$ for $p\in\cbr[0]{3,5,10}$ and all $n$ considered. The power enhancement (PE) principle of fan2015power, which combines the $2$- and supremum-norm based tests, is implemented by setting $\alpha_2=\alpha_\infty=\alpha/2$ and $\alpha_I=0$. For $n=5{,}000$ we also implement the jacknifed AR test of mikusheva2020inference, which we refer to as MS.\footnote{For larger sample sizes the implementation time became prohibitive for the jacknifed AR test. However, unreported simulations revealed that for sufficiently small $n$ (relative to $d$) it controls size better than the $p$-norm based tests that we study. This is in line with the theoretical results on size control of the jackknifed AR test in mikusheva2020inference.} The number of replications is $1{,}000$ for $n=5{,}000$ and $n={25{,}000}$, but $500$ for $n=100{,}000$. Figure (ref) plots by how much the power of $\psi_n(\bm{\beta}_n^*)$ exceeds that of each of the other tests considered. Thus, when comparing to the $2$-norm based test we plot the power difference $\mathbb{E} \psi_n(\bm{\beta}_n^*)-\mathbb{P}\del[1]{S_{n,2}(\bm{\beta}_n^*)\geq \kappa_{n,2}}$ and similarly for the other tests considered. Positive numbers mean that $\psi_n(\bm{\beta}_n^*)$ is superior and vice versa for negative numbers. The raw power function can be found in Figure (ref) in Section (ref) of the appendix. Figure (ref) reveals:

In the next section we shall see that the power gains from using $\psi_n(\bm{\beta}_n^*)$ instead of a 2- or supremum-norm based test (or a power enhancement based combination of these) become even larger in higher dimensions.

Gaussian sequence model

To illustrate the power gains that one can expect when $d$ becomes even larger than in the IV-based example in the previous subsection, we study testing

when one observes one realization of $\bm{Z}_d\sim\mathsf{N}_d(\bm{\theta}_d,\mathbf{I}_d)$ and $p$-norm based tests are on the form $\mathds{1}\del[0]{\enVert[0]{\bm{Z}_d}_p\geq \kappa_{d,p}}$, $p\in[2,\infty]$.\footnote{Here we write $\kappa_{d,p}$ rather than $\kappa_{n,p}$ as $n=1$ in the “limit experiment” (note that also in the previous sections the critical values $\kappa_{n,p}$ only depended on $n$ via $d(n)$. A similar remark applies in the sequel.} By (ref) the rejection frequencies $\mathbb{P}\del[0]{\enVert[0]{\bm{Z}_d}_p\geq \kappa_{d,p}}$ in this “limit experiment” will be close to $\mathbb{P}\del[1]{S_{n,p}(\bm{\beta}_n^*)\geq \kappa_{n,p}}$ in a large set of moment testing problems as $n\to\infty$ and with $\bm{\theta}_d=\bm{\theta}_n(\bm{\beta}^*_n)=\sqrt{n}\bm{\Sigma}^{-1/2}_{n}(\bm{\beta}^*_n)\mathbb{E} h_{n}(\bm{X}_{1,n},\bm{\beta}^*_n)$. We study $d=1{,}000,5{,}000, 50{,}000, 250{,}000$ as well as sparse, semi-sparse and dense deviations from $H_{0,d}$. The sparse and dense $\bm{\theta}_d$ have one and $d$ (equal) non-zero entries, respectively. The semi-sparse alternatives have $15$, $30$, $80$, and $170$ (equal) non-zero entries, respectively (for increasing values of $d$). All tests are implemented as in the IV-based simulations in the previous subsection. Analogously to Figure (ref), Figure (ref) plots by how much the power of $\psi_d$ exceeds that of the other tests studied. It is based on $10{,}000$ replications.\footnote{The figures for $d=50{,}000$ are identical to those in Figure (ref).} The raw power function can be found in Figure (ref) in Section (ref) of the appendix. Figure (ref) reveals:

Thus, significant power gains can be obtained against semi-sparse alternatives from using $\psi_d$ instead of a $2$- or supremum-norm based test (or a combination of these) without sacrificing power against sparse or dense alternatives. These power gains are obtained by harnessing the strength of norms beyond $2$ and $\infty$; in our case the $3$-, $5$- and $10$-norm.