Simultaneous Confidence Intervals for High-dimensional Linear Models with Many Endogenous Variables

High-dimensional Linear Models with Many Endogenous Variables

Introduction

We consider estimation of parameters in high-di\-men\-sional linear regression models with many endogenous variables:

We allow for cases where $p$, the dimension of the endogenous variable $x_i$, is large relative to the sample size $n$; and we consider cases where many or all of the parameters in $\beta$ are of interest. This structure arises naturally in many structural economic models. For example, one may be interested in estimating a demand relationship in which the quantity demanded of a given product may depend on its own price as well as the price of other related products. In this case, $y_i$ represents the ($\log$) quantity sold of the product of interest, $x_i$ represents the vector of ($\log$) prices of all potentially related products including the price of the product of interest, and $i$ indexes independent markets in which data on these products are available. $\beta_0$ then provides information on own- and cross-price elasticities for a given product.

Of course, informative estimation and inference about $\beta$ is impossible without further structure. We adopt an instrumental variables (IV) framework and assume that a set $z_i$ of $K$ instruments with $K \geq p$ is available such that ${\rm E}[z_i\xi_i] = 0$. We thus work in a model with many endogenous variables and many instruments.

To cope with the high-dimensionality of $\beta_0$, we adopt a sparse modeling framework and assume that the number of non-zero elements in $\beta_0$ is small relative to the sample size. We also make use of sparsity assumptions over first-stage relationships. Specifically, suppose we are interested in a particular element, $\beta_{0,j}$, of the vector $\beta_0$. We define an instrument specific to $\beta_{0,j}$ as a linear combination $z_i'\mu_j$ of the available high-dimensional set of instruments that satisfies the usual IV moment restriction ${\rm E}[(y_i - \beta_{0,j}x_{j,i} - \sum_{k \ne j}\beta_{0,k}x_{k,i})(z_i'\mu_j)] = 0$ and require that $\mu_j$ is a sparse vector. When we are interested in multiple elements of $\beta_0$, we form a different linear combination of the available set of instruments for each element of interest and require that all such linear combinations are sparse. We also note that the rank condition for identification, which we maintain, will implicitly restrict the association between each of these constructed linear combinations.

In principle, there are many ways to choose the $\mu_j$ used to define the linear combinations of instruments we use to identify the $\beta_{0,j}$. We exploit this to choose the $\mu_j$ to guarantee a type of orthogonality between the constructed instrument $z_i'\mu_j$ and the nuisance function in the moment condition for $\beta_{0,j}$, $\sum_{k \ne j}\beta_{0,k}x_{k,i}$. Specifically, we choose the $\mu_j$ so that ${\rm E}[x_{i,-j}z_i'\mu^j_0] = 0$ while requiring that ${\rm E}[x_{i,j}z_i'\mu^j_0] \ne 0$. The second condition is simply the usual instrument relevance condition. The first condition ensures that the moment condition used to identify $\beta_{0,j}$ is locally insensitive, or orthogonal, to the value of the high-dimensional set of nuisance parameters $\{\beta_{0,k}\}_{k \ne j}$. This orthogonality is a key ingredient to producing post-model-selection estimators that are $\sqrt{n}$-consistent and asymptotically normal and post-model-selection inferential procedures that are uniformly valid across a wide range of data generating processes (DGPs) that include cases where perfect model selection is theoretically impossible. For further detail and discussion, see, for example, BCFH:Policy, CHS:AnnRev, and DML.

We provide a simple-to-implement multi-step estimation and inference procedure for the structural parameters. We obtain initial estimates of $\beta_0$ by adapting the estimator of gautier2011high. We then obtain estimates of the $\mu_{0,j}$ through a novel estimation procedure that builds upon BCKRT2016a. Given these estimates, we construct the empirical analog of the IV moment condition for $\beta_{0,j}$ by plugging in the estimated values for $\{\beta_{0,k}\}_{k \ne j}$ and $\mu_{0,j}$ and solve for the final estimator of $\beta_{0,j}$. The final step of the procedure is trivial, and the first two steps proceed by solving high-dimensional convex optimization problems for which efficient optimization routines are available. A major novelty of our approach is that it is formulated in such a way that we may use simple, nuisance parameter free values for the penalty parameters involved in the two penalized optimization problems even when model errors are heteroskedastic. Thus, we avoid potentially complicated and theoretically problematic data-dependent choice of tuning parameters. The proposed high-dimensional estimation procedure may be of substantive interest outside of the present context. Finally, we employ the multiplier bootstrap for inference about parameters of interest in $\beta_0$. Following, e.g., chernozhukov2013gaussian, we establish that the multiplier bootstrap provides valid simultaneous confidence regions allowing for a high-dimensional set of parameters of interest; see also Buehlmann:etal:bootstrap:2016 for an application in a linear model with exogenous right-hand-side variables.

Our paper contributes to the growing literature on inference for structural parameters in sparse high-dimensional IV settings. Much work in this setting focuses on the case where the dimension of the endogenous variable is small but there are a large number of available instruments; e.g. BaiNg2009a, BaiNg2010, BellChernHans:Gauss, BellChenChernHans:nonGauss, bch:jep; or a large number of instruments and controls; e.g. CHS:PnP, CHS:AnnRev. BCFH:Policy considers estimation of treatment effects in IV models with binary instrument and endogenous variable in the presence of a high-dimensional set of control variables in detail. In a different but tangentially related direction, kangetal1 and Caner:invalid, among others, consider estimation in the presence of a high-dimensional set of instruments allowing for some of the instruments to be invalid with data-dependent selection of the valid and invalid instruments. The work in the present paper is most closely related to fan:liao and especially gautier2011high, both of which consider estimation in models with a large number of endogenous variables. Both papers adopt similar estimation strategies to that employed in this paper though neither explicitly constructs instruments to guarantee orthogonality and fan:liao focuses on establishing oracle estimation results rather than providing a uniformly valid inferential procedure.

Regarding uniform valid inference, the work of neykov2015unified proposes a unifying treatment for various models with linear moment conditions based in de-biasing ideas and many endogenous variables are discussed in the appendix as one of the many examples. Another related work is gold2017inference, developed concurrently with this work, proposes the use of related de-biasing ideas after the use of lasso on estimated regressors (the projection of the endogenous variables on the instruments) to obtain initial estimates of the coefficients of these endogenous variables. This has the nice feature to allow the authors to invoke the regularity conditions on the design matrix that are standard in the literature (e.g. restricted eigenvalue conditions) but additional penalization is needed to handle the error in the estimated regressors. We also note that the work gautier2011high is currently being revised and also pursues uniform valid inference building upon ideas from javanmard2014confidence.

Our paper provides a useful complement to the existing high-dimensional IV work by allowing for a high-dimensional set of endogenous variables, providing an easy-to-implement estimation procedure with a simple pivotal tuning parameter choice, and establishing a uniformly valid inferential procedure for a high-dimensional set of coefficients of interest within the linear IV model. We explicitly allow non-i.i.d. observations, non-exponential tails, provide procedures where penalty parameters are pivotal, and confidence regions that are simultaneously valid for many parameters. An important technical contribution of our work is a new bound on the sensitivity quantity that was proposed in gautier2011high. We note that in Section 9 of gautier2011high, a related pivotal problem was proposed to identify invalid moment conditions when an initial estimator $\hat\beta$ is available. The pivotal estimation for the orthogonalization/de-biasing step seems to be new as well.

The remainder of the paper is organized as follows. We first restate the model and provide a detailed outline of the estimation approach in Section (ref). In Section (ref), we provide conditions under which we verify that the proposed estimator has desirable properties and under which we can produce inferential statements that are uniformly valid over a large class of models. We present proofs of the main results in the appendix.

Model and Method

We consider a linear instrumental variable model

where potentially many of the components of $x$ are correlated with the noise term $\xi$ where a $K$-dimensional vector of valid instruments $z$ is available. Valid instruments satisfy the following orthogonal condition

which can be used to identify $\beta_0$ under appropriated conditions. (Therefore if the $j$th component of $x$ is not correlated with the noise, $x_j$ is a valid instrument.) It follows that we need at least as many instruments as the dimension of $\beta_0$ (i.e. $K\geq p$).

To construct estimators and associated honest confidence regions, we will construct an instrument tailored for each component $\beta_{0j}$ of interest. More formally we will consider a linear combination of the original instruments $z'\mu$, to create a moment condition whose unique solution is $\theta=\beta_{0j}$ $$ {\rm E}[(y-x_j\theta-x_{-j}'\beta_{0,-j})z'\mu^j]=0 $$ where $\mu^j$ is a non-zero $K$-dimensional vector. It follows that there are many choices of $\mu^j$ that satisfy the relation above by ((ref)). We exploit the freedom to pick $\mu^j$ to choose $\mu^j_0$ so that an orthogonality condition also holds, namely

while having $z'\mu^j_0$ correlated with $x_j$, i.e. ${\rm E}\left[\frac{1}{n}\sum_{i=1}^n x_{ij}z_i'\mu^j_0 \right] \neq 0$. Such orthogonality condition plays a key role in our results to reduce the impact of estimation mistakes of $\beta_{0,-j}$. We will define $\mu^0_j$ as $$ (\mu^j_0,\vartheta^j_0)= \arg\min_{\mu,\vartheta} {\rm E}[\mbox{$\frac{1}{n}\sum_{i=1}^n$} (x_{ij}-z_i'\mu-x_{i,-j}'\vartheta)^2] \ : \ {\rm E}[\mbox{$\frac{1}{n}\sum_{i=1}^n$}x_{-j}z_i'\mu]=0. $$ In order to construct our estimator for $\beta_{0j}$, we will need estimates of the nuisance parameters $\eta^j_0=(\beta_{0,-j},\mu^j_0)$ which are attainable under sparsity conditions to cope with their high-dimensionality.

Estimators for $\beta_0$ that can achieve good rates of convergence have been proposed in the literature gautier2011high. Here we propose to use

In order to estimate a $\mu^j_0$ we will also build upon these self-normalization ideas to achieve a procedure that does not require to tune parameters. However additional constrains are necessary to achieve the orthogonality condition ((ref)). Our proposed estimator seems to be new and can be of independent interest in other setting:

where $c\geq 1$ is a fixed constant.\footnote{This constant accounts for non-i.i.d. setting. We can take $c=1$ if the data is i.i.d..}

Based on the estimates $\hat\eta^j=(\hat\beta_{-j},\hat\mu^j)$ of $\eta^j_0=(\beta_{0,-j},\mu^j_0)$ based on ((ref)) and ((ref)), we compute our estimate $\check\beta_j$ as the solutions of

It follows the estimator can be computed in closed form as

where $\hat\Omega_j = \frac{1}{n}\sum_{i=1}^nx_{ij}z_i'\hat\mu^j$ is an estimate of $\Omega_j={\rm E}[\frac{1}{n}\sum_{i=1}^nx_{ij}z_i'\mu^j_0]$.

Next we construct simultaneous confidence regions for the coefficients corresponding to all components in a set $ S\subseteq\{1,\dots,p\}$, i.e. for a given $\alpha\in(0,1)$, we choose a critical value $c_{\alpha,S}^*$ such that with probability converging to $1-\alpha$ we have

where the estimate of $\sigma_j^2 = \Omega_j^{-2}{\rm E}[\frac{1}{n}\sum_{i=1}^n\{(y_i-x_i'\beta_0)z_i'\mu^j_0\}^2]$ is given by

We apply the procedure ((ref))-((ref)) to each component $j\in S$ to obtain $\check\beta_j$ and use a multiplier bootstrap to compute the appropriate critical value in ((ref)). The confidence bands have the desired asymptotic coverage even in cases where the cardinality of $S$ is larger than the sample size $n$ as our analysis builds upon results for high-dimensional central limit theorems developed in chernozhukov2013gaussian,chernozhukov2014clt,chernozhukov2012gaussian,chernozhukov2012comparison,chernozhukov2015noncenteredprocesses. We define the vector $\widehat{\mathcal{G}}$ as

where $(g_i)_{i=1}^n$ are independent standard normal random variables independent from $(y_i,x_i,z_i)_{i=1}^n$. We compute the critical value $c_{\alpha,S}^*$ as the $(1-\alpha)$-quantile of the conditional distribution of $$\max_{j\in S} |\widehat{\mathcal{G}}_j|$$ given the data. For the readers convenience Appendix A provides a detailed description of the algorithm to be implemented for the proposed estimators and confidence regions.

Theoretical Results

Next we provide regularity conditions on the data generating process ((ref))-((ref)) under which the estimators and confidence regions proposed in Section (ref) are guaranteed to perform well despite model selection mistakes and high-dimensionality of nuisance parameters.

Rates of Convergence of $\hat\beta$ in ((ref))

We first establish bounds for the $\ell_q$-error of the estimator $\hat\beta$ defined as in ((ref)). Such bounds will yield rates of convergence that are useful to the development of confidence regions ((ref)) since the procedure relies on having estimators of the nuisance parameters. In order to analyze this estimator we consider the $\ell_q$-sensitivity coefficients proposed in gautier:tsybakov which account for the instruments. For $q\geq 1$, and $\Psi = \frac{1}{n}Z'X$, define

where $C_J(u)=\{\theta \in \mathbb{R}^p : \|\theta_{J^c}\|_1 \leq u \|\theta_J\|_1\}$, $u>0$ and $J\subseteq \{1,\ldots,p\}$. These sensitivity quantities are useful to establish convergence in the $\ell_q$-norms for penalized estimators as shown in gautier2013pivotal,gautier2011high. Next we establish new lower bounds on the $\ell_q$-sensitivity coefficients for $q\in\{1,2\}$ that are of independent interest. These lower bounds lead to sharp rates for many design of interest in econometric settings.

Theorem (ref) extends the scope of designs for which we have lower bounds. The main difficulty is to handle non-symmetric matrices $\Psi$ with off-diagonal elements that are not small. The proof relies on sparse singular values decomposition (via $\sigma_{\min}(m)$ and $\sigma_{\max}(m)$) which are akin to sparse eigenvalues that were used in the literature, see e.g. BickelRitovTsybakov2009,BellChenChernHans:nonGauss. The following corollaries provides bounds under primitive conditions. The result of Theorem (ref) can also be applied in the presence of weak instruments as illustrated below.

The next corollary considers the case of random design with strong instruments.

Under conditions of Corollary (ref) and the typical case that $u \leq 3$, we have

with high probability. In turn, under Condition IV1 below, ((ref)) allows the pivotal estimator proposed in ((ref)) to achieve optimal rates. In the analysis of subsequent sections we will assume directly ((ref)) given that Corollary (ref) provides an interesting set of data generating processes that imply it with high probability.

\\ {\bf Condition IV1.} {\it (i) The vectors $\{(y_i,z_i,x_i,\xi_i), i=1,\ldots,n\}$ are independent across i obeying ((ref)) and ((ref)) with $\|\beta_0\|_0 \leq s$. (ii) Suppose that the sequence $M_{nz\xi}:=\max_{j\leq K}{\rm E}[\frac{1}{n}\sum_{i=1}^n|z_{ij}\xi_i|^3]^{1/3} / {\rm E}[\frac{1}{n}\sum_{i=1}^n|z_{ij}\xi_i|^2]^{1/2}$ satisfies $M_{nz\xi}\log^{1/2}(npK) = o(n^{1/6})$ and define $H_n:=\max_{j\leq K} \|\frac{1}{n}\sum_{i=1}^nz_{ij}^2 x_ix_i'\|_{\infty}$.} \\

Condition IV1 is a standard condition allows for heteroskedasticity. It restricts the number of covariates and instruments, but it allows for them to be larger than the sample size. The next theorem establishes bounds on the estimation error $\hat\beta-\beta_0$.

Theorem (ref) has several important features. First it allows for heteroskedasticity which is prevalent in econometric applications. Second, the definition of the estimator ((ref)) uses penalty parameters that do not depend on any unknown quantity. In particular, it bypasses the need for cross validation which can be effective in some cases but validity is not typically established in the literature. This approach builds upon self-normalized ideas that were applied in BCKRT2016a to error-in-variables models.

Rates of Convergence of $\hat\mu^j$ in ((ref))

Next we consider the estimation of the nuisance parameter $\mu^j_0$ for all $j\in S$. Our analysis still also relies on $\ell_q$-sensitivity coefficients proposed in gautier:tsybakov but for different choice of design matrix given by $\Psi_j = \frac{1}{n}\widetilde Z' \widetilde Z$ where $\widetilde Z=(Z, X_{-j})$ and further restricting to directions $\delta \in C_J(u)$. We define

where $\widetilde C_{j,J}(u) = \{ \theta \in C_J(u) : \exists \theta^1,\theta^2, \ \ \theta=\theta^1-\theta^2, \ \ \|{\mathbb{E}_n}[x_{-j}z'\theta^m]\|_\infty\leq 2\tau\max_{k\in[p]\setminus\{j\}}\{{\mathbb{E}_n}[x_k^2(z'\theta^m)^2]\}^{1/2}, m=1,2\}$. In many designs we can establish lower bounds on $\kappa_q^{\widetilde Z\widetilde Z}(s,u)$ ignoring the additional restrictions.

In what follows we define $v_{ij}:=x_{ij}-z_i'\mu^j_0-x_i'\vartheta_0^j$, $j\in S$. Define:\\ $\bar M_{nvz} := {\rm E}[\max_{i\leq n,j\in S,k}|v_{ij}z_{ik}{\rm E}[v_{ij}z_{ik}]|^2]^{1/2}$,\\ $\bar M_{nvx} :={\rm E}[\max_{i\leq n,j\in S,k}|v_{ij}x_{ik}{\rm E}[v_{ij}x_{ik}]|^2]^{1/2}$,\\ $\bar M_{nxz} :={\rm E}[\max_{i\leq n,j\in S,k}|x_{ik}z_i'\mu_0^j{\rm E}[x_{ik}z_i'\mu_0^j]|^2]^{1/2}$, and\\ $\bar M_n := \bar M_{nxz}\vee \bar M_{nvz}\vee \bar M_{nvx}$. \\ Note that in the i.i.d. setting we have $\bar M_n=0$.

\\ {\bf Condition IV2.} {\it For each $j \in S$: (i) The vectors $\{(z_i,x_i), i=1,\ldots,n\}$ are independent across $i\in [n]$ obeying ((ref)), ((ref)), and ((ref)) with $\|\mu^j_0\|_0 \leq s$. (ii) We assume: ${\rm Var}(\frac{1}{\sqrt{n}}\sum_{i=1}^nx_{ik}z_i'\mu_0^j)^{1/2}\geq \tilde c$, ${\rm Var}(\frac{1}{\sqrt{n}}\sum_{i=1}^nv_{ij}z_{ik})^{1/2}\geq \tilde c$, ${\rm Var}(\frac{1}{\sqrt{n}}\sum_{i=1}^nv_{ij}x_{i\ell})^{1/2}\geq \tilde c$, ${\rm E}[\frac{1}{n}\sum_{i=1}^n|x_{ik}z_i'\mu_0^j|^3] \leq C$, ${\rm E}[\frac{1}{n}\sum_{i=1}^n|v^j_iz_{ik}|^3]^{1/3}\leq C$, and ${\rm E}[\frac{1}{n}\sum_{i=1}^n|v_{ij}x_{ik}|^3]^{1/3}\leq C$. Furthremore, It holds that $\log^{1/2}(pnK)=o(n^{1/6})$ and $Cn^{-1}\bar M_n\log(Kpn)=o(1)$. Also define $H_n:= H_n^x\vee H_n^z\vee H_n^{xz}$ where $H_n^x := \max_{k\in[K]} \|\frac{1}{n}\sum_{i=1}^nz_{ik}^2 x_ix_i'\|_{\infty}$, $H_n^x := \max_{\ell \in [p]} \|\frac{1}{n}\sum_{i=1}^nx_{i\ell}^2 z_iz_i'\|_{\infty}$, and $H_n^{z} := \|\frac{1}{n}\sum_{i=1}^nz_{ik}^2 z_iz_i'\|_{\infty}.$} \\

Condition IV2 contains a mild condition on the design, is a standard condition allows for heteroskedasticity and fixed design. The next theorem summarizes the properties of the estimator defined in ((ref)).

Linear Representation and Simultaneous Confidence Bands

In this section we turn to the construction of confidence regions. A critical intermediary step is to establish a linear representation for the estimators $\check\beta_j, j\in S$, despite of the high-dimensional nuisance parameters. Moreover, we will achieve results uniformly over a large class of data generating processes including many where we cannot avoid model selection mistakes.

We will assume the following condition. In what follows the sequence $\delta_n\to 0$ is fixed, and we let $v_{ij}:= x_{ij}-z_i'\mu^j_0-x_{i,-j}'\vartheta^j_0$, and define the following moments $M_{nvz}:={\max_{j\in S, k\in [K]}}{\rm E}[\frac{1}{n}\sum_{i=1}^n|v^j_iz_{ik}|^3]^{1/3}/{\rm Var}(\frac{1}{\sqrt{n}}\sum_{i=1}^nv_iz_{ik})^{1/2}$, $M_{nxz}:={\displaystyle\max_{j\in S, k\in [p]\setminus\{j\}}}{\rm E}[\frac{1}{n}\sum_{i=1}^n|x_{ik}z_i'\mu_0^j|^3]^{1/3}/{\rm Var}(\frac{1}{\sqrt{n}}\sum_{i=1}^nx_{ik}z_i'\mu_0^j)^{1/2}$.

{\bf Condition IV.} {\it The data $\{W_i=(y_i,\xi_i,x_i,z_i,z_i'\mu^S_0, v_{iS}), i=1,\ldots,n\}$ is independent across $i$ and obey the model ((ref))-((ref)). We have $\|\beta_0\|_0+\max_{j\in S}\|\mu^j_0\|_0\leq s$ and $\|\beta_0\|_\infty+\max_{j\in S}\|\mu^j_0\|_\infty\leq C$. The following moment conditions hold: ${\rm E}[\frac{1}{n}\sum_{i=1}^n(W_i'\theta)^4]^{1/4} \leq C\|\theta\|$ for every $\theta \in \mathbb{R}^{{\rm dim}(W_i)}$, $\min_{j\in S} |{\rm E}[\frac{1}{n}\sum_{i=1}^nx_{ij}z_i'\mu^j_0]|\geq c$, $K_n = {\rm E}[\max_{i\leq n}\|W_i\|_\infty^q] < \infty$ for some $q>4$, $K_n^{2/q}\log^{1/2}(Kpn) \leq \delta_n n^{1/2}$, $s\log(pKn) \leq \delta_n \sqrt{n}$. With probability $1-o(1)$ we have $\kappa_q^{ZX}(s,3) \geq s^{-1/q}c$ and $\kappa_{q}^{\widetilde Z \widetilde Z}(s,3) \geq s^{-1/q}c$ for $q\in \{1,2\}$. Conditoins IV1 and IV2 also hold.}

Condition IV corresponds to standard assumptions in the literature with strong instruments. The moments conditions are similar to the ones used in BellChenChernHans:nonGauss and allow the use of self-normalized moderate deviation theory. We note that Condition IV also imposes Conditions IV1 and IV2 which in turn allows us to use the estimators studies in sections (ref) and (ref) of the nuisance parameters. The conditions on the sensitivity quantities are comparable to the restricted eigenvalue condition in the literature of high-dimensional sparse models, see gautier:tsybakov.

The following is among our main results. It establishes a linear representation of the proposed estimators $\check\beta_j, j\in S$.

An important feature of Theorem (ref) is its validity uniformly over the class of data generating processes that satisfy Condition IV. In particular, the result covers data generating processes for which model selection mistakes are likely to happen. Next we build upon Theorem (ref) to establish various useful results.

A consequence of Theorem (ref) is the construction of confidence intervals for each component as $\sqrt{n}\sigma_j^{-1}(\check\beta_j-\beta_{0j})\rightsquigarrow N(0,1)$. Importantly, it holds uniformly over data generating processes with arbitrary small coefficients. Indeed, the orthogonality condition mitigates the impact of model selection mistakes which are unavoidable for those components.

Next we turn to simultaneous confidence bands over $S\subset\{1,\ldots,p\}$ components of $\beta_0$. We allow for the cardinality of $S$ to also grow with the sample size (and potentially $S=\{1,\dots,p\}$). This exploits new central limit theorems for high dimensional vectors, see chernozhukov2015noncenteredprocesses and the references therein. The following result provides sufficient conditions under which the multiplier bootstrap procedure based on ((ref)) yields (honest) simultaneous confidence bands.

In what follows we let $A_n\geq \max_{j\in S}{\rm E}[\frac{n^{-1}}{\sigma_j^{3}\Omega_j^{3}}\sum_{i=1}^n|\xi_iz_i'\mu^j_0|^3]$ and $B_{\bar q,n} \geq \frac{1}{n}\sum_{i=1}^n{\rm E}[ \max_{j\in S}|\xi_iz_i'\mu^j_0|^{\bar q}]$ for some $\bar q>3$. Note that $B_{\bar q,n} \leq K_n^{\bar q}$ provided that $\bar q \leq q/2$. However, $B_{\bar q,n}$ can be bounded in various settings if $|S|$ is bounded.

Theorem (ref) yields simultaneous confidence regions that are uniformly valid across models that satisfy Condition IV and the additional growth conditions on the $n$, $\log |S|$ and various moments. In particular, they allow for (sequence of) models where model selection mistakes are unavoidable.

Monte Carlo Experiments

In this section we investigate the finite sample performance of the proposed estimators via numerical simulations. We consider the following data generating process $$ y = x'\beta_0 + \varepsilon, \ \ \ x_j = \tilde x_j + \sum_{\ell=1}^Lz_{L(j-1)+k}, \ \ \ \varepsilon = \zeta + \tilde x'\gamma_0 $$ where $\beta_0=(1,.8,.6,.4,.2,0,\ldots)'$, $\gamma_0=(.1,.2,.3,.4,.5,.6,.7,.8,.9,1,0,\ldots)'$, $z \sim N(0,I_{Fp})$, $\tilde x \sim N(0,\Sigma)$, $\Sigma_{ij}=0.3^{|i-j|}$, and $\zeta \sim N(0,1/4^2)$. This corresponds to the case that $L$ instruments are useful for each covariate.

The table above illustrates how the proposed procedure accurately controls size where the confidence region aims to simultaneously cover the first three components. The table also reports the bias in the estimation of each component. To achieve correct size in this high-dimensional settings it is critical to obtain a low bias and this is what the table reports.

{\bf Acknowledgement.} We thank Victor Chernozhukov and Denis Chetverikov for several comments and suggestions.