Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
111,205 characters · 18 sections · 4 citation commands
Sparse Models and Methods for Optimal Instruments with an Application to Eminent Domain
Instrumental variables (IV) techniques are widely used in applied economic research. While these methods provide a useful tool for identifying structural effects of interest, their application often results in imprecise inference. One way to improve the precision of instrumental variables estimators is to use many instruments or to try to approximate the optimal instruments as in \citeasnoun{amemiya:optimalIV}, \citeasnoun{chamberlain}, and \citeasnoun{newey:optimaliv}. Estimation of optimal instruments will generally be done nonparametrically and thus implicitly makes use of many constructed instruments such as polynomials. The promised improvement in efficiency is appealing, but IV estimators based on many instruments may have poor properties. See, for example, \citeasnoun{bekker}, \citeasnoun{chao:swanson:05}, \citeasnoun{hhn:weakiv}, and \citeasnoun{neweyetal:hetmanyiv} which propose solutions for this problem based on “many-instrument” asymptotics.\footnote{It is important to note that the precise definition of “many-instrument" is $p \propto n$ with $p< n$ where $p$ is the number of instruments and $n$ is the sample size. The current paper allows for this case and also for “very many-instrument" asymptotics where $p \gg n$.}
In this paper, we contribute to the literature on IV estimation with many instruments by considering the use of Lasso and Post-Lasso for estimating the first-stage regression of endogenous variables on the instruments. Lasso is a widely used method that acts both as an estimator of regression functions and as a model selection device. Lasso solves for regression coefficients by minimizing the sum of the usual least squares objective function and a penalty for model size through the sum of the absolute values of the coefficients. The resulting Lasso estimator selects instruments and estimates the first-stage regression coefficients via a shrinkage procedure. The Post-Lasso estimator discards the Lasso coefficient estimates and uses the data-dependent set of instruments selected by Lasso to refit the first stage regression via OLS to alleviate Lasso's shrinkage bias. For theoretical and simulation evidence regarding Lasso's performance, see Bai and Ng BaiNg2008,BaiNg2009b, \citeasnoun{BickelRitovTsybakov2009}, Bunea, Tsybakov, and Wegkamp BuneaTsybakovWegkamp2006,BuneaTsybakovWegkamp2007b,BuneaTsybakovWegkamp2007, \citeasnoun{CandesTao2007}, \citeasnoun{horowitz:lasso}, \citeasnoun{knight:shrinkage}, \citeasnoun{Koltchinskii2009}, \citeasnoun{Lounici2008}, \citeasnoun{LouniciPontilTsybakovvandeGeer2009}, \citeasnoun{MY2007}, \citeasnoun{RosenbaumTsybakov2008}, \citeasnoun{T1996}, \citeasnoun{vdGeer}, \citeasnoun{Wainright2006}, \citeasnoun{ZhangHuang2006}, \citeasnoun{BC-PostLASSO}, and \citeasnoun{BuhlmannGeer2011} among many others. See \citeasnoun{BC-PostLASSO} for analogous results on Post-Lasso.
Using Lasso-based methods to form first-stage predictions in IV estimation provides a practical approach to obtaining the efficiency gains from using optimal instruments while dampening the problems associated with many instruments. We show that Lasso-based procedures produce first-stage predictions that provide good approximations to the optimal instruments even when the number of available instruments is much larger than the sample size when the first-stage is approximately sparse -- that is, when there exists a relatively small set of important instruments whose identities are unknown that well-approximate the conditional expectation of the endogenous variables given the instruments. Under approximate sparsity, estimating the first-stage relationship using Lasso-based procedures produces IV estimators that are root-n consistent and asymptotically normal. The IV estimator with Lasso-based first stage also achieves the semi-parametric efficiency bound under the additional condition that structural errors are homoscedastic. Our results allow imperfect model selection and do not impose “beta-min” conditions that restrict the minimum allowable magnitude of the coefficients on relevant regressors. We also provide a consistent asymptotic variance estimator. Thus, our results generalize the IV procedure of \citeasnoun{newey:optimaliv} and \citeasnoun{hahn:SeriesRate} based on conventional series approximation of the optimal instruments. Our results also generalize \citeasnoun{BickelRitovTsybakov2009} by providing inference and confidence sets for the second-stage IV estimator based on Lasso or Post-Lasso estimates of the first-stage predictions. To our knowledge, our result is the first to verify root-n consistency and asymptotic normality of an estimator for a low-dimensional structural parameter in a high-dimensional setting without imposing the very restrictive “beta-min" condition.\footnote{The “beta-min" condition requires the relevant coefficients in the regression to be separated from zero by a factor that exceeds the potential estimation error. This condition implies the identities of the relevant regressors may be perfectly determined. There is a large body of theoretical work that uses such a condition and thus implicitly assumes that the resulting post-model selection estimator is the same as the oracle estimator that knows the identities of the relevant regressors. See \citeasnoun{BuhlmannGeer2011} for the discussion of the “beta-min" condition and the theoretical role it plays in obtaining “oracle" results.} Our results also remain valid in the presence of heteroscedasticity and thus provide a useful complement to existing approaches in the many instrument literature which often rely on homoscedasticity and may be inconsistent in the presence of heteroscedasticity; see \citeasnoun{neweyetal:hetmanyiv} for a notable exception that allows for heteroscedasticity and gives additional discussion.
Instrument selection procedures complement existing/traditional methods that are meant to be robust to many-instruments but are not a universal solution to the many instruments problem. The good performance of instrument selection procedures relies on approximate sparsity. Unlike traditional IV methods, instrument selection procedures do not require the identity of these “important” variables to be known a priori as the identity of these instruments will be estimated from the data. This flexibility comes with the cost that instrument selection will tend not to work well when the first-stage is not approximately sparse. When approximate sparsity breaks down, instrument selection procedures may select too few or no instruments or may select too many instruments. Two scenarios where this failure is likely to occur are the weak-instrument case; e.g. \citeasnoun{ss:weakiv}, \citeasnoun{ams:optimal}, \citeasnoun{andrews:stock}, \citeasnoun{moreira:weakivlr}, \citeasnoun{kleib:weakiv}, and \citeasnoun{kleib:weakgmm}; and the many-weak-instrument case; e.g. \citeasnoun{bekker}, \citeasnoun{chao:swanson:05}, \citeasnoun{hhn:weakiv}, and \citeasnoun{neweyetal:hetmanyiv}. We consider two modifications of our basic procedure aimed at alleviating these concerns. In Section 4, we present a sup-score testing procedure that is related to \citeasnoun{anderson:rubin} and \citeasnoun{ss:weakiv} but is better suited to cases with very many instruments; and we consider a split sample IV estimator in Section 5 which combines instrument selection via Lasso with the sample-splitting method of \citeasnoun{AngristKruegerSplitSample1995}. While these two procedures are steps toward addressing weak identification concerns with very many instruments, further exploration of the interplay between weak-instrument or many-weak-instrument methods and variable selection would be an interesting avenue for additional research.
Our paper also contributes to the growing literature on Lasso-based methods by providing results for Lasso-based estimators of nonparametric conditional expectations. We consider a modified Lasso estimator with penalty weights designed to deal with non-Gaussianity and heteroscedastic errors. This new construction allows us to innovatively use the results of moderate deviation theory for self-normalized sums of \citeasnoun{jing:etal} to provide convergence rates for Lasso and Post-Lasso. The derived convergence rates are as sharp as in the homoscedastic Gaussian case under the weak condition that the log of the number of regressors $p$ is small relative to $n^{1/3}$, i.e. $\log p = o(n^{1/3})$. Our construction generalizes the standard Lasso estimator of \citeasnoun{T1996} and allows us to generalize the Lasso results of \citeasnoun{BickelRitovTsybakov2009} and Post-Lasso results of \citeasnoun{BC-PostLASSO} both of which assume homoscedasticity and Gaussianity. The construction as well as theoretical results are important for applied economic analysis where researchers are concerned about heteroscedasticity and non-Gaussianity in their data. We also provide a data-driven method for choosing the penalty that must be specified to obtain Lasso and Post-Lasso estimates, and we establish its asymptotic validity allowing for non-Gaussian, heteroscedastic disturbances. Ours is the first paper to provide such a data-driven penalty which was previously not available even in the Gaussian case.\footnote{One exception is the work of \citeasnoun{BCW-SqLASSO} which considers square-root-Lasso estimators and shows that their use allows for pivotal penalty choices. Those results strongly rely on homoscedasticity.} These results are of independent interest in a variety of theoretical and applied settings.
We illustrate the performance of Lasso-based IV through simulation experiments. In these experiments, we find that a feasible Lasso-based procedure that uses our data-driven penalty performs well across a range of simulation designs where sparsity is a reasonable approximation. In terms of estimation risk, it outperforms the estimator of \citeasnoun{fuller} (FULL),\footnote{Note that this procedure is only applicable when the number of instruments $p$ is less than the sample size $n$. As mentioned earlier, procedures developed in this paper allow for $p$ to be much larger $n$.} which is robust to many instruments (e.g. \citename{hhn:weakiv}, \citeyear*{hhn:weakiv}), except in a design where sparsity breaks down and the sample size is large relative to the number of instruments. In terms of size of 5% level tests, the Lasso-based IV estimator performs comparably to or better than FULL in all cases we consider. Overall, the simulation results are in line with the theory and favorable to the proposed Lasso-based IV procedures.
Finally, we demonstrate the potential gains of the Lasso-based procedure in an application where there are many available instruments among which there is not a clear a priori way to decide which instruments to use. We look at the effect of judicial decisions at the federal circuit court level regarding the government's exercise of eminent domain on house prices and state-level GDP as in \citeasnoun{chen:yeh:takings}. We follow the identification strategy of \citeasnoun{chen:yeh:takings} who use the random assignment of judges to three judge panels that are then assigned to eminent domain cases to justify using the demographic characteristics of the judges on the realized panels as instruments for their decision. This strategy produces a situation in which there are many potential instruments in that all possible sets of characteristics of the three judge panel are valid instruments. We find that the Lasso-based estimates using the data-dependent penalty produce much larger first-stage Wald-statistics and generally have smaller estimated second stage standard errors than estimates obtained using the baseline instruments of \citeasnoun{chen:yeh:takings}.
Relationship to econometric literature on variable selection and shrinkage. The idea of instrument selection goes back to \citeasnoun{KloekMennes1960} and \citeasnoun{Amemiya1966} who searched among principal components to approximate the optimal instruments. Related ideas appear in dynamic factor models as in \citeasnoun{BaiNg2010}, \citeasnoun{KapetaniosMarcellino2010}, and \citeasnoun{KapetaniosKhalafMarcellino2011}. Factor analysis differs from our approach though principal components, factors, ridge fits, and other functions of the instruments could be considered among the set of potential instruments to select from.\footnote{Approximate sparsity should be understood to be relative to a given structure defined by the set of instruments considered. Allowing for principle components or ridge fits among the potential regressors considerably expands the applicability of the approximately sparse framework.}
There are several other papers that explore the use of modern variable selection methods in econometrics, including some papers that apply these procedures to IV estimation. \citeasnoun{BaiNg2009a} consider an approach to instrument selection that is closely related to ours based on boosting. The latter method is distinct from Lasso, cf. \citeasnoun{BuhlmannBoosting2006}, but it also does not rely on knowing the identity of the most important instruments. They show through simulation examples that instrument selection via boosting works well in the designs they consider but do not provide formal results. \citeasnoun{BaiNg2009a} also expressly mention the idea of using the Lasso method for instrument selection, though they focus their analysis on the boosting method. Our paper complements their analysis by providing a formal set of conditions under which Lasso variable selection will provide good first-stage predictions and providing theoretical estimation and inference results for the resulting IV estimator. One of our theoretical results for the IV estimator is also sufficiently general to cover the use of any other first-stage variable selection procedure, including boosting, that satisfies a set of provided rate conditions. \citeasnoun{caner:LGMM} considers estimation by penalizing the GMM criterion function by the $\ell_{\gamma}$-norm of the coefficients for $0 < \gamma < 1$. The analysis of \citeasnoun{caner:LGMM} assumes that the number of parameters $p$ is fixed in relation to the sample size, and so it is complementary to our approach where we allow $p \to \infty$ as $n \to \infty$. Other uses of Lasso in econometrics include \citeasnoun{BaiNg2008}, \citeasnoun{BCH-PLM}, \citeasnoun{Brodie:etal2009}, \citeasnoun{DeMiguel2009}, \citeasnoun{horowitz:lasso}, \citeasnoun{knight:shrinkage}, and others. An introductory treatment of this topic is given in \citeasnoun{BC-Intro}, and \citeasnoun{BCC-Review} provides a review of Lasso targeted at economic applications.
Our paper is also related to other shrinkage-based approaches to dealing with many instruments. \citeasnoun{chamberlain:imbens:reiv} considers IV estimation with many instruments using a shrinkage estimator based on putting a random coefficients structure over the first-stage coefficients in a homoscedastic setting. In a related approach, \citeasnoun{okui:manyiv} considers the use of ridge regression for estimating the first-stage regression in a homoscedastic framework where the instruments may be ordered in terms of relevance. \citeasnoun{okui:manyiv} derives the asymptotic distribution of the resulting IV estimator and provides a method for choosing the ridge regression smoothing parameter that minimizes the higher-order asymptotic mean-squared-error (MSE) of the IV estimator. These two approaches are related to the approach we pursue in this paper in that both use shrinkage in estimating the first-stage but differ in the shrinkage methods they use. Their results are also only supplied in the context of homoscedastic models. \citeasnoun{donald:newey} consider a variable selection procedure that minimizes higher-order asymptotic MSE which relies on a priori knowledge that allows one to order the instruments in terms of instrument strength. Our use of Lasso as a variable selection technique does not require any a priori knowledge about the identity of the most relevant instruments and so provides a useful complement to \citeasnoun{donald:newey} and \citeasnoun{okui:manyiv}. \citeasnoun{carrasco:regularizedIV} provides an interesting approach to IV estimation with many instruments based on directly regularizing the inverse that appears in the definition of the 2SLS estimator; see also \citeasnoun{carrasco:LIML}. \citeasnoun{carrasco:regularizedIV} considers three regularization schemes, including Tikhohov regularization which corresponds to ridge regression, and shows that the regularized estimators achieve the semi-parametric efficiency bound under some conditions. \citeasnoun{carrasco:regularizedIV}'s approach implicitly uses $\ell_2$-norm penalization and hence differs from and complements our approach. A valuable feature of \citeasnoun{carrasco:regularizedIV} is the provision of a data-dependent method for choosing the regularization parameter based on minimizing higher-order asymptotic MSE following \citeasnoun{donald:newey} and \citeasnoun{okui:manyiv}. Finally, in work that is more recent that the present paper, \citeasnoun{GautierTsybakovHDIV} consider the important case where the structural equation in an instrumental variables model is itself very high-dimensional and propose a new estimation method related to the Dantzig selector and the square-root-Lasso. They also provide an interesting inference method which differs from the one we consider.
Notation. In what follows, we work with triangular array data $\{\(z_{i,n}, i=1,...,n\right), n=1,2,3,...\}$ defined on some common probability space $(\Omega, \mathcal{A}, \mathrm{P})$. Each $z_{i,n}= (y_{i,n}', x_{i,n}', d_{i,n}')'$ is a vector, with components defined below in what follows, and these vectors are i.n.i.d. -- independent across $i$, but not necessarily identically distributed. The law ${\mathrm{P}}_n$ of $\{z_{i,n}, i=1,...,n\}$ can change with $n$, though we do not make explicit use of ${\mathrm{P}}_n$. Thus, all parameters that characterize the distribution of $\{z_{i,n}, i=1,...,n\}$ are implicitly indexed by the sample size $n$, but we omit the index $n$ in what follows to simplify notation. We use triangular array asymptotics to better capture some finite-sample phenomena and to retain the robustness of conclusions to perturbations of the data-generating process. We also use the following empirical process notation, ${\mathbb{E}_n}[f] := {\mathbb{E}_n}[f(z_i)] := \sum_{i=1}^n f(z_i)/n$, and $\mathbb{G}_n(f) := \sum_{i=1}^n ( f(z_i) - {\mathrm{E}}[f(z_i)] )/\sqrt{n}.$ Since we want to deal with i.n.i.d. data, we also introduce the average expectation operator: $ \bar {\mathrm{E}}[f] := {\mathrm{E}} {\mathbb{E}_n}[f] = {\mathrm{E}} {\mathbb{E}_n}[f(z_i)] = \sum_{i=1}^n {\mathrm{E}}[f(z_i)]/n. $ The $\ell_2$-norm is denoted by $\|\cdot\|_2$, and the $\ell_0$-norm, $\|\cdot\|_0$, denotes the number of non-zero components of a vector. We use $\|\cdot\|_{\infty}$ to denote the maximal element of a vector. The empirical $L^2(\mathbb{P}_n)$ norm of a random variable $W_i$ is defined as $ \|W_i\|_{2,n} := \sqrt{{\mathbb{E}_n}[ W_i^2]}. $ When the empirical $L^2(\mathbb{P}_n)$ norm is applied to regressors $f_1,\ldots,f_p$ and a vector $\delta\in {\mathbb{R}}^p$, $\|f_i'\delta\|_{2,n}$, it is called the prediction norm. Given a vector $\delta \in {\mathbb{R}}^p$ and a set of indices $T \subset \{1,\ldots,p\}$, we denote by $\delta_T$ the vector in which $\delta_{Tj} = \delta_j$ if $j\in T$, $\delta_{Tj}=0$ if $j \notin T$. We also denote $T^c:=\{1,2,\ldots,p\}\setminus T$. We use the notation $(a)_+ = \max\{a,0\}$, $a \vee b = \max\{ a, b\}$ and $a \wedge b = \min\{ a , b \}$. We also use the notation $a \lesssim b$ to denote $a \leqslant c b$ for some constant $c>0$ that does not depend on $n$; and $a\lesssim_{\mathrm{P}} b$ to denote $a=O_{\mathrm{P}}(b)$. For an event $E$, we say that $E$ wp $\to$ 1 when $E$ occurs with probability approaching one as $n$ grows. We say $X_n =_d Y_n + o_{\mathrm{P}}(1)$ to mean that $X_n$ has the same distribution as $Y_n$ up to a term $o_{\mathrm{P}}(1)$ that vanishes in probability.
In this section of the paper, we present the model and provide an overview of the main results. Sections 3 and 4 provide a technical presentation that includes a set of sufficient regularity conditions, discusses their plausibility, and establishes the main formal results of the paper.
The model is $y_i = {d}_i'\alpha_0 + \epsilon_i$ where $\alpha_0$ denotes the true value of a vector-valued parameter $\alpha$. $y_i$ is the response variable, and $d_i$ is a finite $k_d$-vector of variables whose first $k_e$ elements contain endogenous variables. The disturbance $\epsilon_i$ obeys for all $i$ (and $n$): $$ {\mathrm{E}}[\epsilon_i|{x}_i] = 0, $$ where ${x}_i$ is a $k_{{x}}$-vector of instrumental variables.
As a motivation, suppose that the structural disturbance is conditionally homoscedastic, namely, for all $i$, $ {\mathrm{E}}[\epsilon_i^2|{x}_i] = \sigma^2. $ Given a $k_d$-vector of instruments $A({x}_i)$, the standard IV estimator of $\alpha_0$ is given by $ \widehat \alpha = ({\mathbb{E}_n}[A({x}_i)d_i'])^{-1} {\mathbb{E}_n} [A({x}_i)y_i], $ where $\{({x}_i, d_i, y_i), i=1,...,n\}$ is an i.i.d. sample from the IV model above. For a given $A({x}_i)$, $ \sqrt{n}(\widehat \alpha - \alpha_0) =_d N(0, Q^{-1}_0 \Omega_0 {Q_0^{-1}}' ) + o_{\mathrm{P}}(1)$, where $Q_0 = \bar {\mathrm{E}} [A({x}_i)d_i']$ and $\Omega_0 = \sigma^2 \bar {\mathrm{E}} [A({x}_i) A({x}_i)']$ under standard conditions. Setting $ A({x}_i) = D({x}_i) = {\mathrm{E}}[d_i|{x}_i]$ minimizes the asymptotic variance which becomes $$\Lambda^* = \sigma^2 \{\bar {\mathrm{E}} [D({x}_i) D({x}_i)'] \}^{-1},$$ the semi-parametric efficiency bound for estimating $\alpha_0$; see \citeasnoun{amemiya:optimalIV}, \citeasnoun{chamberlain}, and \citeasnoun{newey:optimaliv}. In practice, the optimal instrument $D(x_i)$ is an unknown function and has to be estimated. In what follows, we investigate the use of sparse methods -- namely Lasso and Post-Lasso -- for use in estimating the optimal instruments. The resulting IV estimator is asymptotically as efficient as the infeasible optimal IV estimator above.
Note that if $d_i$ contains exogenous components $w_i$, then $d_i = (d_{i1},..., d_{ik_e}, w_i')'$ where the first $k_e$ variables are endogenous. Since the rest of the components $w_i$ are exogenous, they appear in ${x}_i = (w_i', \tilde x_i')'$. It follows that $ D_i := D(x_i) := {\mathrm{E}}[d_i|{x}_i]= ({\mathrm{E}}[d_{1}|{x}_i],..., {\mathrm{E}}[d_{k_e}|{x}_i], w_i')'; $ i.e. the estimator of $w_i$ is simply $w_i$. Therefore, we discuss estimation of the conditional expectation functions: $$ D_{il} := D_{l}({x}_i) := {\mathrm{E}}[d_{{l}}|{x}_i], \ {l} =1,...,k_e. $$ In what follows, we focus on the strong instruments case which translates into the assumption that $Q = \bar {\mathrm{E}}[D({x}_i)D({x}_i)']$ has eigenvalues bounded away from zero and from above. We also present an inference procedure that remains valid in the absence of strong instruments which is related to \citeasnoun{anderson:rubin} and \citeasnoun{ss:weakiv} but allows for $p \gg n$.
Suppose there is a very large list of instruments, $$ f_i := (f_{i1},...,f_{ip})' := ( f_1({x}_i),...,f_p({x}_i))', $$ to be used in estimation of conditional expectations $D_{{l}}(x_i), \ {l} =1,...,k_e$, where the number of instruments $p$ is possibly much larger than the sample size $n$.
For example, high-dimensional instruments $f_i$ could arise as any combination of the following two cases. First, the list of available instruments may simply be large, in which case $f_i=x_i$ as in e.g. \citeasnoun{amemiya:optimalIV} and \citeasnoun{bekker}. Second, the list $f_i$ could consist of a large number of series terms with respect to some elementary regressor vector $x_i$; e.g., $f_i$ could be composed of B-splines, dummies, polynomials, and various interactions as in \citeasnoun{newey:optimaliv} or \citeasnoun{hahn:SeriesRate} among others. We term the first example the many instrument case and the second example the many series instrument case and note that our formulation does not require us to distinguish between the two cases. We mainly use the term “series instruments" and contrast our results with those in the seminal work of \citeasnoun{newey:optimaliv} and \citeasnoun{hahn:SeriesRate}, though our results are not limited to canonical series regressors as in \citeasnoun{newey:optimaliv} and \citeasnoun{hahn:SeriesRate}. The most important feature of our approach is that by allowing $p$ to be much larger than the sample size, we are able to consider many more series instruments than in \citeasnoun{newey:optimaliv} and \citeasnoun{hahn:SeriesRate} to approximate the optimal instruments.
The key assumption that allows effective use of this large set of instruments is sparsity. To fix ideas, consider the case where $D_{{l}}({x}_i)$ is a function of only $s \ll n$ instruments:
This simple sparsity model generalizes the classic parametric model of optimal instruments of \citeasnoun{amemiya:optimalIV} by letting the identities of the relevant instruments $ T_{{l}} = \text{support}(\beta_{l0})=\{ j \in \{1,\ldots,p\} \ : \ |\beta_{{l0} j}| > 0\} $ be unknown.
The model given by ((ref)) is unrealistic in that it presumes exact sparsity. We make no formal use of this model, but instead use a much more general approximately sparse or nonparametric model:
Condition AS.(Approximately Sparse Optimal Instrument). Each optimal instrument function $D_{{l}}({x}_i)$ is well-approximated by a function of unknown $s\geqslant 1$ instruments:
Condition AS is the key assumption. It requires that there are at most $s$ terms for each endogenous variable that are able to approximate the conditional expectation function $D_{{l}}({x}_i)$ up to approximation error $a_{{l}}(x_i)$ chosen to be no larger than the conjectured size $\sqrt{s/n}$ of the estimation error of the infeasible estimator that knows the identity of these important variables, the “oracle estimator.” In other words, the number $s$ is defined so that the approximation error is of the same order as the estimation error, $\sqrt{s/n}$, of the oracle estimator. Importantly, the assumption allows the identity $$ T_{{l}} = \text{support}(\beta_{l0})$$ to be unknown and to differ for $l=1,\ldots,k_e$.
For a detailed motivation and discussion of this assumption, we refer the reader to \citeasnoun{BCC-Review}. Condition AS generalizes the conventional series approximation of optimal instruments in Newey newey:optimaliv,newey:series and \citeasnoun{hahn:SeriesRate} by letting the identities of the most important $s$ series terms $T_{{l}}$ be unknown. The rate $\sqrt{s/n}$ generalizes the rate obtained with the optimal number $s$ of series terms in \citeasnoun{newey:optimaliv} for estimating conditional expectation by not relying on knowledge of what $s$ series terms to include. Knowing the identities of the most important series terms is unrealistic in many examples. The most important series terms need not be the first $s$ terms, and the optimal number of series terms to consider is also unknown. Moreover, an optimal approximation could come from the combination of completely different bases e.g by using both polynomials and B-splines.
Lasso and Post-Lasso use the data to estimate the set of the most relevant series terms in a manner that allows the resulting IV estimator to achieve good performance if a key growth condition, $$ \frac{s^2 \log^2 (p\vee n)}{n} \to 0, $$ holds along with other more technical conditions. The growth condition requires the optimal instruments to be sufficiently smooth so that a small (relative to $n$) number of series terms can be used to approximate them well. The use of a small set of instruments ensures that the impact of first-stage estimation on the IV estimator is asymptotically negligible. We can weaken this condition to $s \log (p\vee n) = o(n)$ by using the sample-splitting idea from the many instruments literature.
Let us write the first-stage regression equations as
Given the sample $\{({x}_i, d_{i{l}}, {l}=1,..., k_e), i=1,...,n\}$, we consider estimators of the optimal instrument $D_{il}:= D_{{l}}({x}_i)$ that take the form $$ \widehat D_{il}:=\widehat D_{l}({x}_i) = f_i'\widehat \beta_{l}, \ \ {l}=1,...,k_e, $$ where $\widehat \beta_{{l}}$ is the Lasso or Post-Lasso estimator obtained by using $d_{il}$ as the dependent variable and $f_i$ as regressors.
Consider the usual least squares criterion function: $$ \widehat Q_{{l}} (\beta):= {\mathbb{E}_n} [(d_{i{l}} - f_i'\beta)^2]. $$ The Lasso estimator is defined as a solution of the following optimization program:
where $\lambda$ is the penalty level and $\widehat \Upsilon_l = \text{diag}(\widehat \gamma_{l1},..., \widehat \gamma_{lp})$ is a diagonal matrix specifying penalty loadings.
Our analysis will first employ the following “ideal" penalty loadings: $$ \widehat \Upsilon_{{l}}^0 = \text{diag}(\widehat \gamma^0_{{l} 1},..., \widehat \gamma^0_{{l} p}), \ \widehat \gamma^0_{lj} = \sqrt{ {\mathbb{E}_n} [ f^2_{ij} v^2_{il}]}, \ j =1,...,p. $$ The ideal option is not feasible but leads to rather sharp theoretical bounds on estimation risk. This option is not feasible since $v_{il}$ is not observed. In practice, we estimate the ideal loadings by first using conservative penalty loadings and then plugging-in the resulting estimated residuals in place of $v_{il}$ to obtain the refined loadings. This procedure could be iterated via Algorithm A.1 stated in the appendix.
The idea behind the ideal penalty loading is to introduce self-normalization of the first-order condition of the Lasso problem by using data-dependent penalty loadings. This self-normalization allows us to apply moderate deviation theory of \citeasnoun{jing:etal} for self-normalized sums to bound deviations of the maximal element of the score vector $$S_{l} = 2 {\mathbb{E}_n} [(\widehat \Upsilon^{0}_l)^{-1} f_i v_{il}]$$ which provides a representation of the estimation noise in the problem. Specifically, the use of self-normalized moderate deviation theory allows us to establish that
from which we obtain sharp convergence results for the Lasso estimator under non-Gaussianity and heteroscedasticity. Without using these loadings, we may not be able to achieve the same sharp rate of convergence. It is important to emphasize that our construction of the penalty loadings for Lasso is new and differs from the canonical penalty loadings proposed in \citeasnoun{T1996} and \citeasnoun{BickelRitovTsybakov2009}. Finally, to insure the good performance of the Lasso estimator, one needs to select the penalty level $\lambda/n$ to dominate the noise for all $k_e$ regression problems simultaneously; i.e. the penalty level should satisfy
for some constant $c>1$. The bound ((ref)) suggests that this can be achieved by selecting
which implements ((ref)). Our current recommendation is to set the confidence level $\gamma = 0.1/\log(p\vee n)$ and the constant $c = 1.1$.\footnote{We note that there is not much room to change $c$. Theoretically, we require $c>1$, and finite-sample experiments show that increasing $c$ away from $c=1$ worsens the performance. Hence a value slightly above unity, namely $c=1.1$, is our current recommendation. The simulation evidence suggests that setting $c$ to any value near $1$, including $c=1$, does not impact the result noticeably.}
The Post-Lasso estimator is defined as the ordinary least square regression applied to the model $\widehat I_l \supseteq \widehat T_l$ where $\widehat T_l$ is the model selected by Lasso: $$\widehat T_{{l}} = {\rm support}( \widehat \beta_{{l} L} ) = \{ j \in \{1,\ldots,p\} \ : \ |\widehat\beta_{{l} L j }| > 0\}, \ \ {l} =1,..., k_e.$$ The set $\widehat I_l$ can contain additional variables not selected by Lasso, but we require the number of such variables to be similar to or smaller than the number selected by Lasso. The Post-Lasso estimator $\widehat\beta_{{l} PL}$ is
In words, this estimator is ordinary least squares (OLS) using only the instruments/regressors whose coefficients were estimated to be non-zero by Lasso and any additional variables the researcher feels are important despite having Lasso coefficient estimates of zero.
Lasso and Post-Lasso are motivated by the desire to predict the target function well without overfitting. Clearly, the OLS estimator is not consistent for estimating the target function when $p > n$. Some approaches based on BIC-penalization of model size are consistent but computationally infeasible. The Lasso estimator of \citeasnoun{T1996} resolves these difficulties by penalizing model size through the sum of absolute parameter values. The Lasso estimator is computationally attractive because it minimizes a convex function. Moreover, under suitable conditions, this estimator achieves near-optimal rates in estimating the regression function $D_{{l}}(x_i)$. The estimator achieves these rates by adapting to the unknown smoothness or sparsity of $D_{l}(x_i)$. Nonetheless, the estimator has an important drawback: The regularization by the $\ell_1$-norm employed in ((ref)) naturally lets the Lasso estimator avoid overfitting the data, but it also shrinks the estimated coefficients towards zero causing a potentially significant bias. The Post-Lasso estimator is meant to remove some of this shrinkage bias. If model selection by Lasso works perfectly -- that is, if it selects exactly the “relevant" instruments -- then the resulting Post-Lasso estimator is simply the standard OLS estimator using only the relevant variables. In cases where perfect selection does not occur, Post-Lasso estimates of coefficients will still tend to be less biased than Lasso. We prove the Post-Lasso estimator achieves the same rate of convergence as Lasso, which is a near-optimal rate, despite imperfect model selection by Lasso.
The introduction of self-normalization via the penalty loadings allows us to contribute to the broad Lasso literature cited in the introduction by showing that under possibly heteroscedastic and non-Gaussian errors the Lasso and Post-Lasso estimators obey the following near-oracle performance bounds:
The performance bounds in ((ref)) are called near-oracle because they coincide up to a $\sqrt{\log p}$ factor with the bounds achievable when the the ideal series terms $T_l$ for each of the $k_e$ regressions equations in ((ref)) are known. Our results extend those of \citeasnoun{BickelRitovTsybakov2009} for Lasso with Gaussian errors and those of \citeasnoun{BC-PostLASSO} for Post-Lasso with Gaussian errors. Notably, these bounds are as sharp as the results for the Gaussian case under the weak condition $\log p = o(n^{1/3})$. They are also the first results in the literature that allow for data-driven choice of the penalty level.
It is also useful to contrast the rates given in ((ref)) with the rates available for nonparametrically estimating conditional expectations in the series literature; see, for example, \citeasnoun{newey:series}. Obtaining rates of convergence for series estimators relies on approximate sparsity just as our results do. Approximate sparsity in the series context is typically motivated by smoothness assumptions, but approximate sparsity is more general than typical smoothness assumptions.\footnote{See, e.g., \citeasnoun{BCC-Review} and \citeasnoun{BCH-PLM} for detailed discussion of approximate sparsity.} The standard series approach postulates that the first $K$ series terms are the most important for approximating the target regression function $D_{il}$. The Lasso approach postulates that $s$ terms from a large number $p$ of terms are important but does not require knowledge of the identity of these terms or the number of terms, $s$, needed to approximate the target function well-enough that approximation errors are small relative to estimation error. Lasso methods estimate both the optimal number of series terms $s$ as well as the identities of these terms and thus automatically adapt to the unknown sparsity (or smoothness) of the true optimal instrument (conditional expectation). This behavior differs sharply from standard series procedures that do not adapt to the unknown sparsity of the target function unless the number of series terms is chosen by a model selection method. Lasso-based methods may also provide enhanced approximation of the optimal instrument by allowing selection of the most important terms from a among a set of very many series terms with total number of terms $p \gg K$ that can be much larger than the sample size.\footnote{We can allow for $p \gg n$ for series formed with orthonormal bases with bounded components, such as trigonometric bases, but further restrictions on the number of terms apply if bounds on components of the series are allowed to increase with the sample size. For example, if we work with B-spline series terms, we can only consider $p = o(n)$ terms. } For example, a standard series approach based on $K$ terms will perform poorely when the terms $m+1$, $m+2$,...,$m+j$ are the most important for approximating the optimal instrument for any $K < m$. On the other hand, lasso-based methods will find the important terms as long as $p > m+j$ which is much less stringent than what is required in usual series approaches since $p$ can be very large. This point can also be made using the array asymptotics where the model changes with $n$ in such a way that the important series terms are always missed by the first $K \rightarrow \infty$ terms. Of course, the additional flexibility allowed for by Lasso-based methods comes with a price, namely slowing the rate of convergence by $\sqrt{\log p}$ relative to the usual series rates.
Given Condition AS, we take advantage of the approximate sparsity by using Lasso and Post-Lasso methods to construct estimates of $D_{{l}}(x_i)$ of the form $$ \widehat D_{{l}}(x_i) = f_i'\widehat \beta_{{l}}, \ \ {l} =1,...,k_e, $$ and then set $$ \widehat D_i = ( \widehat D_{1}(x_i), ..., \widehat D_{k_{e}}(x_i), w_i')'. $$ The resulting IV estimator takes the form $$ \widehat \alpha = {\mathbb{E}_n} [\widehat D_id_i']^{-1} {\mathbb{E}_n} [\widehat D_i y_i]. $$
The main result of this paper is to show that, despite the possibility of $p$ being very large, Lasso and Post-Lasso can select a set of instruments to produce estimates of the optimal instruments $\widehat D_i$ such that the resulting IV estimator achieves the efficiency bound asymptotically: $$ \sqrt{n}(\widehat \alpha - \alpha_0) =_d N(0, \Lambda^*) + o_{\mathrm{P}}(1). $$ The estimator matches the performance of the classical/standard series-based IV estimator of \citeasnoun{newey:optimaliv} and has additional advantages mentioned in the previous subsection. We also show that the IV estimator with Lasso-based optimal instruments continues to be root-$n$ consistent and asymptotically normal in the presence of heteroscedasticity:
where $\Omega := \bar {\mathrm{E}}[\epsilon_i^2 D({x}_i) D({x}_i)']$ and $Q:= \bar {\mathrm{E}}[D({x}_i) D({x}_i)']$. A consistent estimator for the asymptotic variance is
where $ \widehat \epsilon_i := y_i-d_i'\widehat\alpha$, $i=1,\ldots,n$. Using ((ref)) we can perform robust inference.
We note that our result ((ref)) for the IV estimator do not rely on the Lasso and Lasso-based procedure specifically. We provide the properties of the IV estimator for any generic sparsity-based procedure that achieves the near-oracle performance bounds ((ref)).
We conclude by stressing that our result ((ref)) does not rely on perfect model selection. Perfect model selection only occurs in extremely limited circumstances that are unlikely to occur in practice. We show that model selection mistakes do not affect the asymptotic distribution of the IV estimator $\widehat \alpha$ under mild regularity conditions. The intuition is that the model selection mistakes are sufficiently small to allow the Lasso or Post-Lasso to estimate the first stage predictions with a sufficient, near-oracle accuracy, which translates to the result above. Using analysis like that given in \citeasnoun{BCH-PLM}, the result ((ref)) can be shown to hold over models with strong optimal instruments which are uniformly approximately sparse. We also offer an inference test procedure in Section 4.2 that remains valid in the absence of a strong optimal instrument, is robust to many weak instruments, and can be used even if $p \gg n$. This procedure could also be shown to be uniformly valid over a large class of models.
In this section, we present our main results on Lasso and Post-Lasso estimators of conditional expectation functions under non-classical assumptions and data-driven penalty choices. The problem we analyze in this section has many applications outside the IV framework of the present paper.
The key condition concerns the behavior of the empirical Gram matrix ${\mathbb{E}_n}[f_if_i']$. This matrix is necessarily singular when $p > n$, so in principle it is not well-behaved. However, we only need good behavior of certain moduli of continuity of the Gram matrix. The first modulus of continuity is called the restricted eigenvalue and is needed for Lasso. The second modulus is called the sparse eigenvalue and is needed for Post-Lasso.
In order to define the restricted eigenvalue, first define the restricted set: $$\Delta_{C,T} = \{\delta \in \mathbb{R}^p: \|\delta_{T^c}\|_{1} \leqslant C \| \delta_{T}\|_{1}, \delta \neq 0\}.$$ The restricted eigenvalue of a Gram matrix $M = {\mathbb{E}_n}[f_if_i']$ takes the form:
This restricted eigenvalue can depend on $n$, but we suppress the dependence in our notation.
In making simplified asymptotic statements involving the Lasso estimator, we will invoke the following condition:
Condition RE. For any $C>0$, there exists a finite constant $\kappa> 0$, which does not depend on $n$ but may depend on $C$, such that the restricted eigenvalue obeys $\kappa_{C}({\mathbb{E}_n} [f_i f_i']) \geqslant \kappa$ with probability approaching one as $n \to \infty$.
The restricted eigenvalue ((ref)) is a variant of the restricted eigenvalues introduced in \citeasnoun{BickelRitovTsybakov2009} to analyze the properties of Lasso in the classical Gaussian regression model. Even though the minimal eigenvalue of the empirical Gram matrix ${\mathbb{E}_n}[f_if_i']$ is zero whenever $p \geqslant n$, \citeasnoun{BickelRitovTsybakov2009} show that its restricted eigenvalues can be bounded away from zero. Lemmas (ref) and (ref) below contain sufficient conditions for this. Many other sufficient conditions are available from the literature; see \citeasnoun{BickelRitovTsybakov2009}. Consequently, we take restricted eigenvalues as primitive quantities and Condition RE as a primitive condition.
In order to define the sparse eigenvalues, let us define the $m$-sparse subset of a unit sphere as $$ \Delta(m) = \{ \delta \in \mathbb{R}^p: \|\delta\|_0\leqslant m, \|\delta\|_2 = 1\}, $$ and also define the minimal and maximal $m$-sparse eigenvalue of the Gram matrix $M= {\mathbb{E}_n}\[f_if_i'\right]$ as $$ \phi_{{\rm min}}(m)(M) = \min_{\delta \in \Delta(m)} \delta'M\delta \ \ \mbox{and} \ \ \phi_{{\rm max}}(m)(M) = \max_{\delta \in \Delta(m)} \delta'M\delta. $$
To simplify asymptotic statements for Post-Lasso, we use the following condition:
Condition SE. For any $C>0$, there exist constants $0< \kappa' < \kappa'' < \infty$, which do not depend on $n$ but may depend on $C$, such that with probability approaching one, as $n \to \infty$, $\kappa' \leqslant \phi_{{\rm min}}(Cs)({\mathbb{E}_n}[f_if_i']) \leqslant \phi_{{\rm max}}(Cs)({\mathbb{E}_n}[f_if_i']) \leqslant \kappa''$.
Condition SE requires only that certain “small" $m \times m$ submatrices of the large $p \times p$ empirical Gram matrix are well-behaved, which is a reasonable assumption and will be sufficient for the results that follow. Condition SE implies Condition RE by the argument given in \citeasnoun{BickelRitovTsybakov2009}. The following lemmas show that Conditions RE and SE are plausible for both many-instrument and many series-instrument settings. We refer to \citeasnoun{BC-PostLASSO} for proofs; the first lemma builds upon results in \citeasnoun{ZhangHuang2006} and the second builds upon results in \citeasnoun{RudelsonVershynin2008}. The lemmas could also be derived from \citeasnoun{RudelsonZhou2011}.
In the context of i.i.d. sampling, a standard assumption in econometric research is that the population Gram matrix ${\mathrm{E}}[f_i f_i']$ has eigenvalues bounded from above and below, see e.g. \citeasnoun{newey:series}. The lemmas above allow for this and more general behavior, requiring only that the sparse eigenvalues of the population Gram matrix ${\mathrm{E}}[f_i f_i']$ are bounded from below and from above. The latter is important for allowing functions $f_i$ to be formed as a combination of elements from different bases, e.g. a combination of B-splines with polynomials. The lemmas above further show that the good behavior of the population sparse eigenvalues translates into good behavior of empirical sparse eigenvalues under some restrictions on the growth of $s$ in relation to the sample size $n$. For example, if $p$ grows polynomially with $n$ and the components of technical regressors are uniformly bounded, Lemma (ref) holds provided $s = o(n/\log^5 n)$.
We also impose the following moment conditions on the reduced form errors $v_{il}$ and regressors $f_i$, where we let $\tilde d_{il} := d_{il} - \bar {\mathrm{E}}[d_{il}]$.
Condition RF. (i) $ \max_{l \leqslant k_e, j \leqslant p} \ \bar {\mathrm{E}}[\tilde d^2_{il}] +\bar {\mathrm{E}}[ |f^2_{ij} \tilde d^2_{il}|] + 1/\bar {\mathrm{E}}[f_{ij}^2 v_{il}^2] \lesssim 1$, (ii) $ {\displaystyle \max_{l \leqslant k_e, j \leqslant p} } \bar {\mathrm{E}}[|f_{ij}^3 v_{il}^3|] \lesssim K_n$, (iii) $K_n^{2}\log^3 (p\vee n) = o(n) $ and $\ s \log (p\vee n) = o(n) $, (iv) $ \max_{i \leqslant n, j \leqslant p} f_{ij}^2 [s \log (p\vee n)]/n \to_{{\mathrm{P}}} 0$ and $ \max_{l \leqslant k_e, j \leqslant p} |({\mathbb{E}_n} - \bar {\mathrm{E}})[f_{ij}^2 v^2_{il}] | + | ({\mathbb{E}_n} - \bar {\mathrm{E}})[f^2_{ij}\tilde d^2_{il}]| \to_{{\mathrm{P}}} 0.$
We emphasize that the conditions given above are only one possible set of sufficient conditions, which are presented in a manner that reduces the complexity of the exposition.
The following lemma shows that the population and empirical moment conditions appearing in Condition RF are plausible for both many-instrument and many series-instrument settings. Note that we say that a random variable $g_{i}$ has uniformly bounded conditional moments of order $K$ if for some positive constants $ 0< B_1 < B_2< \infty$: $$ B_1 \leqslant {\mathrm{E}}\Big[ |g_{i}|^k \Big |{x}_i \Big ] \leqslant B_2 \text{ with probability } 1, \ \mbox{for} \ k = 1,\ldots, K, \ i=1, \ldots, n.$$
We consider Lasso and Post-Lasso estimators defined in equations ((ref)) and ((ref)) in the system of $k_e$ nonparametric regression equations ((ref)) with non-Gaussian and heteroscedastic errors. These results extend the previous results of \citeasnoun{BickelRitovTsybakov2009} for Lasso and of \citeasnoun{BC-PostLASSO} for Post-Lasso with classical i.i.d. errors. In addition, we account for the fact that we are simultaneously estimating $k_e$ regressions and account for the dependence of our results on $k_e$.
The following theorem presents the properties of Lasso. Let us call asymptotically valid any penalty loadings $\widehat \Upsilon_l $ that obey a.s.
with $ 0< \ell \leqslant 1 \leqslant u$ such that $\ell \to_{\mathrm{P}} 1$ and $u \to_{\mathrm{P}} u'$ with $u' \geqslant 1$. The penalty loadings constructed by Algorithm A.1 satisfy this condition.
The theorem provides a rate result for the Lasso estimator constructed specifically to deal with non-Gaussian errors and heteroscedasticity. The rate result generalizes, and is as sharp as, the rate results of \citeasnoun{BickelRitovTsybakov2009} obtained for the homoscedastic Gaussian case. This generalization is important for real applications where non-Gaussianity and heteroscedasticity are ubiquitous. Note that the obtained rate is near-optimal in the sense that if we happened to know the model $T_\ell$, i.e. if we knew the identities of the most important variables, we would only improve the rate by the $\log p$ factor. The theorem also shows that the data-driven penalty loadings defined in Algorithm A.1 are asymptotically valid.
The following theorem presents the properties of Post-Lasso which requires a mild assumption on the number of additional variables in the set $\widehat I_l$, $l=1,\ldots,k_e$. We assume that the size of these sets are not substantially larger than the model selected by Lasso, namely, a.s.
The theorem provides a rate result for the Post-Lasso estimator with non-Gaussian errors and heteroscedasticity. The rate result generalizes the results of \citeasnoun{BC-PostLASSO} obtained for the homoscedastic Gaussian case. The Post-Lasso achieves the same near-optimal rate of convergence as Lasso. As stressed in the introductory sections, our analysis allows Lasso to make model selection mistakes which is expected generically. We show that these model selection mistakes are small enough to allow the Post-Lasso estimator to perform as well as Lasso.\footnote{Under further conditions stated in proofs, Post-Lasso can sometimes achieve a faster rate of convergence. In special cases where perfect model selection is possible, Post-Lasso becomes the so-called oracle estimator and can completely remove the $\log p$ factor.}
Rates of convergence in different norms can also be of interest in other applications. In particular, the $\ell_2$-rate of convergence can be derived from the rate of convergence in the prediction norm and Condition SE using a sparsity result for Lasso established in Appendix (ref). Below we specialize the previous theorems to the important case that Condition SE holds.
The rates of convergence in the prediction norm and $\ell_2$-norm are faster than the rate of convergence in the $\ell_1$-norm which is typical of high dimensional settings.
In this section we present our main inferential results on instrumental variable estimators.
We impose the following moment conditions on the instruments, the structural errors, and regressors.
Condition SM. (i) The eigenvalues of $Q=\bar {\mathrm{E}}[D({x}_i)D({x}_i)']$ are bounded uniformly from above and away from zero, uniformly in $n$. The conditional variance ${\mathrm{E}}[\epsilon_i^2|{x}_i]$ is bounded uniformly from above and away from zero, uniformly in $i$ and $n$. Given this assumption, without loss of generality, we normalize the instruments so that $\bar {\mathrm{E}}[f_{ij}^2\epsilon_i^2] =1$ for each $1 \leqslant j \leqslant p$ and for all $n$. (ii) For some $q>2$ and $q_{\epsilon} > 2$, uniformly in $n$, $$ \max_{1\leqslant j \leqslant p}\bar {\mathrm{E}}[ |f_{ij}\epsilon_i|^3] + \bar {\mathrm{E}}[\|D_i\|^q_2 |\epsilon_i|^{2q}] + \bar {\mathrm{E}}[ \| D_i\|^q_2] + \bar {\mathrm{E}}[|\epsilon_i|^{q_{\epsilon}}] + \bar {\mathrm{E}}[ \|d_i\|^q_2] \lesssim 1. $$ (iii) In addition to $\log^3 p = o(n)$, the following growth conditions hold: $$(a) \ \frac{s \log (p\vee n) }{n} n^{2/q_{\epsilon}} \to 0 \ \ (b) \ \frac{s^2 \log^2 (p\vee n)}{n} \to 0, \ \ (c) \max_{1\leqslant j \leqslant p} {\mathbb{E}_n}[f^2_{ij} \epsilon^2_i] \lesssim_P 1. $$
The following lemma shows that the moment assumptions in Condition SM (iii) are plausible for both many-instrument and many series-instrument settings.
The first result describes the properties of the IV estimator with the optimal IV constructed using Lasso or Post-Lasso in the setting of the standard model. The result also provides a consistent estimator for the asymptotic variance of this estimator under heteroscedasticity.
In the setting with homoscedastic structural errors the estimator achieves the efficiency bound asymptotically. In the case of heteroscedastic structural errors, the estimator does not achieve the efficiency bound, but we can expect it to be close to achieving the bound if heteroscedasticity is mild.
The final result of this section extends the previous result to any IV-estimator with a generic sparse estimator of the optimal instruments.
This result shows that the previous two theorems apply for any first-stage estimator that attains near-oracle performance given in ((ref)). Examples of other sparse estimators covered by this theorem are Dantzig and Gauss-Dantzig (\citename{CandesTao2007}, \citeyear*{CandesTao2007}), $\sqrt{\textrm{Lasso}}$ and post-$\sqrt{\textrm{Lasso}}$ (\citename{BCW-SqLASSO}, \citeyear*{BCW-SqLASSO2} and \citeyear*{BCW-SqLASSO}), thresholded Lasso and Post-thresholded Lasso (\citename{BC-PostLASSO}, \citeyear*{BC-PostLASSO}), group Lasso and Post-group Lasso (\citename{horowitz:lasso}, \citeyear*{horowitz:lasso}; and \citename{LouniciPontilTsybakovvandeGeer2009}, \citeyear*{LouniciPontilTsybakovvandeGeer2009}), adaptive versions of the above (\citename{horowitz:lasso}, \citeyear*{horowitz:lasso}), and boosting (\citename{BuhlmannBoosting2006}, \citeyear*{BuhlmannBoosting2006}). Verification of the near-oracle performance ((ref)) can be done on a case by case basis using the best conditions in the literature.\footnote{Post-$\ell_1$-penalized procedures have only been analyzed for the case of Lasso and $\sqrt{\textrm{Lasso}}$; see \citeasnoun{BC-PostLASSO} and \citeasnoun{BCW-SqLASSO2}. We expect that similar results carry over to other procedures listed above.} Our results extend to Lasso-type estimators under alternative forms of regularity conditions that fall outside the framework of Conditions RE and Conditions RF; all that is required is the near-oracle performance of the kind ((ref)).
When instruments are weak individually, Lasso may end up selecting no instruments or may produce unreliable estimates of the optimal instruments. To cover this case, we propose a method for inference based on inverting pointwise tests performed using a sup-score statistic defined below. The procedure is similar in spirit to \citeasnoun{anderson:rubin} and \citeasnoun{ss:weakiv} but uses a different statistics that is better suited to cases with very many instruments. In order to describe the approach, we rewrite the main structural equation as:
where $y_{i}$ is the response variable, $d_{ei}$ is a vector of endogenous variables, $w_i$ is a $k_w$-vector of control variables, $x_i = (z_i',w_i')'$ is a vector of elementary instrumental variables, and $\epsilon_i$ is a disturbance such that $\epsilon_1,..., \epsilon_n$ are i.n.i.d. conditional on $X=[x_1',...,x_n']$. We partition $d_i = ({d_{ei}}', w_i')'$. The parameter of interest is $\alpha_1 \in \mathcal{A}_1 \subset \mathbb{R}^{k_e}$. We use $f_{i} = P(x_i)$, a vector which includes $w_i$, as technical instruments. In this subsection, we treat $X$ as fixed; i.e. we condition on $X$.
We would like to use a high-dimensional vector $f_i$ of technical instruments for inference on $\alpha_1$ that is robust to weak identification. In order to formulate a practical sup-score statistic, it is useful to partial-out the effect of $w_i$ on the key variables. For an $n$-vector $\{u_i, i=1,...,n\}$, define $\tilde u_{i} = u_{i} - w_{i}'{\mathbb{E}_n}[w_i w_i']^{-1} {\mathbb{E}_n}[w_i u_i]$, i.e. the residuals left after regressing this vector on $\{w_i, i=1,...,n\}$. Hence $\tilde y_i$, $\tilde d_{ei}$, and $\tilde f_{ij}$ are residuals obtained by partialling out controls $w_i$. Also, let
In this formulation, we omit elements of $w_i$ from $\tilde f_{ij}$ since they are eliminated by partialling out. We then normalize these technical instruments so that
The sup-score statistic for testing the hypothesis $\alpha_1 = a $ takes the form
As before, we apply self-normalized moderate deviation theory for self-normalized sums to obtain $$ {\mathrm{P}} ( \Lambda_{\alpha_1} \leqslant c \sqrt{n}\Phi^{-1}(1- \gamma/2p) ) \geqslant 1- \gamma + o(1). $$ Therefore, we can employ $\Lambda(1- \gamma):= c \sqrt{n}\Phi^{-1}(1- \gamma/2p)$ for $c > 1$ as a critical value for testing $\alpha_1 = a$ using $\Lambda_a$ as the test-statistic. The asymptotic $(1- \gamma)$ -- confidence region for $\alpha_1$ is then given by $ \mathcal{C}:=\{ a \in \mathcal{A}_1: \Lambda_a \leqslant \Lambda(1- \gamma)\}. $
The construction of confidence regions above can be given the following Inverse Lasso interpretation. Let $$ \widehat \beta_a \in \arg\min_{\beta \in \mathbb{R}^p} {\mathbb{E}_n}[ (\tilde y_{i} - \tilde d_{ei}'a) - \tilde f_{ij}'\beta]^2 + \frac{\lambda}{n} \sum_{j=1}^p \gamma_{aj} | \beta_j |, \ \ \gamma_{aj} = \sqrt{{\mathbb{E}_n}[ (\tilde y_{i} - \tilde d_{ei}'a)^2 \tilde f^2_{ij} ]}. $$ If $\lambda = 2\Lambda(1- \gamma)$, then $ \mathcal{C}$ is equivalent to the region $\{ a \in \mathbb{R}^{k_e}: \widehat \beta_a = 0\}$. In words, this confidence region collects all potential values of the structural parameter where the Lasso regression of the potential structural disturbance on the instruments yields zero coefficients on the instruments. This idea is akin to the Inverse Quantile Regression and Inverse Least Squares ideas in \citename{ch:iqrWeakId} ch:iqrWeakId,ch:WeakId.
Below, we state the main regularity condition for the validity of inference using the sup-score statistic as well as the formal inference result.
Condition SM2. Suppose that for each $n$ the linear model ((ref)) holds with $\alpha_1 \in \mathcal{A}_1 \subset \mathbb{R}^{k_e}$ such that $\epsilon_1,..., \epsilon_n$ are i.n.i.d., $X$ is fixed, and $\tilde f_1,... ,\tilde f_n$ are $p$-vectors of technical instruments defined in ((ref)) and ((ref)). Suppose that (i) the dimension of $w_i$ is $k_w$ and $\|w_i\|_2 \leqslant \zeta_w$ such that $\sqrt{k_w} \zeta_w/\sqrt{n} \to 0$, (ii) the eigenvalues of ${\mathbb{E}_n}[w_i w_i']$ are bounded away from zero and eigenvalues of $\bar {\mathrm{E}}[\epsilon_i^2w_i w_i']$ are bounded away from above, uniformly in $n$, (iii) $\max_{1 \leqslant j \leqslant p }\bar {\mathrm{E}}[|\epsilon_i|^3 |\tilde f_{ij}|^3] ^{1/3}/ \bar {\mathrm{E}}[\epsilon_i^2 \tilde f^2_{ij}] ^{1/2} \leqslant K_n$ , and (iv) $K^2_n \log (p\vee n) = o(n^{1/3})$.
The theorem shows that the confidence region $\mathcal{C}$ constructed above is valid in large samples and that the probability of including a false point $a$ in $\mathcal{C}$ tends to zero as long as $a$ is sufficiently distant from $\alpha_1$ and instruments are not too weak. In particular, if there is a strong instrument, the confidence regions will eventually exclude points $a$ that are further than $\sqrt{\log (p\vee n)/n}$ away from $\alpha_1$. Moreover, if there are instruments whose correlation with the endogenous variable is of greater order than $\sqrt{\log (p\vee n)/n}$, then the confidence regions will asymptotically be bounded.
In this section we provide further estimation and inference results. We develop an overidentification test which compares the IV-Lasso based estimates to estimates obtained using a baseline set of instruments. We also combine the IV selection using Lasso with a sample-splitting technique from the many instruments literature which allows us to relax the growth requirement on the number of relevant instruments.
Here we develop a Hausman-style specification test for the validity of the instrumental variables. Let $A_i =A(x_i)$ be a baseline set of instruments, with $\dim(A_i) \geqslant \dim(\alpha)=k_{\alpha}$ bounded. Let $\tilde \alpha$ be the baseline instrumental variable estimator based on these instruments: $$ \tilde \alpha = ({\mathbb{E}_n}[d_i A_i']{\mathbb{E}_n}[A_i A_i']^{-1}{\mathbb{E}_n}[A_i d_i'])^{-1}{\mathbb{E}_n}[d_i A_i'] {\mathbb{E}_n}[A_i A_i']^{-1} {\mathbb{E}_n}[A_i y_i]. $$ If the instrumental variable exclusion restriction is valid, then the unscaled difference between this estimator and the IV estimator $\widehat \alpha$ proposed in the previous sections should be small. If the exclusion restriction is not valid, the difference between $\tilde \alpha$ and $\widehat \alpha$ should be large. Therefore, we can reject the null hypothesis of instrument validity if the difference is large.
We formalize the test as follows. Suppose we care about $R'\alpha$ for some $k\times k_d$ matrix $R$ of $\text{rank}(R) = k$. For instance, we might care only about the first $k$ components of $\alpha$, in which case $R=[I_k \ 0]$ is a $k \times k_d$ matrix that selects the first $k$ coefficients of $\alpha$. Define the estimand for $\tilde \alpha$ as $$ \alpha = (\bar {\mathrm{E}}[d_i A_i']\bar {\mathrm{E}}[A_i A_i']^{-1}\bar {\mathrm{E}}[A_i d_i'])^{-1}\bar {\mathrm{E}}[d_i A_i'] \bar {\mathrm{E}}[A_i A_i']^{-1} \bar {\mathrm{E}}[A_i y_i], $$ and define the estimand of $\widehat \alpha$ as $$\alpha_a = \bar {\mathrm{E}}[ D({x}_i) D({x}_i)']^{-1} \bar {\mathrm{E}}[ D({x}_i) y_i].$$ The null hypothesis $H_0$ is $R(\alpha - \alpha_a)= 0$ and the alternative $H_a$ is $R(\alpha - \alpha_a) \neq 0$. We can form a test statistic $$ J= \sqrt{n} (\tilde \alpha - \widehat \alpha)'R' (R\widehat \Sigma R')^{-1} \sqrt{n} R(\tilde \alpha - \widehat \alpha) $$ for a matrix $\widehat \Sigma$ defined below and reject $H_0$ if $J > c_\gamma$ where $c_\gamma$ is the $(1-\gamma)$-quantile of chi-square random variable with $k$ degrees of freedom. The justification for this test is provided by the following theorem which builds upon the previous results coupled with conventional results for the baseline instrumental variable estimator.\footnote{The proof of this result is provided in a supplementary appendix.}
The rate condition $s^2\log^2 (p\vee n) = o(n)$ can be substantive and cannot be substantially weakened for the full-sample IV estimator considered above. However, we can replace this condition with the weaker condition that $$ s \log (p\vee n) = o(n)$$ by employing a sample splitting method from the many instruments literature (\citename{AngristKruegerSplitSample1995}, \citeyear*{AngristKruegerSplitSample1995}). Specifically, we consider dividing the sample randomly into (approximately) equal parts $a$ and $b$, with sizes $n_a = \lceil n/2 \rceil$ and $n_b = n - n_a$. We use superscripts $a$ and $b$ for variables in the first and second subsample respectively. The index $i$ will enumerate observations in both samples, with ranges for the index given by $1 \leqslant i \leqslant n_a$ for sample $a$ and $1 \leqslant i \leqslant n_b$ for sample $b$. We can use each of the subsamples to fit the first stage via Lasso or Post-Lasso to obtain the first stage estimates $\widehat \beta^k_l, k=a, b,$ and $l=1,\ldots,k_e$. Then setting $\widehat D_{il}^a = {f_i^a}'\widehat \beta^b_l, 1 \leqslant i \leqslant n_a $, $\widehat D_{il}^b = {f_i^b}'\widehat \beta^a_l, 1 \leqslant i \leqslant n_b$, $\widehat D^k_i = ( {\widehat{D}_{i1}^{k}},\ldots, \widehat D_{ik_e}^k, {w^k_i}')', k=a, b $, we form the IV estimates in the two subsamples: $$ \widehat \alpha_a = {\mathbb{E}_{n_a}} [\widehat D_i^a {d_i^a}' ]^{-1} {\mathbb{E}_{n_a}} [\widehat D_i^a y^a_{i}] \ \ \ \mbox{and} \ \ \ \widehat \alpha_b = {\mathbb{E}_{n_b}} [\widehat D_i^b {d_i^b}' ]^{-1} {\mathbb{E}_{n_b}} [{\widehat D_i^b} y^b_{i}]. $$ Then we combine the estimates into one
The following result shows that the split-sample IV estimator $\widehat \alpha_{ab}$ has the same large sample properties as the estimator $\widehat \alpha$ of the previous section but requires a weaker growth condition.
The previous sections' results suggest that using Lasso for fitting first-stage regressions should result in IV estimators with good estimation and inference properties. In this section, we provide simulation evidence regarding these properties in a situation where there are many possible instruments. We also compare the performance of the developed Lasso-based estimators to many-instrument robust estimators that are available in the literature.
Our simulations are based on a simple instrumental variables model data generating process (DGP):
where $\beta=1$ is the parameter of interest, and $z_i = (z_{i1},z_{i2},...,z_{i100})' \sim N(0,\Sigma_Z)$ is a 100 x 1 vector with $E[z_{ih}^2] = \sigma^2_z$ and $\textrm{Corr}(z_{ih},z_{ij}) = .5^{|j-h|}$. In all simulations, we set $\sigma^2_e = 1$ and $\sigma^2_z = 1$. We also set $\textrm{Corr}(e,v) = 0.6.$
For the other parameters, we consider various settings. We provide results for sample sizes, $n$, of 100 and 250. We set $\sigma^2_v$ so that the unconditional variance of the endogenous variable equals one; i.e. $\sigma^2_v = 1 - \Pi'\Sigma_Z\Pi$. We use three different settings for the pattern of the first-stage coefficients, $\Pi$. In the first, we set $\Pi = C\widetilde\Pi = C (1,.7,.7^2,...,.7^{98},.7^{99})'$; we term this the “exponential” design. In the second and third case, we set $\Pi = C\widetilde\Pi = C(\iota_s,0_{n-s})'$ where $\iota_s$ is a $1 \times s$ vector of ones and $0_{n-s}$ is a $1 \times n-s$ vector of zeros. We term this the “cut-off” design and consider two different values of $s$, $s = 5$ and $s = 50$. In the exponential design, the model is not literally sparse although the majority of explanatory power is contained in the first few instruments. While the model is exactly sparse in the cut-off design, we expect Lasso to perform poorly with $s = 50$ since treating $\frac{s^2\log^2 p}{n}$ as vanishingly small seems like a poor approximation given the sample sizes considered. We consider different values of the constant $C$ that are chosen to generate target values for the concentration parameter, $\mu^2 = \frac{n\Pi'\Sigma_Z\Pi}{\sigma^2_v}$, which plays a key role in the behavior of IV estimators; see, e.g. \citeasnoun{stock:survey} or \citeasnoun{hhn:weakiv}.\footnote{The concentration parameter is closely related to the first-stage Wald statistic and first-stage F-statistic for testing that the coefficients on the instruments are equal to 0. Under homoscedasticity, the first-stage Wald statistic is $W = \widehat\Pi' (Z'Z) \widehat\Pi/\widehat\sigma^2_v$ and the first-stage F-statistic is $W/\dim(Z)$.} Specifically, we choose $C$ to solve $\mu^2 = \frac{nC^2 \widetilde\Pi \Sigma_Z \widetilde\Pi}{1-C^2\widetilde\Pi \Sigma_Z \widetilde\Pi}$ for $\mu^2 = 30$ and for $\mu^2 = 180$. These values of the concentration parameter were chosen by using estimated values from the empirical example reported below as a benchmark.\footnote{In the empirical example, first-stage Wald statistics based on the selected instruments range from between 44 and 243. In the cases with constant coefficients, our concentration parameter choices correspond naturally to “infeasible F-statistics” defined as $\mu^2/s$ of 6 and 36 with $s = 5$ and .6 and 3.6 with $s = 50$. In an online appendix, we provide additional simulation results. The results reported in the current section are sufficient to capture the key patterns.}
For each setting of the simulation parameter values, we report results from seven different procedures. A simple possibility when presented with many instrumental variables (with $p < n$) is to just estimate the model using 2SLS and all of the available instruments. It is well-known that this will result in poor-finite sample properties unless there are many more observations than instruments; see, for example, \citeasnoun{bekker}. The estimator proposed in \citeasnoun{fuller} (FULL) is robust to many instruments (with $p < n$) as long as the presence of many instruments is accounted for when constructing standard errors for the estimators; see \citeasnoun{hhn:weakiv} for example.\footnote{FULL requires a user-specified parameter. We set this parameter equal to one which produces a higher-order unbiased estimator. See \citeasnoun{hhk:weakmse} for additional discussion. LIML is another commonly proposed estimator which is robust to many instruments. In our designs, its performance was generally similar to that of FULL, and we report only FULL for brevity.} We report results for these estimators in rows labeled 2SLS(100) and FULL(100) respectively.\footnote{With $n = 100$, estimates are based on a randomly selected 99 instruments.} For our variable selection procedures, we use Lasso to select among the instruments using the refined data-dependent penalty loadings given in ((ref)) described in Appendix A and consider two post-model selection estimation procedures. The first, Post-Lasso, runs 2SLS using the instruments selected by Lasso; and the second, Post-Lasso-F, runs FULL using the instruments selected by Lasso. In cases where no instruments are selected by Lasso, we use the point-estimate obtained by running 2SLS with the single instrument with the highest within sample correlation to the endogenous variable as the point estimate for Post-Lasso and Post-Lasso-F. In these cases, we use the sup-Score test for performing inference.\footnote{Inference based on the asymptotic approximation when Lasso selects instruments and based on the sup-Score test when Lasso fails to select instruments is our preferred procedure.} We report inference results based on the weak-identification robust sup-score testing procedure in rows labeled “sup-Score”.
The other two procedure “Post-Lasso (Ridge)” and “Post-Lasso-F (Ridge)” use a combination of Ridge regression, Lasso, and sample-splitting. For these procedures, we randomly split the sample into two equal-sized parts. Call these sub-samples “sample A” and “sample B.” We then use leave-one-out cross-validation with only the data in sample A to select a ridge penalty parameter and then estimate a set of ridge coefficients using this penalty and the data from sample A. We then use the data from sample B with these coefficients estimated using only data from sample A to form first-stage fitted values for sample B. Then, we take the full-set of instruments augmented with the estimated fitted value just described and perform Lasso variable selection using only the data from sample B. We use the selected variables to run either 2SLS or Fuller in sample B to obtain estimates of $\beta$ (and associated standard errors), say $\widehat\beta_{B,2SLS}$ ($s_{B,2SLS}$) and $\widehat\beta_{B,Fuller}$ ($s_{B,Fuller}$). We then repeat this exercise switching sample A and B to obtain estimates of $\beta$ (and associated standard errors) from sample A, say $\widehat\beta_{A,2SLS}$ ($s_{A,2SLS}$) and $\widehat\beta_{A,Fuller}$ ($s_{A,Fuller}$). Post-Lasso (Ridge) is then $w_{A,2SLS} \widehat\beta_{A,2SLS} + (1-w_{A,2SLS}) \widehat\beta_{B,2SLS}$ for $w_{A,2SLS} = \frac{s^2_{B,2SLS}}{s^2_{A,2SLS}+s^2_{B,2SLS}}$, and Post-Lasso-F (Ridge) is defined similarly. If instruments are selected in one subsample but not in the other, we put weight one on the estimator from the subsample where instruments were selected. If no instruments are selected in either subsample, we use the single-instrument with the highest correlation to obtain the point estimate and use the sup-score test for performing inference.
For each estimator, we report median bias (Med. Bias), median absolute deviation (MAD), and rejection frequencies for 5% level tests (rp(.05)). For computing rejection frequencies, we estimate conventional, homoscedastic 2SLS standard errors for 2SLS(100) and Post-Lasso and the many instrument robust standard errors of \citeasnoun{hhn:weakiv} which rely on homoscedasticity for FULL(100) and Post-Lasso-F. We report the number of cases in which Lasso selected no instruments in the column labeled N(0).
We summarize the simulation results in Table 1. It is apparent that the Lasso procedures are dominant when $n = 100.$ In this case, the Lasso-based procedures outperform 2SLS(100) and FULL(100) on all dimensions considered. When the concentration parameter is 30 or $s = 50$, the instruments are relatively weak, and Lasso accordingly selects no instruments in many cases. In these cases, inference switches to the robust sup-score procedure which controls size. With a concentration parameter of 180, the instruments are relatively more informative and sparsity provides a good approximation in the exponential design and $s = 5$ cut-off design. In these cases, Lasso selects instruments in the majority of replications and the procedure has good risk and inference properties relative to the other procedures considered. In the $n = 100$ case, the simple Lasso procedures also clearly dominate Lasso augmented with Ridge as this procedure often results in no instruments being selected and relatively low power; see Figure 1. We also see that the sup-score procedure controls size across the designs considered.
In the $n = 250$ case, the conventional many-instrument asymptotic sequence which has $p$ proportional to $n$ but $p/n < 1$ provides a reasonable approximation to the DGP, and one would expect FULL to perform well. In this case, 2SLS(100) is clearly dominated by the other procedures. However, there is no obvious ranking between FULL(100) and the Lasso-based procedures. With $s = 50$, sparsity is a poor approximation in that there is signal in the combination of the 50 relevant instruments but no small set of instruments has much explanatory power. In this setting, FULL(100) has lower estimation risk than the Lasso procedure which is not effectively able to capture the diffuse signal though both inference procedures have size close to the prescribed level. Lasso augemented with the Ridge fit also does relatively well in this setting, being roughly on par with FULL(100). In the exponential and cut-off with $s = 5$ designs, sparsity is a much better approximation. In these cases, the simple Lasso-based estimators have smaller risk than FULL(100) or Lasso with Ridge and produce tests that have size close to the nominal 5% level. Finally, we see that the sup-score procedure continues to control size with $n = 250$.
Given that the sup-score procedure uniformly controls size across the designs considered but is actually substantially undersized, it is worth presenting additional results regarding power. We plot size-adjusted power curves for the sup-score test, Post-Lasso-F, Post-Lasso-F (Ridge), and FULL(100) across the different designs in the $\mu^2 = 180$ cases in Figure 1. We focus on $\mu^2 = 180$ since we expect it is when identification is relatively strong that differences in power curves will be most pronounced. From these curves, it is apparent that the robustness of the sup-score test comes with a substantial loss of power in cases where identification is strong. Exploring other procedures that are robust to weak identification, allow for $p \gg n$, and do not suffer from such power losses may be interesting for future research.
The evidence from the simulations is supportive of the derived theory and favorable to Lasso-based IV methods. The Lasso-IV estimators clearly dominate on all metrics considered when $p = n$ and $s \ll n$. The Lasso-based IV estimators generally have relatively small median bias and estimator risk and do well in terms of testing properties, though they do not dominate FULL in these dimensions across all designs with $p < n$. The simulation results verify that FULL becomes more appealing as the sparsity assumption breaks down. This breakdown of sparsity is likely in situations with weak instruments, be they many or few, where none of the first-stage coefficients are well-separated from zero relative to sampling variation. Overall, the simulation results show that simple Lasso-based procedures can usefully complement other many-instrument methods.
As an example of the potential application of Lasso to select instruments, we consider IV estimation of the effects of federal appellate court decisions regarding eminent domain on a variety of economic outcomes.\footnote{See \citeasnoun{chen:yeh:takings} for a detailed discussion of the economics of takings law (or eminent domain), relevant institutional features of the legal system, and a careful discussion of endogeneity concerns and the instrumental variables strategy in this context.} To try to uncover the relationship between takings law and economic outcomes, we estimate structural models of the form $$ y_{ct} = \alpha_c + \alpha_t + \gamma_c t + \beta \ Takings \ Law_{ct} + W_{ct}'\delta + \epsilon_{ct} $$ where $y_{ct}$ is an economic outcome for circuit $c$ at time $t$, Takings Law$_{ct}$ represents the number of pro-plaintiff appellate takings decisions in circuit $c$ and year $t$; $W_{ct}$ are judicial pool characteristics,\footnote{The judicial pool characteristics are the probability of a panel being assigned with the characteristics used to construct the instruments. There are 30, 33, 32, and 30 controls available for FHFA house prices, non-metro house prices, Case-Shiller house prices, and GDP respectively.} a dummy for whether there were no cases in that circuit-year, and the number of takings appellate decisions; and $\alpha_c$, $\alpha_t$, and $\gamma_c t$ are respectively circuit-specific effects, time-specific effects, and circuit-specific time trends. An appellate court decision is coded as pro-plaintiff if the court ruled that a taking was unlawful, thus overturning the government's seizure of the property in favor of the private owner. We construe pro-plaintiff decisions to indicate a regime that is more protective of individual property rights. The parameter of interest, $\beta$, thus represents the effect of an additional decision upholding individual property rights on an economic outcome.
We provide results using four different economic outcomes: the log of three home-price-indices and log(GDP). The three different home-price-indices we consider are the quarterly, weighted, repeat-sales FHFA/OFHEO house price index that tracks single-family house prices at the state level for metro (FHFA) and non-metro (Non-Metro) areas and the Case-Shiller home price index (Case-Shiller) by month for 20 metropolitan areas based on repeat-sales residential housing prices. We also use state level GDP from the Bureau of Economic Analysis to form log(GDP). For simplicity and since all of the controls, instruments, and the endogenous variable vary only at the circuit-year level, we use the within-circuit-year average of each of these variables as the dependent variables in our models. Due to the different coverage and time series lengths available for each of these series, the sample sizes and sets of available controls differ somewhat across the outcomes. These differences lead to different first-stages across the outcomes as well. The total sample sizes are 312 for FHFA and GDP which have identical first-stages. For Non-Metro and Case-Shiller, the sample sizes are 110 and 183 respectively.
The analysis of the effects of takings law is complicated by the possible endogeneity between governmental takings and takings law decisions and economic variables. To address the potential endogeneity of takings law, we employ an instrumental variables strategy based on the identification argument of \citeasnoun{chen:sethi} and \citeasnoun{chen:yeh:takings} that relies on the random assignment of judges to federal appellate panels. Since judges are randomly assigned to three judge panels to decide appellate cases, the exact identity of the judges and, more importantly, their demographics are randomly assigned conditional on the distribution of characteristics of federal circuit court judges in a given circuit-year. Thus, once the distribution of characteristics is controlled for, the realized characteristics of the randomly assigned three judge panel should be unrelated to other factors besides judicial decisions that may be related to economic outcomes.
There are many potential characteristics of three judge panels that may be used as instruments. While the basic identification argument suggests any set of characteristics of the three judge panel will be uncorrelated with the structural unobservable, there will clearly be some instruments which are more worthwhile than others in obtaining precise second-stage estimates. For simplicity, we consider only the following demographics: gender, race, religion, political affiliation, whether the judge's bachelor was obtained in-state, whether the bachelor is from a public university, whether the JD was obtained from a public university, and whether the judge was elevated from a district court along with various interactions. In total, we have 138, 143, 147, and 138 potential instruments for FHFA prices, non-metro prices, Case-Shiller, and GDP respectively that we select among using Lasso.\footnote{Given the sample sizes and numbers of variables, estimators using all the instruments without shrinkage are only defined in the GDP and FHFA data. For these outcomes, the \citeasnoun{fuller} point estimate (standard error) is -.0020 (3.123) for FHFA and .0120 (.1758) for GDP.}
Table 3 contains estimation results for $\beta$. We report OLS estimates and results based on three different sets of instruments. The first set of instruments, used in the rows labeled 2SLS, are the instruments adopted in \citeasnoun{chen:yeh:takings}.\footnote{\citeasnoun{chen:yeh:takings} used two variables motivated on intuitive grounds, whether a panel was assigned an appointee who did not report a religious affiliation and whether a panel was assigned an appointee who earned their first law degree from a public university, as instruments.} We consider this the baseline. The second set of instruments are those selected through Lasso using the refined data-driven penalty.\footnote{Lasso selects the number of panels with at least one appointee whose law degree is from a public university (Public) cubed for GDP and FHFA. In the Case-Shiller data, Lasso selects Public and Public squared. For non-metro prices, Lasso selects Public interacted with the number of panels with at least one member who reports belonging to a mainline protestant religion, Public interacted with the number of panels with at least one appointee whose BA was obtained in-state (In-State), In-State interacted with the number of panels with at least one non-white appointee, and the interaction of the number of panels with at least one Democrat appointee with the number of panels with at least one Jewish appointee.} The number of instruments selected by Lasso is reported in the row “S”. We use the Post-Lasso 2SLS estimator and report these results in the rows labeled “Post-Lasso”. The third set of instruments is simply the union of the first two instrument sets. Results for this set of instruments are in the rows labeled “Post-Lasso+”. In this case, “S” is the total number of instruments used. In all cases, we use heteroscedasticity consistent standard error estimators. Finally, we report the value of the test statistic discussed in Section 4.3.1 comparing estimates using the first and second sets of instruments in the row labeled “Spec. Test”.
The most interesting results from the standpoint of the present paper are found by comparing first-stage Wald-statistics and estimated standard errors across the instrument sets. The Lasso instruments are clearly much better first-stage predictors as measured by the first-stage Wald-statistic compared to the \citeasnoun{chen:yeh:takings} benchmark. Given the degrees of freedom, this increase obviously corresponds to Lasso-based IV providing a stronger first-stage relationship for FHFA prices, GDP, and the Case-Shiller prices. In the non-metro case, the p-value from the Wald test with the baseline instruments of \citeasnoun{chen:yeh:takings} is larger than that of the Lasso-selected instruments. This improved first-stage prediction is associated with the resulting 2SLS estimator having smaller estimated standard errors than the benchmark case for non-metro prices, Case-Shiller prices, and GDP. The reduction in standard errors is sizable for both non-metro and Case-Shiller. Tthe standard error estimate is somewhat larger in the FHFA case despite the improvement in first-stage prediction. Given that the Post-Lasso first-stage produces a larger first-stage Wald-statistic while choosing fewer instruments than the benchmark suggests that we might prefer the Post-Lasso results in any case. We also see that the test statistics for testing the difference between the estimate using the \citeasnoun{chen:yeh:takings} instruments and the Post-Lasso estimate is equal to zero are uniformly small. Given the small differences between estimates using the first two sets of instruments, it is unsurprising that the results using the union of the two instrument sets are similar to those already discussed.
The results are also economically interesting. The point estimates for the effect of an additional pro-plaintiff decision, a decision in favor of individual property holders, are positive, suggesting these decisions are associated with increases in property prices and GDP. These point estimates are all small, and it is hard to draw any conclusion about the likely effect on GDP or the FHFA index given their estimated standard errors. On the other hand, confidence intervals for non-metro and Case-Shiller constructed at usual confidence levels exclude zero. Overall, the results do suggest that the causal effect of decisions reinforcing individual property rights is an increase in the value of holding property, at least in the short term. The results are also consistent with the developed asymptotic theory in that the 2SLS point-estimates based on the benchmark instruments are similar to the estimates based on the Lasso-selected instruments while Lasso produces a stronger first-stage relationship and the Post-Lasso estimates are more precise in three of the four cases. The example suggests that there is the potential for Lasso to be fruitfully employed to choose instruments in economic applications.