Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
85,486 characters · 15 sections · 81 citation commands
The Robust F-Statistic as a Test for Weak Instruments
{\smallKeywords:}{ Instrumental variables, weak instruments, heteroskedasticity, robust F-statistic, GMM}
{\smallJEL Codes:}{ C12, C26}
\thispagestyle{empty}
\baselineskip=20pt
\pagenumbering{arabic} \setcounter{page}{1}
It is commonplace to report the first-stage F-statistic to test for weak instruments in linear models with a single endogenous variable, estimated by two-stage least squares (2SLS). This follows the work of StaigerStock1997 and StockYogo2005, with the latter providing critical values for the first-stage non-robust F-statistic for null hypotheses of weak instruments in terms of bias of the 2SLS estimator relative to that of the OLS estimator and Wald-test size distortions. These weak-instruments critical values for the non-robust F-statistic are valid only under homoskedasticity (i.e.\ conditional homoskedasticity, no serial correlation and no clustering) of both the first-stage and structural errors, and do not apply to the robust F-statistic in general designs, see BunHaan2010, OleaPflueger2013 and IAndrewsREStat2018. In particular, IAndrewsREStat2018 found for some cross-sectional heteroskedastic designs that the standard 2SLS confidence intervals had large coverage distortions even for very large values of the robust F-statistic. For example, he found for a high endogeneity design that “the 2SLS confidence set has a 15% coverage distortion even when the mean of the first-stage robust F-statistic is 100,000”, IAndrewsREStat2018.
For general heteroskedasticity, which we mean to cover the cases of conditional heteroskedasticity, serial correlation and clustering, OleaPflueger2013 proposed the first-stage effective F-statistic and derived critical values for the null of weak instruments in terms of the Nagar bias of the 2SLS or LIML estimator, relative to an estimator-specific benchmark worst-case bias. We focus here on their results for the 2SLS estimator. As shown in Section (ref), the effective F-statistics in the designs of IAndrewsREStat2018 do not reject the null of weak instruments. In their review paper \citet*[p 729]{AndrewsStockSun2019} recommend “that researchers judge instrument strength based on the effective F-statistic of OleaPflueger2013”.
The effective F-statistic is specific to the Nagar bias of the 2SLS or LIML estimator and the main contribution of this paper is that the 2SLS related methods of OleaPflueger2013 apply to a wider class of linear generalized method of moments (GMM) estimators resulting in a class of associated generalized effective F-statistics. The robust F-statistic is a member of this class, and we call its associated GMM estimator the GMMf estimator, with the extension “f” for first-stage. This is because the weight matrix of the GMMf estimator is based on the first-stage residuals, with $k_{z}$ times the robust F-statistic being the denominator of the GMMf estimator, where $k_{z}$ is the number of excluded instruments. This is similar to the relationship of the non-robust F-statistic and the 2SLS estimator.
In practice, reported 2SLS estimation results with robust standard errors are often accompanied by the robust first-stage F-statistic, as most statistical packages automatically provide these together. Whilst the robust F-statistic can be used as a test for underidentification, it can not be used as a test for weak instruments related to the performance of the 2SLS estimator. Hence, for the single-endogenous variable case, robust 2SLS estimation results should be accompanied by the effective F-statistic and its critical value, as per the advice of AndrewsStockSun2019. The critical value of the robust F-statistic in relation to the test for weak instruments for the GMMf estimator could also be computed. If the situation is such, like in the IAndrewsREStat2018 example, that the effective F-statistic is small and indicates weak instruments problems for the 2SLS estimator, but the robust F-statistic is large, rejecting the null of weak instruments for the GMMf estimator, then the latter could be preferred and reported.\footnote{An extension of the “weakivtest” command of weakivtest in Stata, Stata, called “gfweakivtest” for calculating these is available from https://github.com/stat0357/gfweakivtest.} This is illustrated in Section (ref), where we replicate the Monte Carlo analysis of IAndrewsREStat2018. The design is the same as a grouped-data one, see AngristJoE1991 and the discussion in AngristPischke2009, where the instruments are mutually exclusive group membership indicators. In the two designs considered, there is in each only one informative group, but the first-stage heteroskedasticity is such that the 2SLS estimator does not utilize this information well, whereas the GMMf estimator gives almost all the weight to the informative groups.
Section (ref) introduces the single-endogenous variable linear model specification, main assumptions, effective and robust F-statistics and the GMMf estimator. Section (ref) then formulates the class of generalized F-statistics for the class of linear GMM estimators and shows that the weak-instruments testing methods developed by OleaPflueger2013 apply straightforwardly to this class. The section then provides a summary of the OleaPflueger2013 results. Section (ref) shows how the general results and specifications simplify for the robust-F statistic in relation to the Nagar bias of the GMMf estimator. As the Nagar bias is relative to a benchmark worse-case bias, which is estimator specific, we harmonize in Section (ref) the benchmark bias and propose the use of the worst-case OLS bias as the benchmark, which applies to the class of GMM estimators considered.
The weak-instruments test procedures considered here only apply to linear models with a single endogenous regressor. LewisMertens2022 develop an extension of the OleaPflueger2013 method to the multiple endogenous variable case for the 2SLS estimator, but they do not consider such an extension for the wider class of GMM estimators.
We have a sample $\left\{ y_{i},x_{i},z_{i}\right\} _{i=1}^{n}$, where $z_{i}$ is a $k_{z}$-vector of instrumental variables. We are interested in the effect of $x$ on $y$ in a linear model specification, where $x$ is endogenously determined. We consider the linear structural and first-stage specifications
where $y$, $x$, $u$ and $v_{2}$ are $n$-vectors and $Z$ an $n\times k_{z}$ matrix. Other exogenous explanatory variables, including the constant have been partialled out. The reduced-form specification for $y$ is then given by
where $v_{1}=u+\beta v_{2}$, and $\pi_{y}=\pi\beta$.
Following OleaPflueger2013 (henceforth MOP), we make the following assumptions.
In the remainder, we drop the subscript $n$ from $\widehat{W}_{n}$ for ease of exposition, and, commensurate with the partitioning of $W$, \[ \widehat{W}=\left[
\right]. \] The two-stage least squares (2SLS) estimator is given by \[ \widehat{\beta}_{2sls}=\frac{x'P_{Z}y}{x'P_{Z}x}, \] where $P_{Z}=Z\left(Z'Z\right)^{-1}Z'$. The standard non-robust first-stage F-statistic is \[ \widehat{F}=\frac{x'P_{Z}x}{k_{z}\widehat{\sigma}_{v_{2}}^{2}}, \] where $\widehat{\sigma}_{v_{2}}^{2}=\widehat{v}_{2}'\widehat{v}_{2}/n$, $\widehat{v}_{2}=\left(I_{n}-P_{Z}\right)x$, where $I_{n}$ is the identity matrix of order $n$. Note that we refrain throughout from finite sample degrees-of-freedom corrections in the exposition. It follows that we can alternatively express the 2SLS estimator as
where $\widehat{\pi}$ and $\widehat{\pi}_{y}$ are the OLS estimators of $\pi$ and $\pi_{y}$ in the first-stage and reduced-form models ((ref)) and ((ref)), and $V\widehat{a}r\left(\widehat{\pi}\right)=\widehat{\sigma}_{v_{2}}^{2}\left(Z'Z\right)^{-1}$, the non-robust estimator of the variance of $\widehat{\pi}$.
$\widehat{F}$ can be used as a test for underidentification and as a test for weak instruments in terms of the bias of the 2SLS estimator relative to that of the OLS estimator of $\beta$, or the size distortion of the Wald test for hypotheses on $\beta$, StockYogo2005. The Stock and Yogo critical values are valid only under conditional homoskedasticity of both $u$ and $v_{2}$, i.e.\ $\mathbb{E}\left[u_{i}|z_{i}\right]=\sigma_{u}^{2}$ and $\mathbb{E}\left[v_{2i}|z_{i}\right]=\sigma_{v_{2}}^{2}$, or $W=\Sigma_{v}\otimes Q_{zz}$, with $\Sigma_{v}=\mathbb{E}\left[v_{i}v_{i}'\right]$ and $v_{i}=\left(v_{1i},v_{2i}\right)'$.
The robust F-statistic is given by
and is a standard test statistic for testing $H_{0}:\pi=0$ under general forms of heteroskedasticity. But $\widehat{F}_{r}$ cannot be used as a test for weak instruments in relation to the behaviour of the 2SLS estimator, see the discussion in AndrewsStockSun2019, summarized at the end of this section. IAndrewsREStat2018 showed in a grouped-data IV design that $\widehat{F}_{r}$ could take very large values, of the order of $100,000$, whereas the 2SLS estimator was still poorly behaved in terms of bias and Wald test size.
MOP proposed the effective F-statistic
and showed that this F-statistic can be used as a test for weak instruments in relation to the Nagar1959 bias of the 2SLS and LIML estimators, relative to a worst-case benchmark. AndrewsStockSun2019 advocate the use of $\widehat{F}_{\text{eff}}$ to gauge instrument strength for the 2SLS estimator. Although this weak-instrument test is related to the bias, the results presented in AndrewsStockSun2019 for a sample of 106 specifications from papers published in the American Economic Review suggest that the effective F-statistic “may convey useful information about the instrument strength more broadly, since we see that conventional asymptotic approximations appear reasonable in specifications where the effective F-statistic exceeds 10.”, AndrewsStockSun2019.
In the next section, we introduce a class of generalized effective F-statistics, denoted $\widehat{F}_{\text{geff}}$, associated with a class of linear Generalized Method of Moments (GMM) estimators. We show that the weak-instrument Nagar bias results of MOP, derived for the effective F-statistic in relation to the 2SLS estimator, applies to this general class. The robust F-statistic is a member of this class and the associated GMM estimator, denoted GMMf, is defined as
where the extension “f” is for first stage, as the weight matrix is based on the first-stage residuals. As we will show and explain below, for the IAndrewsREStat2018 design with large values for the robust F-statistic, but small values for the effective F-statistic, this estimator is much better behaved in terms of bias and also inference than the 2SLS estimator.
Like the expression of the 2SLS estimator in ((ref)), we can write the GMMf estimator as
where $V\widehat{a}r_{r}\left(\widehat{\pi}\right)=n\left(Z'Z\right)^{-1}\widehat{W}_{2}\left(Z'Z\right)^{-1}$ is the robust estimator of the variance of $\widehat{\pi}$.
The discussion in AndrewsStockSun2019 provides the intuition of why $\widehat{F}_{\text{eff}}$ is an appropriate statistic for testing instrument strength when using 2SLS. As they argue, 2SLS behaves badly when its denominator $x'P_{Z}x$ is close to zero. $\widehat{F}_{\text{eff}}$ measures this object, as $x'P_{Z}x$ is its numerator, and gets the standard errors right on average in the case of general heteroskedasticity, unlike the non-robust $\widehat{F}$. The robust F-statistic $\widehat{F}_{r}$ measures a different object, $x'Z\widehat{W}_{2}^{-1}Z'x$, and, asymptotically, “while it has a noncentral chi-square distribution, its noncentrality parameter does not correspond to the distribution of $\widehat{\beta}_{2sls}$”, AndrewsStockSun2019. But $x'Z\widehat{W}_{2}^{-1}Z'x$ is the denominator of the GMMf estimator, and so $\widehat{F}_{r}$ is the appropriate statistic for testing instrument strength when using $\widehat{\beta}_{gmmf}$.
Consider the class of linear Generalized Methods of Moments (GMM) estimators of $\beta$, given by
where $\Omega_{n}$ is a $k_{z}\times k_{z}$ possibly data dependent weight matrix. It satisfies the following assumption,
Assumption (ref) precludes the standard two-step GMM estimator with $\Omega_{n}=\Omega_{n}\left(\widehat{\beta}_{1}\right)$, e.g.\ in the cross-sectional setting, $\Omega_{n}\left(\widehat{\beta}_{1}\right)=\left(\frac{1}{n}\sum_{i=1}^{n}\widehat{u}_{1i}^{2}z_{i}z_{i}'\right)^{-1}$, where $\widehat{u}_{1i}=y_{i}-x_{i}\widehat{\beta}_{1}$ and $\widehat{\beta}_{1}$ is an initial estimator, for example the 2SLS estimator. The initial estimator is consistent under standard strong-instruments asymptotics, but it converges to a non-degenerate random variable under weak-instruments asymptotics, see ((ref)) below, and hence Assumption (ref) does not hold.
Let \[ W_{\Omega}=\left[
\right]=\left(I_{2}\otimes\Omega^{1/2}\right)W\left(I_{2}\otimes\Omega^{1/2}\right), \] and
Then consider the class of generalized effective F-statistics, given by
For the 2SLS estimator we have $\Omega_{n}=\left(\frac{1}{n}Z'Z\right)^{-1}$ and \[ \widehat{F}_{\text{geff}}\left(\left(Z'Z/n\right)^{-1}\right)=\frac{x'P_{Z}x}{\text{tr}\left(\widehat{W}_{2}\left(Z'Z/n\right)^{-1}\right)}=\widehat{F}_{\text{eff}}. \] For the GMMf estimator as defined in ((ref)), we have $\Omega_{n}=\widehat{W}_{2}^{-1}$. Therefore \[ \widehat{F}_{\text{geff}}\left(\widehat{W}_{2}^{-1}\right)=\frac{x'Z\widehat{W}_{2}^{-1}Z'x}{nk_{z}}=\widehat{F}_{r}. \]
This remainder of this section together with the proofs in the Appendix draw heavily on OleaPflueger2013. We show that the MOP weak-instruments testing methodology for 2SLS applies to the class of generalized F-statistics in relation to the Nagar bias of the linear GMM estimators. We do this by restating parts of their Lemma 1 and Theorem 1, OleaPflueger2013, that directly apply to our GMM setting.
As $v_{1}=u+\beta v_{2}$, it follows from Assumption (ref) that, as $n\rightarrow\infty$, \[ \frac{1}{\sqrt{n}}\left(
\right)\overset{d}{\rightarrow}N\left(0,S\left(\beta,W_{\Omega}\right)\right), \] with
where
We can now state Theorem 1 of OleaPflueger2013 on the Nagar bias approximation for our general class of GMM estimators.
MOP interpret the benchmark $\text{BM}\left(\beta,W_{\Omega}\right)$ as a “worst-case” bias. It is derived by approximating the expectation of the ratio by the ratio of expectations,
This expected (absolute) bias expression is maximized when the concentration parameter $\mu_{\Omega}^{2}=0$ and when the the first-stage and structural errors are perfectly correlated, and it follows that then $\mathbb{E}\left[\beta_{\Omega}^{*}\right]\leq\sqrt{\text{tr}\left(S_{1}\left(\beta,W_{\Omega}\right)\right)/\text{tr}\left(W_{\Omega,2}\right)}$, see OleaPflueger2013.
The results of Lemma (ref) and Theorem (ref) are those in MOP for the 2SLS estimator, with $\Omega_{n}=\left(\frac{1}{n}Z'Z\right)^{-1}$ and $\Omega=Q_{zz}^{-1}$. The Lemma and Theorem are replicated here to show that their methodology applies directly to the larger class of GMM estimators ((ref)), under Assumption (ref). We can therefore also apply the MOP procedure for testing for weak instruments directly, as we describe next.
The null hypothesis of weak instruments is specified as in MOP as \[ H_{0}:\mu_{\Omega}^{2}\in\mathcal{H}\left(W_{\Omega},\tau\right)\,\,\,\text{against}\,\,\,H_{1}:\mu_{\Omega}^{2}\ensuremath{\notin}\mathcal{H}\left(W_{\Omega},\tau\right), \] where \[ \mathcal{H}\left(W_{\Omega},\tau\right)=\left\{ \mu_{\Omega}^{2}\in\mathbb{R}_{+}:\sup_{\beta\in\mathbb{R},c_{\Omega,0}\in\mathcal{S}^{k_{z}-1}}\left(\frac{\left|N\left(\beta,\mu_{\Omega}\sqrt{\text{tr}\left(W_{\Omega,2}\right)}c_{\Omega,0},W_{\Omega}\right)\right|}{\text{BM}\left(\beta,W_{\Omega}\right)}\right)>\tau\right\} , \] or equivalently \[ \mathcal{H}\left(W_{\Omega},\tau\right)=\left\{ \mu_{\Omega}^{2}\in\mathbb{R}_{+}:\mu_{\Omega}^{2}<\frac{B\left(W_{\Omega}\right)}{\tau}\right\} . \] Under the null hypothesis of weak instruments, the Nagar bias exceeds a fraction $\tau$ of the benchmark for at least some value of the structural parameter $\beta$ and some direction of the first-stage parameters, $c_{\Omega,0}$. The parameter $\tau$ is a user specified threshold, commonly set to $\tau=0.10$.
The generalization of the MOP test for weak instruments is then based on $\widehat{F}_{\text{geff}}\left(\Omega_{n}\right)$ which is asymptotically distributed as $\gamma_{\Omega,2}^{\prime}\gamma_{\Omega,2}/\text{tr}\left(W_{\Omega,2}\right)$, with $\gamma_{\Omega,2}\sim N\left(c_{\Omega},W_{\Omega,2}\right)$, which has mean $1+\mu_{\Omega}^{2}$. It follows that we reject $H_{0}$ when $\widehat{F}_{\text{geff}}\left(\Omega_{n}\right)$ is large. Denote by $F_{c_{\Omega},W_{\Omega,2}}^{-1}\left(\alpha\right)$ the upper $\alpha$ quantile of the distribution of $\gamma_{\Omega,2}^{\prime}\gamma_{\Omega,2}/\text{tr}\left(W_{\Omega,2}\right)$ and let \[ cv\left(\alpha,W_{\Omega,2},d_{\Omega}\right):=\sup_{c_{\Omega}\in\mathbb{R}^{k_{z}}}\left\{ F_{c_{\Omega},W_{\Omega,2}}^{-1}\left(\alpha\right)1_{\left(\frac{c_{\Omega}^{\prime}c_{\Omega}}{\text{tr}\left(W_{\Omega,2}\right)}<d_{\Omega}\right)}\right\} , \] where $1_{\left(A\right)}$ denotes the indicator function over a set $A$. The null of weak instruments is then rejected if \[ \widehat{F}_{\text{geff}}\left(\Omega_{n}\right)>cv\left(\alpha,\widehat{W}_{\Omega_{n},2},B\left(\widehat{W}_{\Omega_{n}}\right)/\tau\right), \] which is shown in Lemma 2 of OleaPflueger2013 to be pointwise asymptotically valid, \[ \sup_{\mathcal{H}\left(W_{\Omega},\tau\right)}\lim_{n\rightarrow\infty}\mathbb{P}\left(\widehat{F}_{\text{geff}}\left(\Omega_{n}\right)>cv\left(\alpha,\widehat{W}_{\Omega_{n},2},B\left(\widehat{W}_{\Omega_{n}}\right)/\tau\right)\right)\leq\alpha, \] and, provided that $B\left(\widehat{W}_{\Omega_{n}}\right)$ is bounded in probability, \[ \lim_{\mu_{\Omega}^{2}\rightarrow\infty}\lim_{n\rightarrow\infty}\mathbb{P}\left(\widehat{F}_{\text{geff}}\left(\Omega_{n}\right)>cv\left(\alpha,\widehat{W}_{\Omega_{n},2},B\left(\widehat{W}_{\Omega_{n}}\right)/\tau\right)\right)=1. \]
After obtaining $B\left(\widehat{W}_{\Omega_{n}}\right)$ by a numerical routine, MOP show that the critical values can be obtained by Monte Carlo methods or by the Patnaik1949 curve-fitting methodology. The Patnaik critical value is obtained as the the upper $\alpha$ quantile of $\chi_{\widehat{k}_{\text{geff}}\left(\Omega_{n}\right)}^{2}\left(d_{\Omega_{n},\tau}\widehat{k}_{\text{geff}}\left(\Omega_{n}\right)\right)/\widehat{k}_{\text{geff}}\left(\Omega_{n}\right)$ where $\chi_{\widehat{k}_{\text{geff}}\left(\Omega_{n}\right)}^{2}\left(d_{\Omega_{n},\tau}\widehat{k}_{\text{geff}}\left(\Omega_{n}\right)\right)$ denotes the noncentral $\chi^{2}$ distribution with $\widehat{k}_{\text{geff}}$$\left(\Omega_{n}\right)$ degrees of freedom and noncentrality parameter $d_{\Omega_{n},\tau}\widehat{k}_{\text{geff}}\left(\Omega_{n}\right)$, with
and where $\lambda_{\text{max}}\left(\widehat{W}_{\Omega_{n},2}\right)$ denotes the maximum eigenvalue of $\widehat{W}_{\Omega_{n},2}$.
To summarize, and following MOP, the weak-instruments test procedure related to the Nagar approximation of the bias of the GMM estimator $\widehat{\beta}_{\Omega_{n}}$as defined in ((ref)), under Assumptions (ref) and (ref) is as follows.
As an alternative to step (ref).\ one can use Monte Carlo methods, see OleaPflueger2013. As $B\left(W_{\Omega}\right)\leq1$, MOP propose a simplified asymptotically valid but conservative test. For this simplified test procedure, step (ref).\ is not needed, instead replacing $B\left(\widehat{W}_{\Omega_{n}}\right)$ by $1$, and so $d_{\Omega_{n},\tau}$ by $1/\tau$, in steps (ref).\ and (ref). Hence the simplified test rejects the null hypothesis of weak instruments if \[ \widehat{F}_{\text{geff}}>cv\left(\alpha,\widehat{W}_{\Omega_{n},2},1/\tau\right), \] with $cv\left(\alpha,\widehat{W}_{\Omega_{n},2},1/\tau\right)\geq cv\left(\alpha,\widehat{W}_{\Omega,2},d_{\Omega_{n},\tau}\right)$.
For the robust F-statistic in relation to the Nagar bias of the GMMf estimator as defined in ((ref)), \[ \widehat{\beta}_{gmmf}=\frac{x'Z\widehat{W}_{2}^{-1}Z'y}{x'Z\widehat{W}_{2}^{-1}Z'x}, \] the above expressions apply, but simplify significantly. With $\Omega_{n}=\widehat{W}_{2}^{-1}\stackrel{p}{\rightarrow}W_{2}^{-1}=\Omega$, it follows that \[ W_{\Omega}=\left[
\right]. \] For the Nagar bias expression, we get \[ N_{gmmf}\left(\beta,c_{\Omega},W_{\Omega}\right)=\frac{1}{\mu_{\Omega}^{2}}n_{gmmf}\left(\beta,c_{\Omega},W_{\Omega}\right), \] with $\mu_{\Omega}^{2}=c'Q_{zz}W_{2}^{-1}Q_{zz}c/k_{z}$ and \[ n_{gmmf}\left(\beta,c_{\Omega,0},W_{\Omega}\right)=\frac{\text{tr}\left(S_{12}\left(\beta,W_{\Omega}\right)\right)-2c_{\Omega,0}^{\prime}S_{12}\left(\beta,W_{\Omega}\right)c_{\Omega,0}}{k_{z}}, \] where \[ S_{12}\left(\beta,W_{\Omega}\right)=W_{\Omega,12}-\beta I_{k_{z}}. \] The expression therefore simplifies to \[ n_{gmmf}\left(\beta,c_{\Omega,0},W_{\Omega}\right)=\frac{1}{k_{z}}\left(\text{tr}\left(W_{\Omega,12}\right)-2c_{\Omega,0}^{\prime}W_{\Omega,12}c_{\Omega,0}-\left(k_{z}-2\right)\beta\right). \] The benchmark worst-case bias for the GMMf estimator is then given by \[ \text{BM}\left(\beta,W_{\Omega}\right)=\sqrt{\frac{\text{tr}\left(S_{1}\left(\beta,W_{\Omega}\right)\right)}{k_{z}}} \] where \[ \text{tr}\left(S_{1}\left(\beta,W_{\Omega}\right)\right)=\text{tr}\left(W_{\Omega,1}\right)-2\beta\text{tr}\left(W_{\Omega,12}\right)+k_{z}\beta^{2}. \] Then
As $\Omega=W_{2}^{-1}$ we have that $\gamma_{\Omega,2}\sim N\left(c_{\Omega},I_{k_{z}}\right)$ and so it follows that $\gamma_{\Omega,2}^{\prime}\gamma_{\Omega,2}\sim\chi_{k_{z}}^{2}\left(c_{\Omega}^{\prime}c_{\Omega}\right)=\chi_{k_{z}}^{2}\left(k_{z}\mu_{\Omega}^{2}\right)$. Therefore, the null of weak instruments for the GMMf estimator, specified as \[ H_{0}:\mu_{\Omega}^{2}\in\mathcal{H}_{gmmf}\left(W_{\Omega},\tau\right), \] \[ \mathcal{H}_{gmmf}\left(W_{\Omega},\tau\right)=\left\{ \mu_{\Omega}^{2}\in\mathbb{R}_{+}:\mu_{\Omega}^{2}<\frac{B_{gmmf}\left(W_{\Omega}\right)}{\tau}\right\} , \] is rejected if \[ \widehat{F}_{r}>cv\left(\alpha,k_{z},d_{\Omega_{n},\tau}\right), \] with $cv\left(\alpha,k_{z},d_{\Omega_{n},\tau}\right)$ the upper $\alpha$ quantile of $\chi_{k_{z}}^{2}\left(k_{z}d_{\Omega_{n}}\right)/k_{z}$, and where $d_{\Omega_{n},\tau}=B_{gmmf}\left(\widehat{W}_{\Omega_{n}}\right)/\tau$ and $\widehat{W}_{\Omega_{n}}=\left(I_{2}\otimes\widehat{W}_{2}^{-1/2}\right)\widehat{W}\left(I_{2}\otimes\widehat{W}_{2}^{-1/2}\right)$. Relative to the general results for the generalized effective F-statistic, we see that for the GMMf estimator, $B_{gmmf}\left(\widehat{W}_{\Omega_{n}}\right)$ is a simpler function to maximize with respect to $\beta$ and $c_{\Omega,0}$ than $B\left(\widehat{W}_{\Omega_{n}}\right)$ from ((ref)) when $\Omega_{n}\neq\widehat{W}_{2}^{-1}$. There is further no need for Monte Carlo simulations or Patnaik's curve-fitting methodology to compute the critical values, as $\widehat{F}_{r}$ follows an asymptotic scaled noncentral chi-square distribution, with the “effective” degrees of freedom here equal to $k_{z}$.
To summarize the weak-instruments testing procedure in relation to the approximate Nagar bias of $\widehat{\beta}_{gmmf}$,
The simplified conservative test based on the fact that $B_{gmmf}\left(W_{\Omega}\right)\leq1$ is obtained using the critical value $cv\left(\alpha,k_{z},1/\tau\right)$, which is simply the upper $\alpha$ quantile of $\chi_{k_{z}}^{2}\left(k_{z}/\tau\right)/k_{z}$.
We can illustrate divergence of $\widehat{F}_{\text{eff}}$ and $\widehat{F}_{r}$, building on examples in OleaPflueger2013 and AndrewsStockSun2019. Let $k_{z}=2$, $Q_{zz}=I_{2}$ and $W_{2}=\left(
\right)$. Then for 2SLS/$\widehat{F}_{eff}$ we have the concentration parameter $\mu_{Q_{zz}^{-1}}^{2}=\frac{c_{1}^{2}+c_{2}^{2}}{\xi_{1}^{2}+\xi_{2}^{2}}$, and for GMMf/$\widehat{F}_{r}$ we have $\mu_{W_{2}^{-1}}^{2}=\frac{1}{2}\left(\frac{c_{1}^{2}}{\xi_{1}^{2}}+\frac{c_{2}^{2}}{\xi_{2}^{2}}\right)$. Then, for $c_{1}^{2}>0$ and $c_{2}^{2}>0$, if for example $\xi_{1}^{2}\rightarrow0$ and $\xi_{2}^{2}>c_{1}^{2}+c_{2}^{2}$ then $\mu_{Q_{zz}^{-1}}^{2}\rightarrow\frac{c_{1}^{2}+c_{2}^{2}}{\xi_{2}^{2}}<1$ and $\mu_{W_{2}^{-1}}^{2}\rightarrow\infty$. In this situation, there is no weak-instruments problem for the GMMf estimator, but there is for the 2SLS estimator. A design where such heteroskedasticity can be generated is the grouped-data IV one, and is the one used in \citet{IAndrewsREStat2018}. We consider this design in Section \ref{sec:Grouped-Data-IV-Model}, where we further show how the GMMf estimator utilizes the information in such designs better than the 2SLS estimator.\footnote{\label{fn:concpar} Note that for this example it is the case that if $\left(\xi_{1}^{2}>\xi_{2}^{2}\right)$ \textit{and} $\left(\frac{c_{1}^{2}}{\xi_{1}^{2}}>\frac{c_{2}^{2}}{\xi_{2}^{2}}\right)$ then $\mu_{Q_{zz}^{-1}}^{2}>\mu_{W_{2}^{-1}}^{2}$.}
Under standard strong-instruments asymptotics, the 2SLS estimator is consistent and asymptotically efficient if $\mathbb{E}\left[u_{i}^{2}z_{i}z_{i}'\right]=\sigma_{u}^{2}Q_{zz}$, and for the GMMf estimator this is the case if $\mathbb{E}\left[v_{2,i}^{2}z_{i}z_{i}'\right]=\text{\ensuremath{\delta}}\mathbb{E}\left[u_{i}^{2}z_{i}z_{i}'\right]$, for some constant $\delta>0$. However, under the weak-instruments asymptotics setting of Assumption (ref), it follows from the expression of the random variable $\beta_{\Omega}^{*}$ in ((ref)) that the GMM estimators are biased and inconsistent. Therefore, the weak-instruments test is based on the relative Nagar bias, and GMM estimators can then be ranked on their bias performance, not on their relative efficiency. As the MOP benchmark bias is estimator specific, we first propose to harmonize the bias by using the worst-case OLS bias as a benchmark.
The benchmark bias $\text{BM}\left(\beta,W_{\Omega}\right)$ as defined in ((ref)) is estimator/F-statistic specific and is derived using an ad hoc approximation of $\mathbb{E}\left(\beta_{\Omega}^{*}\right)$. This makes a comparison of weak-instrument test results between generalized F-statistics for different GMM estimators difficult, as the critical values of the test are based on the maximum of the Nagar bias approximation relative to the estimator-specific benchmark. A solution is to harmonize the benchmark bias, such that it is the same for each estimator/F-statistic. For the homoskedastic case, the bias related weak-instruments critical values of StockYogo2005 for the nonrobust F-statistic are in terms of the relative bias, that of the 2SLS estimator relative to that of the OLS estimator. We propose here to harmonize the benchmark worst-case bias and to replace it with the worst-case bias of the OLS estimator.
The probability limit of the worst-case weak-instrument OLS (absolute) bias, under Assumption (ref) is given by
Note that this benchmark bias is only a function of $\beta$ and $\Sigma_{v}$, the marginal variance of $(v_{1}\,v_{2})'$, applies to all forms of general heteroskedasticity and does not depend on homoskedasticity assumptions. However, it can be seen to be a valid benchmark for all GMM estimators defined in ((ref)) by considering the worst-case benchmark bias $\text{BM}\left(\beta,W_{\Omega}\right)$ under the maintained assumption of homoskedasticity. In that case we have $W=\Sigma_{v}\otimes Q_{zz}$ and thus $W_{\Omega}=\Sigma_{v}\otimes\Omega^{-1/2}Q_{zz}\Omega^{-1/2}$. Then it follows that
We have that
where $\lambda_{\text{min}}\left(W_{\Omega,2}\right)$ is the minimum eigenvalue of $W_{\Omega,2}$. It follows further from Assumptions (ref) and (ref) that \[ 0<\frac{\text{BM}\left(\beta,W_{\Omega}\right)}{\text{BM}_{LS}\left(\beta,\Sigma_{v}\right)}<C \] $\forall\beta\in\mathbb{R}$, for some finite $C>1$. Note that the positive definiteness, or full rank assumption of $\Sigma_{v}$ is important, as otherwise $\text{BM}_{LS}\left(\beta,\Sigma_{v}\right)$ would be zero for some value of $\beta$. Therefore the case of $\rho_{12}^{2}=1$ is excluded. It then follows from Theorem (ref) that, \[ B_{LS}\left(W_{\Omega},\Sigma_{v}\right)\coloneqq\sup_{\beta\in\mathbb{R},c_{\Omega,0}\in\mathcal{S}^{k_{z}-1}}\left(\frac{\left|n\left(\beta,c_{\Omega,0},W_{\Omega}\right)\right|}{\text{BM}_{LS}\left(\beta,\Sigma_{v}\right)}\right)<\infty \] and the test procedures described in Section (ref) apply, replacing $B\left(W_{\Omega}\right)$ by $B_{LS}\left(W_{\Omega},\Sigma_{v}\right)$ from step (ref) onwards in the summaries of the testing procedures. The null of weak instruments is then rejected if \[ \widehat{F}_{\text{geff}}\left(\Omega_{n}\right)>cv^{LS}\left(\alpha,\widehat{W}_{\Omega_{n},2},d_{\Omega_{n},\tau}^{LS}\right), \] where $d_{\Omega_{n},\tau}^{LS}=B_{LS}\left(\widehat{W}_{\Omega_{n}},\widehat{\Sigma}_{v}\right)/\tau$.
The interpretation of the weak-instruments null hypothesis $H_{0}:\mu_{\Omega}^{2}\in\mathcal{H}\left(W_{\Omega},\Sigma_{v},\tau\right)$, where \[ \mathcal{H}\left(W_{\Omega},\Sigma_{v},\tau\right)=\left\{ \mu_{\Omega}^{2}\in\mathbb{R}_{+}:\sup_{\beta\in\mathbb{R},c_{\Omega,0}\in\mathcal{S}^{k_{z}-1}}\left(\frac{\left|N\left(\beta,\mu_{\Omega}\sqrt{\text{tr}\left(W_{\Omega,2}\right)}c_{\Omega,0},W_{\Omega}\right)\right|}{\text{BM}_{LS}\left(\beta,\Sigma_{v}\right)}\right)>\tau\right\} , \] is then that the Nagar bias exceeds a fraction $\tau$ of the benchmark maximum OLS bias for at least some value of the structural parameter $\beta$ and some direction of the first-stage coefficients. Whilst this is not the same as the asymptotic relative bias results of StockYogo2005, it is more aligned with it. It makes the interpretation of the null hypothesis and hence that of its rejection the same for different GMM estimators and their associated generalized effective F-statistics and is not based on an ad hoc approximation. In the remainder, this benchmark is used. Note that the simplified conservative test procedure does not apply here, as it is not the case that $B_{LS}\left(W_{\Omega},\Sigma_{v}\right)\leq1$ in general.
We now consider the heteroskedastic model designs from IAndrewsREStat2018. In these designs, very large values of the robust F-statistic are accompanied by a poor performance of the 2SLS estimator, where IAndrewsREStat2018 focused on coverage distortions of confidence sets. We find that in these designs the effective F-statistic is indeed small, indicating a weak-instrument problem for 2SLS, but large values of the robust F-statistic indicate there is not a weak-instrument problem for the GMMf estimator.
Following the example of divergence between $\widehat{F}_{\text{eff}}$ and $\widehat{F}_{r}$ given at at the end of Section (ref), the design in IAndrewsREStat2018 is that of a grouped-data IV setup,
for $i=1,...,n$, where the $G$-vector $z_{i}\in\left\{ e_{1},...,e_{G}\right\} $, with $e_{g}$ a $G$-vector with $g$th entry equal to $1$ and zeros everywhere else, for $g=1,\ldots,G$.
The variance-covariance structure for the errors is modeled fully flexibly by group, and specified as \[ \left(\left(
\right)|z_{i}=e_{g}\right)\sim\left(0,\Sigma_{g}\right), \]
At the group level, we therefore have for group member $j$ in group $g$
\[ \left(
\right)\sim\left(0,\Sigma_{g}\right), \] for $j=1,...,n_{g}$ and $g=1,...,G$, with $n_{g}$ the number of observations in group $g$, $\sum_{g=1}^{G}n_{g}=n$, see also BekkervdPloeg2005. We assume that $\lim_{n\rightarrow\infty}\frac{n_{g}}{n}=f_{g}$, with $0<f_{g}<1$.
The OLS estimator of $\pi_{g}$ is given by $\widehat{\pi}_{g}=\overline{x}_{g}=\frac{1}{n_{g}}\sum_{j=1}^{n_{s}}x_{jg}$ and $Var\left(\widehat{\pi}_{g}\right)=\sigma_{v_{2},g}^{2}/n_{g}$. The OLS residual is $\widehat{v}_{2,jg}=x_{jg}-\overline{x}_{g}$ and the estimator for the variance is given by $V\widehat{a}r\left(\widehat{\pi}_{g}\right)=\widehat{\sigma}_{v_{2},g}^{2}/n_{g}$, where $\widehat{\sigma}_{v,g}^{2}=\frac{1}{n_{g}}\sum_{j=1}^{n_{g}}\widehat{v}_{2,jg}^{2}$. Let $Z$ be the $n\times G$ matrix of instruments. For the vector $\pi$ the OLS estimator is given by \[ \widehat{\pi}=\left(Z^{\prime}Z\right)^{-1}Z^{\prime}x=\left(\overline{x}_{1},\overline{x}_{2},...,\overline{x}_{G}\right)^{\prime}. \] Let
where $\text{Diag}\left(q_{g}\right)$ is a diagonal matrix with $g$th diagonal element $q_{g}$. Then the robust estimator of $Var\left(\widehat{\pi}\right)$ is given by
The non-robust variance estimator is
The group- (or instrument-) specific IV estimators for $\beta$ are given by
with $\overline{y}_{g}=\frac{1}{n_{g}}\sum_{j=1}^{n_{g}}y_{jg}$, and the 2SLS estimator for $\beta$ is
the standard result that $\widehat{\beta}_{2sls}$ is a linear combination of the instrument specific IV estimators, (see e.g.\ Windmeijer2019). The weights are given by
and hence the 2SLS estimator is here a weighted average of the group specific estimators.
For the group specific estimates, the first-stage F-statistics are given by
for $g=1,...,G$. As the errors $\left(u_{jg},v_{2,jg}\right)$ are iid within groups, the standard weak-instruments results of StaigerStock1997 and StockYogo2005 apply to each group-specific IV estimator $\widehat{\beta}_{g}$. As these are just-identified models, we can relate the values of the F-statistics to Wald-test size distortions.
From ((ref)) and ((ref)) it follows that the weights for the 2SLS estimator are related to the individual F-statistics as follows
Under first-stage homoskedasticity, $\sigma_{v_{2},g}^{2}=\sigma_{v_{2},s}^{2}$, for $g,s=1,\ldots,G$, then $\widehat{\sigma}_{v_{2},s}^{2}\approx\widehat{\sigma}_{v_{2},s}^{2}$ for all $g,s$, and hence $\widehat{F}\approx\frac{1}{G}\sum_{g=1}^{G}\widehat{F}_{g}$. Then the weights are given by $\widehat{w}_{2sls,g}\approx\frac{\widehat{F}_{g}}{\sum_{s=1}^{G}\widehat{F}_{\pi_{g}}}\approx\frac{\widehat{F}_{g}}{G\widehat{F}}$, so we see that the groups with the larger individual F-statistics get the larger weights in the 2SLS estimator under homoskedasticity.
This is not necessarily the case under heteroskedasticity. For equal sized groups with approximately the same value of the signal $\widehat{\pi}_{g}^{2}$, $\widehat{F}_{g}$ is larger for groups with, and because of, the smaller values of $\widehat{\sigma}_{v_{2},g}^{2}$. The 2SLS weights ignore this information and give in this example approximately equal weights to groups with similar values of $\widehat{\pi}_{g}^{2}$ . In practice it could then be the case that a small variance, large $\widehat{F}_{g}$ group could receive a small weight in the 2SLS estimator. As shown in the Monte Carlo exercises below, this is exactly what happens in the design of IAndrewsREStat2018. There is one group with a large individual F-statistic. However, this group has a very small population variance $\sigma_{v_{2},g}^{2}$ resulting in a relatively small weight in the 2SLS estimator and a poor performance of the estimator in terms of bias and size of the Wald-test.
The non-robust F-statistic for $\pi$ is given by
The effective F-statistic is given by \[ \widehat{F}_{\text{eff}}=\sum_{g=1}^{G}\frac{\widehat{\sigma}_{v_{2},g}^{2}}{\left(\sum_{s=1}^{G}\widehat{\sigma}_{v_{2},s}^{2}\right)}\widehat{F}_{g}, \] and so $\widehat{F}_{\text{eff}}=\widehat{F}$ if groups sizes are equal, $n_{g}=n/G$ for $g=1,\ldots,G$. In the designs of IAndrewsREStat2018 group sizes are equal in expectation. Both these F-statistics will therefore correctly reflect weak-instruments problems for the 2SLS estimator in these designs.
The robust first-stage F-statistic is given by
It is therefore clear, that if $\widehat{F}_{r}$ is large, then at least one of the $\widehat{F}_{g}$ is large. For the GMMf estimator we have that
with \[ \widehat{w}_{gmmf,g}=\frac{\widehat{F}_{g}}{\sum_{s=1}^{G}\widehat{F}_{s}}=\frac{\widehat{F}_{g}}{G\widehat{F}_{r}}, \] hence the groups with the larger F-statistics get the larger weights, independent of the values of $\widehat{\sigma}_{v_{2},s}^{2}$, mimicking the 2SLS weights under homoskedasticity of the first-stage errors.
We consider here the two heteroskedastic designs of IAndrewsREStat2018 with $G=10$ groups, $\beta=0$ and one with moderate and one with high endogeneity. Tables 9 and 12 in the Supplementary Appendix C.3 of IAndrewsREStat2018 present the values of the conditional group-specific variance matrices $\Sigma_{g}$ as defined in ((ref)) and the first-stage parameters, denoted $\pi_{0g}$, for $g=1,\ldots,10$. The correlation between $u_{i}$ and $v_{2i}$ is $-0.59$ in the moderate and we set it equal to $0.99$ in the high endogeneity case. We multiply the first-stage parameters $\pi_{0}$ by $0.040$ and $0.026$, such that the value of the robust $\widehat{F}_{r}$ is just over $80$ on average for $10,000$ replications and sample size $n=10,000$ in both designs. The group sizes are equal in expectation with $P\left(z_{i}=e_{g}\right)=0.1$ for all $g$. The first two rows in each panel of Table (ref) present the values of $\pi_{g}$ and $\sigma_{v_{2},g}^{2}$ for $g=1,\ldots,10$.
Tables (ref) and (ref) presents the weak-instruments test results and estimation results. We use and present the critical values based on the OLS bias benchmark, $\text{BM}_{LS}\left(\widehat{W}_{\Omega_{n}},\widehat{\Sigma}_{v}\right)$, and set $\tau=0.10$. The means of the effective F-statistics are small in both designs, and the $\widehat{F}_{\text{eff}}$-based test does not reject the null of weak instruments for the 2SLS estimator in any of the replications. This is reflected in the bias of the 2SLS estimator and the 2SLS-based Wald test for $H_{0}:\beta=0$ overrejects. The means and standard deviations of the non-robust F-statistics are virtually the same as those of the effective F-statistics, confirming the results derived in Section (ref). The means of the robust F-statistics are large, and the $\widehat{F}_{r}$-based test for weak instruments rejects the null of weak instruments in all replications. The GMMf estimator is virtually unbiased and the GMMf-based Wald test has good size properties. The means of the critical values for $\widehat{F}_{\text{eff}}$ are $17.09$ and $17.12$, whereas those for $\widehat{F}_{r}$ are $13.45$ in the moderate and $12.26$ in the high endogeneity design. In comparison, the 10% relative bias Stock and Yogo critical value for the homoskedastic case is here given by $11.46$, see SkeelsWindmeijer2018.
The details as given in Table (ref) below make clear what is happening. It reports the population values of $\pi_{g}$, $\sigma_{v_{2},g}^{2}$, $\mu_{n,g}^{2}=1000\pi_{g}^{2}/\sigma_{v_{2},g}^{2}$ and the mean values of $\widehat{F}_{g}$, $\widehat{w}_{2sls,g}$ and $\widehat{w}_{gmmf,g}=\widehat{F}_{g}/\sum_{s=1}^{G}\widehat{F}_{g}$. For the moderate endogeneity design identification in the first group is strong, with an average value of $\widehat{F}_{1}=789.5$. Identification in all other 9 groups is very weak, with the largest average value for $\widehat{F}_{5}=2.23$. The signal for group 1, $\pi_{1}^{2}$, is somewhat larger than those for the other groups, but the population value $\mu_{n,1}^{2}$ is large mainly due to the relatively very small value of $\sigma_{v_{2},1}^{2}$. As detailed in ((ref)), the 2SLS weights ignore the $\sigma_{v_{2},1}^{2}$ part of the information in group 1 which leads to the low average value of $\widehat{w}_{2sls,1}=0.127$. This shows that the 2SLS estimator does not utilize the identification strength of the first group well, with some larger weights given to higher variance, but lower concentration-parameter groups.
Table (ref) further shows that for the GMMf estimator almost all weight is given to the first group, with the average of $\widehat{w}_{gmmf,1}$ equal to $0.984$, resulting in the good behaviour of the GMMf estimator in terms of bias and Wald test size. In this case the standard deviation of the GMMf estimator is quite large relative to that of the 2SLS estimator. This is driven by the value of $\sigma_{u,1}^{2}$, which in this design is equal to $1.10$, much larger than $\sigma_{v_{2},1}^{2}$. Reducing the value of $\sigma_{u,1}^{2}$ (and the value for $\sigma_{uv_{2},1}$ accordingly to keep the same correlation structure within group 1), will reduce the standard deviation of the GMMf estimator.
The pattern of group information for the high endogeneity case is similar to that of the moderate endogeneity case, with one informative group, $g=10$, with an average value of $\widehat{F}_{10}=792.2$. However, the variance $\sigma_{v_{2},10}^{2}$ is now so small in relative terms, that the 2SLS weight for group 10 has an average value of only $\widehat{w}_{2sls,10}=0.003$. The GMMf estimator corrects this, with the average value of $\widehat{w}_{gmmf,10}=0.989$. The standard deviation of the GMMf estimates, $0.029$, is in this case smaller than that of the 2SLS estimates, $0.057$.
The left panels of Figure (ref) displays the relative bias of the 2SLS and GMMf estimators, relative to that of the OLS estimator, as a function of the mean values of the robust F-statistic $\widehat{F}_{r}$, together with the rejection frequency of the $\widehat{F}_{r}$-based test for weak instruments, using the critical values from the least-squares benchmark bias. We present the relative bias here to be in line with the homoskedastic case as presented below. Different values of $\widehat{F}_{r}$ are obtained by different values of the scalar $e$ when setting the first-stage parameters $\pi=e\pi_{0}$. The relative bias of the GMMf estimator decreases quite rapidly with increasing values of $\widehat{F}_{r}$. For the moderate endogeneity case, the test has a rejection frequency of $5\%$ at a mean $\widehat{F}_{r}$ of $10.03$, with the relative bias of the GMMf estimator at that point equal to $0.092$. As shown in the top right-hand panel of Figure (ref), the GMMf estimator based Wald test is well behaved in terms of size, with hardly any size distortion for mean values of $\widehat{F}_{r}$ larger than 5. The GMMf relative bias picture for the high-endogeneity case is very similar to that of the moderate-endogeneity case. Here the $\widehat{F}_{r}$ based test for weak instruments has a rejection frequency of $5\%$ at a mean $\widehat{F}_{r}$ of $8.98$, with the relative bias there being $0.091$. As for the homoskedastic case, where the Wald test size deviation from nominal size is larger for larger values of $\rho_{uv_{2}}^{2}$, the GMMf Wald test has a worse size performance in the high-endogeneity design, and has a $10\%$ rejection frequency at a mean $\widehat{F}_{r}$ of $26.64$. This would imply a critical value at the $5\%$ level of around $32$, which compares to the Stock and Yogo weak-instruments critical value of $38.54$ for a Wald test size of $10\%$ at the $5\%$ nominal level.
We next consider the homoskedastic design for the moderate endogeneity case with $\Sigma_{uv_{2}}=\frac{1}{G}\sum_{g=1}^{G}\Sigma_{uv_{2},g}$, resulting in \[ \Sigma_{uv_{2}}=\left[
\right], \] with $\rho_{uv_{2}}=-0.59$, as above. We consider smaller sample sizes of $n=250$ and $n=500$, or group sizes of $25$ or $50$ on average, to compare the weak-instrument finite sample behaviour of the GMMf estimator to that of the 2SLS estimator. In particular, the noise induced by estimation of $W_{2}^{-1}$ may adversely affect the GMMf estimator.
The results in Figure (ref) shows that for this design and sample sizes the relative biases and Wald rejection frequencies are virtually identical for the two estimators, with the standard deviations of the GMMf estimates slightly larger than those of the 2SLS estimator, as expected. The rejection frequencies of the $\widehat{F}_{r}$-based test are here closer to those of the standard Stock and Yogo $\widehat{F}$-based test compared to the rejection frequencies of the $\widehat{F}_{\text{eff}}$-based test, with the latter test more conservative.
From the results on the relative magnitude of the concentration parameters as discussed in Section (ref) and footnote (ref), we can change the parameter values in the grouped-data design of Section (ref) such that $\widehat{F}_{\text{eff}}>\widehat{F}_{r}$ in expectation. The design is described in Appendix (ref) where Table (ref) presents Monte Carlo results for the weak-instruments tests and estimation results. The null of weak instruments is rejected in all replications for the 2SLS estimator, whereas it is not rejected in virtually all replications for the GMMf estimator and the bias of the 2SLS estimator is smaller than that of the GMMf estimator.
As per the results of Tables (ref) and (ref), a large value of $\widehat{F}_{r}\gg cv_{r}^{LS}$ indicates that the GMMf estimator is quite well behaved in terms of bias. But when that is the case, a value of $\widehat{F}_{\text{eff}}<cv_{\text{eff}}^{LS}$, and so not rejecting the null of weak instruments for the 2SLS estimator, does not necessarily imply that the 2SLS bias is larger than the GMMf bias (and vice versa). Consider an example with fixed group sizes $\frac{n_{g}}{n}=f_{g}$, $0<f_{g}<1$, for $g=1,\ldots,G$. Then for 2SLS, \[ \mu_{2sls}^{2}\coloneqq\mu_{Q_{zz}^{-1}}^{2}=\frac{\sum_{g=1}^{G}c_{g}^{2}f_{g}}{\sum_{g=1}^{G}\sigma_{v_{2},g}^{2}}, \] and for GMMf, \[ \mu_{gmmf}^{2}\coloneqq\mu_{W_{2}^{-1}}^{2}=\frac{1}{G}\sum_{g=1}^{G}c_{g}^{2}f_{g}/\sigma_{v_{2},g}^{2}. \] Then the Nagar bias ((ref)) for the 2SLS and GMMf estimators are respectively given by \[ N_{2sls}=\frac{\sum_{g=1}^{G}\left(1-2\frac{c_{g}^{2}f_{g}}{\sum_{s=1}^{G}c_{s}^{2}f_{s}}\right)\sigma_{uv_{2},g}}{\sum_{g=1}^{G}c_{g}^{2}f_{g}} \] \[ N_{gmmf}=\frac{\sum_{g=1}^{G}\left(1-2\frac{c_{g}^{2}f_{g}/\sigma_{v_{2},g}^{2}}{\sum_{s=1}^{G}c_{s}^{2}f_{s}/\sigma_{v_{2},s}^{2}}\right)\frac{\sigma_{uv_{2},g}}{\sigma_{v_{2},g}^{2}}}{\sum_{g=1}^{G}c_{g}^{2}f_{g}/\sigma_{v_{2},g}^{2}}. \] One can find therefore parameter values in this design, such that $N_{2sls}<N_{gmmf}$ whereas $\mu_{2sls}<\mu_{gmmf}$. It follows for example, that if $c_{g}^{2}f_{g}=c_{f}$, $c_{f}>0$ for $g=1,\ldots,G$, then $N_{2sls}=0$ if $\sum_{g=1}^{G}\sigma_{uv_{2},g}=0$, irrespective of the value of $\mu_{2sls}$. In that case, there is no overall endogeneity, as $\rho_{uv_{2}}=0$. (Conversely, in the constant $c_{g}^{2}f_{g}$ case, $N_{gmmf}=0$ if $\sum_{g=1}^{G}\frac{\sigma_{uv_{2},g}}{\sigma_{v_{2},g}^{4}}=0$ irrespective of the value of $\mu_{gmmf}$).
As another example, consider again $G=10$, with $f_{g}=0.1$ for $g=1,\ldots,10$, with parameter values $c_{g}$ and group variance matrices $\Sigma_{uv_{2},g}$ given in Appendix (ref). The resulting overall endogeneity is a moderate $\rho_{uv_{2}}=0.244$. We get a value of $\mu_{gmmf}^{2}=43.09$ and a smaller value of $\mu_{2sls}^{2}=8.45$, whereas the Nagar bias approximations are virtually identical at $N_{2sls}=0.022<N_{gmmf}=0.023$. Table (ref) in Appendix (ref) presents some Monte Carlo results for this design, for $n=10,000$, confirming the theoretical results. We find there that the null of weak instruments is not rejected in virtually all replications for the 2SLS estimator, whereas it is rejected in all replications for the GMMf estimator. But the Monte Carlo bias of the 2SLS estimator, $0.022$, is slightly smaller than that of the of the GMMf estimator, $0.024$, as indicated by the Nagar bias approximations.
From these latter examples it is clear that a situation with $\widehat{F}_{r}>cv_{r}^{LS}$ and $\widehat{F}_{\text{eff}}<cv_{\text{eff}}^{LS}$ does not necessarily imply that the 2SLS estimator is more biased than the GMMf estimator (and vice versa). However, randomly sampling values $c_{g}$ and group variance matrices $\Sigma_{uv_{2},g}$, for $g=1,\ldots,10$, and collecting the first $1000$ of those sets where $\left|\rho_{uv_{2}}\right|>0.2$, $5<\mu_{2sls}<10$ and $40<\mu_{gmmf}<45$, we find for $989$, or $99\%$ of those, the Nagar bias of the 2SLS estimator to be larger than that of the GMMf estimator.
The IAndrewsREStat2018 grouped-data IV designs are quite extreme in the variation of $\sigma_{v_{2},g}^{2}$, leading to the large differences between the values of $\widehat{F}_{\text{eff}}$ and $\widehat{F}_{r}$ and between the performances of the 2SLS and GMMf estimators. Note that these results carry over to a model with a constant and a full set of mutually exclusive binary indicators as instruments, when the variances $\sigma_{v_{2},g}^{2}$ for at least two groups are relatively small and their $\pi_{g}$ coefficients are different. This is the case if we for example change $\sigma_{v_{2},2}^{2}$ in the moderate endogeneity design above to be equal to the small $\sigma_{v_{2},1}^{2}$. An example where this could be relevant is the judge fixed effects design as pioneered by Kling2006. There are many papers using judge effects as instrumental variables, Table 1 in Frandsen2023 presents a subset of these. Stevenson2018 studied the effect of pretrial detention on conviction, using judge indicators as instruments, as cases are randomly assigned to judges. As the treatment is here binary, with variance $\pi\left(1-\pi\right)$, a very lenient (small $\pi$) and a very strict judge (large $\pi$) in terms of sending defendants to pretrial detention have small values of $\sigma_{v_{2}}^{2}$, but clearly different values of $\pi$. Unlike the 2SLS estimator, the GMMf estimator takes the differential strengths of the instruments due to the different values of $\sigma_{v_{2}}^{2}$ into account, giving more weight to very lenient and very strict judges.\footnote{I would like to thank an anonymous referee for this example.}
For any single-endogenous variable application, most packages will compute the robust F-statistic for estimation results with robust standard errors. For example, in Stata, Stata, the robust first-stage F-statistic is provided with the output of “ivregress” or “ivreg2”, \citet*{ivreg2}, whereas “weakivtest”, weakivtest, calculates the effective F-statistic and critical values for the weak-instruments test. An extended version of the latter, called “gfweakivtest”\footnote{Available from https://github.com/stat0357/gfweakivtest.} also calculates the robust F-statistic and its weak-instruments critical values. It further includes the critical values based on the least-squares benchmark bias for both $\widehat{F}_{r}$ and $\widehat{F}_{\text{eff}}$, and presents the estimation results for the GMMf estimator. As per the advice of AndrewsStockSun2019, 2SLS estimation results with robust standard errors should be accompanied by the effective F-statistic and its critical value. If the situation is as in the IAndrewsREStat2018 examples above, then the GMMf estimator could be the preferred estimator.
Windmeijer2023 presents such comparisons for a study with a set of mutually exclusive binary indicators as instruments, and one of the American Economic Review studies as considered in the review paper by AndrewsStockSun2019. This is from Stephens2014 who study the effect of schooling on wages, using data from the 1960-1980 US Censuses of Population. The endogenous variable is years of schooling for individual $i$, born in state $s$ in year $t$, and the instruments are three indicator variables $RS7$, $RS8$ and $RS9$, corresponding to being required to attend seven, eight or nine or more years of schooling, respectively. All specifications include state-of-birth and year-of-birth fixed effects, and the computed standard errors are robust to heteroskedasticity and clustering at the state-of-birth/year-of-birth cell. Stephens2014 report the robust first-stage F-statistics $\widehat{F}_{r}$ in their Table 1, which presents eight sets of estimates of the returns of schooling on log weekly wages for four different samples and two different model specifications for each sample. None of the $\widehat{F}_{r}$ statistics indicate an underidentification problem. But no effective F-statistics were reported. As the estimator used is the 2SLS estimator, it is therefore important to consider whether the $\widehat{F}_{r}$ statistic misrepresents weak-instruments bias of the 2SLS estimator, in the sense that a large value of $\widehat{F}_{r}$ may not be an indicator of a good performance of the 2SLS estimator. Table 3 in Windmeijer2023 shows that this is not the case here. The $\widehat{F}_{\text{eff}}$ and $\widehat{F}_{r}$ based tests for weak instruments both reject and don't reject the null of weak instruments for the same specifications, their values are similar in magnitude and the 2SLS and GMMf estimation results are virtually identical when the null of weak instruments is rejected and the Hansen $J$-test does not indicate misspecification. This is a reassuring result for the 2SLS estimates that were accompanied by larger values of $\widehat{F}_{r}$, with the only two cases where the null of weak instruments was not rejected had values of $\widehat{F}_{r}$ equal to 8.22 and 6.34 with those of $\widehat{F}_{\text{eff}}$ equal to 8.11 and 6.13 respectively.
For models with a single endogenous explanatory variable, we have introduced a class of generalized effective F-statistics as defined in ((ref)) in relation to a class of linear GMM estimators given in ((ref)) and have shown that the OleaPflueger2013 weak-instruments testing procedure that they established for the effective F-statistic in relation to the Nagar bias of the 2SLS estimator applies to this extended class. In particular, the standard robust F-statistic is a member of this class and is associated with the behaviour in terms of Nagar bias of the GMMf estimator, which has its weight matrix based on the first-stage residuals. We then focused on a comparison of the effective F-statistic and the robust F-statistic and the associated weak-instrument behaviours of the 2SLS and GMMf estimators. In particular, we have shown that and explained why the GMMf estimator's performance is much better in terms of bias than that of the 2SLS estimator in the grouped-data designs of IAndrewsREStat2018, where the robust F-statistic can take very large values, but the effective F-statistic is very small. One should therefore in general not use the robust F-statistic to gauge instrument strength in relation to the performance of the 2SLS estimator, AndrewsStockSun2019, but as shown here, it can be used as a weak-instruments test in relation to the Nagar bias of the GMMf estimator. In practice, therefore, both the effective F-statistic and robust F-statistic should be reported, together with their critical values, and the GMMf estimator could be considered in cases where there is a clear discrepancy with a large value for the robust F-statistic rejecting the null of weak instruments, and when the effective F-statistic is small and does not reject its null of weak instruments.
We have not focused here on the wider applicability of the class of generalized effective F-statistics and their associated GMM estimators, but an example is the one-step ABond1991 GMM estimator for panel data models with a single endogenous variable. Two-step estimators do not fall in the class because of the presence of estimated structural parameters in the weight matrix, but one could test for weak instruments in this setting, fixing the parameter of the endogenous variable in the weight matrix, for example under a specific null value of interest.
A topic for future research for the general heteroskedasticity setting is an extension to the linear model with more than one endogenous variable. LewisMertens2022 is an extension of the OleaPflueger2013 method to the multiple endogenous variable case for the 2SLS estimator, but they do not consider such an extension for the wider class of GMM estimators. Future research should also address the weak-instruments Wald size properties for both the single and multiple endogenous variables settings.
I would like to thank the editor Xiaohong Chen, an associate editor and two anonymous referees for their detailed comments and suggestions that helped to improve the paper substantially. Further thanks to Benedikt Koch and seminar participants for helpful comments.