Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
10,686 characters · 4 sections · 8 citation commands
Generalized Automatic Least Squares: Efficiency Gains from Misspecified Heteroscedasticity Models
It is well known that in the presence of heteroscedasticity ordinary least squares estimator is not efficient (see Section 8.4 in wooldridge2019introductory; early proposals were made in cragg1983more,). I propose a generalized automatic least squares estimator (GALS) that makes partial correction of heteroscedasticity based on a (potentially) misspecified model. Such estimator is guaranteed to be at least as asymptotically efficient as either OLS or WLS but can provide some efficiency gains if the misspecified model is approximately correct. If the heteroscedasticity model is correct, the proposed estimator achieves full asymptotic efficiency. The idea is to frame moment conditions corresponding to OLS and WLS squares based on miss-specified heteroscedasticity as a joint generalized method of moments estimation problem. The resulting optimal GMM estimator is equivalent to a GLS with estimated weight matrix. I also propose heteroscedasticity-robust GMM variance-covariance formula to account for any remaining heteroscedasticity in the residuals.
The procedure works in three steps. First, one need to estimate the usual OLS residuals. Then one need to estimate the approximate model for heteroscedasticity. Third, one computes GALS estimator and its covariance matrix using closed-form formulas. All three steps do not involve any optimization and are very easy to compute.
In a closely related recent paper romano2017resurrecting provide a proof that WLS with estimated (potentially misspecified) heteroscedasticity is heteroscedasticity robust if one applies white1980heteroskedasticity standard errors for the inference. Further, they suggest to make a pretest to determine whether use OLS or WLS. romano2017resurrecting acknowledge that for some misspecified models of heteroscedasticity, WLS is less efficient than OLS, in which case their resulting adaptive least squares (ALS) procedure would be less efficient than OLS. My proposal instead is to avoid the pretest and use GALS estimator, which is at least as efficient as OLS or WLS and can even achieve higher efficiency that both of the two estimators.
Other researchers also considered efficiency gains of GMM over OLS from additional restrictions on higher order moments im2008more. In contrast, I do not assume additional properties of the regression residuals beyond the standard exogeneity condition and existence of approximate non-trivial heteroscedasticity model. ( The true model can be homoscedastic in which case GALS is asymptotically equivalent to OLS; the WLS moments will be will be ignored asymptotically. )
Consider a linear regression model
As usual, $E(\varepsilon_i|X_i)=0$, the vector of regressors $X_i$ has dimension $p$. Suppose that $E(\varepsilon_i^2|X_i)=\sigma_\star^2(X_i)= \exp(\sum_{k=1}^\infty \delta_k p_k(X_i))$, where $p_k(\cdot)$ are approximating basis functions like Chebyshev polynomials. As long as the heteroscedasticity function is square integrable, we can choose, for example, trigonometric polynomial or other orthogonal basis functions on the support of $X_i$.
Suppose that we pick a flexible model for heteroscedasticity given by formula $\sigma^2(X_i)$ (for example, $\sigma^2(X_i)= \exp(\sum_{k=1}^K \delta_k p_k(X_i))$). It is well known that WLS procedure corresponds to moment conditions
If the functional form is correct, i.e. $\sigma^2(X_i) = =\sigma_\star^2(X_i)$, then WLS based on (ref) would be asymptotically efficient for estimation of $\beta$ by the Gauss-Markov theorem. Of course, OLS estimator also corresponds to GMM estimator with moment conditions
Depending on how well we model heteroscedasticity, choice of (ref) may or may not improve efficiency over (ref). However, if we consider joint GMM estimation with optimal GMM-weights based on both (ref) and (ref), we are guaranteed to be at least as efficient as an estimator that uses only moments (ref)(OLS estimator) or moments (ref) ( inefficient WLS estimator). So if our model of heteroscedasticity $\sigma^2(X_i)$ was approximately correct, the joint GMM will be more efficient than OLS by the properties of the optimal GMM estimators.
Consider a joint GMM estimator based on both (ref) and (ref). It turns out that it is itself a GLS estimator, which automatically adapts to heteroscedasticity of the residuals (hence the name choice). It is given by an optimization problem
where $ \varepsilon$ is the vector of residuals, $X$ is matrix of regressors with dimension $(k\times n)$, $W$ is the optimal weight matrix (it is given explicitly in the next subsection), $D = diag(\frac{1}{\sigma^2(X_1)},\dots,\frac{1}{\sigma^2(X_n)})$. Indeed, after simplification the GLS form becomes explicit,
where $A_n = \frac{1}{n} (X'W_{11} X + X' W_{12} X D + D X' W_{21} X + D X' W_{22} X D)$ is a symmetric $n\times n$ matrix .
In practice, of course, $A_n$ has to be replaced with a consistent estimator, which is discussed in the next subsection.
As usual, the optimal GMM weight matrix $W$ is equal to inverse of the covariance matrix $\Omega$ of the moment conditions, which is given by
Suppose that matrices $ \Omega_{11} = E \varepsilon^2_i X_iX'_i$ and $(\Omega_{22}- \Omega_{12} \Omega_{11}^{-1} \Omega_{12}) $ are invertible. Then using Theorem 2.1 from lu2002inverses we can obtain eplicit block representaiton of the inverse, {\tiny
}
So matrix $A_n$ takes explicit form
A feasible version of this matrix is using blocks from
instead of $\Omega$ and $\hat D = diag(\frac{1}{\hat\sigma^2(X_1)},\dots,\frac{1}{\hat\sigma^2(X_n)})$ instead of $D$. Here $\hat\sigma^2(X_i) = \exp(\sum_{k=1}^K \hat\delta_k p_k(X_i) )$ with $\hat \delta_k$ given by OLS regression of $\log \hat \varepsilon_i^2$ on $( p_1(X_i),\dots, p_k(X_i))$. The residuals $ \hat \varepsilon_i $ come from OLS regression of $Y_i$ on $X_i$. Under regularity conditions on the moments that ensure consistency $\hat\delta_k$ and of $\hat\Omega$, one can replace $A_n$ with $\hat A_n $ in Theorem 1.