Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
48,483 characters · 12 sections · 38 citation commands
LM-BIC Model Selection in Semiparametric Models
\doublespacing
This paper considers model selection in semi- and nonparametric econometric models. It develops a consistent series-based model selection procedure for selecting between different classes of models, such as parametric, semiparametric, and nonparametric. The proposed procedure combines the features of the semiparametric LM (Lagrange Multiplier) type test developed in korolev_2018 and of the conventional parametric model selection criteria such as the BIC (Bayesian Information Criterion). It selects a model by minimizing the LM type test statistic but additionally rewards simpler models. Unlike the parametric model selection procedures, which treat the number of parameters in the model as fixed, the proposed LM-BIC procedure allows the number of parameters to grow with the sample size, thus incorporating models with nonparametric components estimated by series methods. Moreover, it takes into account that usually series methods can only approximate the true model and are misspecified for any finite number of series terms.
Model selection plays an important role in both theoretical and applied econometrics. Many papers in econometric theory have studied model selection in various settings, while applied econometricians almost always perform model selection, either formally or informally, as the true model specification is rarely, if ever, known.
Most existing model selection methods, such as the BIC or HQIC (Hannan-Quinn Information Criterion), focus on model selection in parametric models. While choosing appropriate variables in a parametric regression is an important task, it is only part of a bigger problem. Applied economists are often uncertain not only about which variables to include in a regression, but also about which class of models to use.
While parametric models are simple to estimate, easy to interpret, and yield efficient estimates when correctly specified, they may be too restrictive, fail to capture heterogeneity in the data well, and lead to inconsistent estimates under misspecification. On the other hand, nonparametric models are extremely flexible, but they may violate restrictions imposed by economic theory and suffer from the curse of dimensionality.
Semiparametric models offer an attractive middle ground by combining the flexibility of nonparametric models with parametric assumptions about certain components of the model, which alleviates the curse of dimensionality and helps achieve regular models. Nevertheless, semiparametric models yield inconsistent estimates when misspecified and inefficient estimates when the true model is parametric.
Thus, specification choice poses a big challenge for empirical work in economics, and it may be desirable to have econometric methods that help find the most appropriate model. One way of achieving this goal is specification tests, which are reasonable when the researcher has a preferred model (say, obtained from economic theory) and wants to test whether it provides a statistically adequate description of the data. But if the researcher does not have such a model and needs to choose from a wide range of models, she may want to use a model selection procedure that compares all candidate models simultaneously.
One such procedure that has long been known in econometrics is the BIC, which performs model selection in parametric models by penalizing the number of parameters to be estimated, as adding more parameters always improves fit. The BIC was first introduced by schwarz_1978 as a Bayesian solution to model selection in parametric models. hannan_quinn_1979 and hannan_1980 prove that the BIC, as well as its modification called HQIC, consistently selects the true model under fairly general assumptions. shao_1997 develops a unified asymptotic theory for various model selection procedures, including the BIC.
andrews_1999 shows that the BIC can be used for consistent moment selection in GMM (Generalized Method of Moments) models, while andrews_lu_2001 and hong_et_al_2003 extend the BIC to both model and moment selection in GMM and EL (Empirical Likelihood) estimation and prove its consistency.
Another procedure for model selection is based on the Lasso and penalizes the sum of absolute values of the parameters (i.e. the $\ell_1$ norm of the parameters vector). The Lasso was first proposed by tibshirani_1996. The conditions for its consistency in the context of parametric model selection are developed in zhao_yu_2006 and zou_2006. caner_2009 extends the Lasso to model selection in GMM estimation and proves its consistency, while liao_2013 shows that the Lasso can also be used for consistent moment selection in GMM.
As for model selection in semi- and nonparametric models, huang_et_al_2010 prove that the group Lasso can consistently select the correct regressors in nonparametric additive models. lian_2014 further shows that a BIC type criterion is also consistent in this context.
belloni_chernozhukov_2013 study the performance of the Lasso and post-Lasso OLS in high-dimensional sparse models. They analyze the selection properties of the Lasso and develop an asymptotic theory for resulting post-selection estimators.
The focus of most of these papers has been on parametric models or on additive semiparametric models, while I investigate the use of BIC-type criteria in general semiparametric contexts. For example, suppose that the researcher wants to estimate the average treatment effect of treatment $D_i$ on outcome $Y_i$ while controlling for observables $X_i$ and $Z_i$, assuming that the treatment assignment is as good as random after accounting for observable differences. A fairly general way of doing this is to estimate the nonparametric (in $X_i$ and $Z_i$) specification: \[ Y_i = \beta_1 D_i + g(X_i, Z_i) + \varepsilon_i, \quad E[\varepsilon_i | X_i, Z_i, D_i] = 0 \]
However, in practice this specification may be infeasible because of the curse of dimensionality, so the researcher may want to determine if more restricted models are appropriate. One such model which is commonly used in empirical work is fully parametric, with $g(X_i, Z_i) = \beta_0 + X_i' \gamma_1 + Z_i' \gamma_2$. Another possible specification is semiparametric , e.g. nonparametric in $X_i$ but parametric in $Z_i$, wtih $g(X_i, Z_i) = g_1(X_i) + Z_i' \gamma_2$. Yet another possible model is additively separable in $X_i$ and $Z_i$, with $g(X_i, Z_i) = g_1(X_i) + g_2(Z_i)$.
In this paper, I develop a model selection technique that uses series methods to estimate semi- and nonparametric models. It performs model selection by comparing the values of the model selection criterion that is based on the semiparametric LM test statistic and additionally penalizes more complicated models. More specifically, the model selection procedure minimizes the following model selection criterion:
where $\xi$ is the quadratic form in the semiparametric model residuals, $s$ denotes a particular series form, $r_{n,s}$ is the number of restrictions the series form $s$ imposes on the fully nonparametric model, and $\kappa_n \to \infty$ at a certain rate as $n \to \infty$.\footnote{I will formally introduce the notation in the next section.} A natural choice of $\kappa_n$ would be $\kappa_n = \ln{n}$, but other functions satisfying certain rate conditions will also work. Intuitively, $\frac{\xi_s - r_{n,s}}{\sqrt{2 r_{n,s}}}$ is the semiparametric LM type test statistic as in korolev_2018, while $\frac{r_{n,s} (\kappa_n - 1)}{\sqrt{2 r_{n,s}}}$ is the penalty term that rewards simpler models (i.e. models with large $r_{n,s}$). For the fully unrestricted nonparametric model, $r_{n,s} = 0$, and in that case $MSC(s) = 0$. Model search takes place not over individual coefficients but over the groups that correspond to different classes of models.
The remainder of the paper is organized as follows. Section (ref) introduces the model, describes the model selection problem, and introduces the model selection criterion. Section (ref) develops the asymptotic theory for the proposed model selection procedure. Sections (ref) and (ref) discuss an upward and downward testing procedures based on the LM type specification test from korolev_2018. Section (ref) studies the behavior of the proposed model selection procedure in simulations. Section (ref) applies the proposed method to the household gasoline demand dataset from yatchew_no_2001. Section (ref) concludes.
Appendix (ref) collects all tables. Appendix (ref) contains proofs of technical results.
Let $(Y_i, X_i')' \in \mathbb{R}^{1+d_x}$, $d_x \in \mathbb{N}$, $i = 1, ..., n$, be independent and identically distributed random variables with $E[Y_i^2] < \infty$. Then there exists a measurable function $g$ such that $g(X_i) = E[Y_i | X_i]$ a.s., and the nonparametric model can be written as \[ Y_i = g(X_i) + \varepsilon_i, \quad E[\varepsilon_i | X_i] = 0 \]
This nonparametric model is very flexible, but it may suffer from the curse of dimensionality or may be difficult to interpret. Because of this, the researcher may prefer more restrictive models, such as semiparametric ones. If these models are correctly specified, they are likely to provide more efficient estimates of the parameters of interest; however, if these models are misspecified, the resulting estimates will be inconsistent.
Thus, the researcher's goal is to compare different parametric, semiparametric, and nonparametric models and to select the one which is correctly specified yet parsimonious. A semiparametric model $c \in \mathcal{C}$ can be written as \[ Y_i = f_c(X_i, \theta_c, h_c) + e_{c,i} \] where $f_c: \mathcal{X} \times \Theta_c \times \mathcal{H}_c \to \mathbb{R}$ is a known function, $\theta_c \in \Theta_c \subset \mathbb{R}^{d_c}$ is a finite-dimensional parameter, and $h_c \in \mathcal{H}_c = \mathcal{H}_{1,c} \times ... \times \mathcal{H}_{q_c,c}$ is a vector of unknown functions. Let $\mathcal{C}^*$ denote the set of correctly specified models. Formally, \[ \mathcal{C}^* = \{ c \in \mathcal{C} : P_{X} \left(g(X_i) = f_c(X_i, \theta_{c,0}, h_{c,0}) \right) = 1 \text{ for some } \theta_{c,0} \in \Theta_c, h_{c,0} \in \mathcal{H}_c \} \]
Several different models can be correctly specified simultaneously, and the researcher's goal is to find the most parsimonious of them, i.e. the one with the lowest dimensionality of the nonparametric component or with the fastest convergence rate.\footnote{I will define what precisely consistent model selection means in semi- and nonparametric models later.} Before discussing the model selection problem in more detail, I will introduce series methods that will be used in estimation and model selection.
As in korolev_2018, I use series for the purposes of estimation and model selection. For any variable $z$, let $Q^{a_n}(z) = (q_1(z), ..., q_{a_n}(z))'$ be an $a_n$-dimensional vector of approximating functions of $z$, where the number of series terms $a_n$ is allowed to grow with the sample size $n$. Possible choices of series functions include:
Multivariate power series or splines can be formed from products of univariate ones.
Because the class of models $\mathcal{C}$ that are allowed includes semiparametric models and because I use series methods to estimate different models, the formal statement of model selection problem is tricky. In particular, all models written in series forms will only provide approximations to semiparametric models and will most likely be misspecified for any fixed number of series terms. However, if a given series form can approximate a particular model that is correctly specified, this misspecification should disappear as the number of series terms grows to infinity. In this section I formally state the model selection problem and define what consistent model selection means for semiparametric models.
To fix ideas, it is useful to consider model selection in parametric models first and contrast it with model selection in semiparametric models. Usually model selection procedures seek to achieve two goals: select a model that is (a) correctly specified and (b) parsimonious. In parametric models, the model selection problem can usually be formulated as follows. Suppose that the model is given by \[ Y_i = \sum_{j=1}^{k}{X_{ji} \beta_j} + \varepsilon_i, \quad E[\varepsilon_i | X_i] = 0, \] and some of the $\beta_j$s can be zero. Then the goal of model selection is to find out which $\beta_j$s are equal to zero and select the model which includes all of the relevant regressors (i.e. is correctly specified) but none of the irrelevant ones (i.e. is parsimonious).
It might be natural to think of model selection in semiparametric models as the problem of choosing $a_n$ or picking nonzero elements in the series expansion $h(z) = \sum_{j=1}^{a_n}{q_j(z) \beta_j}$, but this is not what I am doing here. The moment selection problem is semiparametric models will be formulated differently, but it will still seek to combine parsimoniousness and correct specification.
First, fix $a_n$, the number of series terms used in series expansions of different semiparametric models. This fixed $a_n$ will lead to a certain number of series terms $m_{n,s}(a_n)$ in every series form $s \in \mathcal{S}$ under consideration. The model selection procedure will treat $a_n$ as fixed and will aim to select the most parsimonious model, i.e. the model with the lowest $m_{n,s}(a_n)$.
But, of course, it is not enough to select the simplest model; the selected model should also be correctly specified. As discussed above, if the true model is semi- or nonparametric, no series form will be exactly correct for any fixed $a_n$. Thus, to define what correct specification means in semiparametric settings, I will let $a_n \to \infty$ and look for series forms that can approximate the true semiparametric model well as $a_n$ grows.
In other words, the goal is to select the series form that (a) can approximate the true model as the number of series terms $a_n$ in every series expansion grows; (b) for each fixed $a_n$, has the lowest number of terms $m_{n,s}(a_n)$ among all models that satisfy (a). Note that the nonparametric model $Y_i = g(X_i) + \varepsilon_i$, written in a series form, will always satisfy (a), but it will have the largest number of parameters $m_{n,s}(a_n)$ among all models for a given $a_n$.
I now state the model selection problem formally. Write model $c$ in series form $s$ as
where $W_{s,i} := W_s^{m_{n,s}}(X_i) = (W_{s,1}(X_i), ..., W_{s,m_{n,s}}(X_i))'$ are appropriate regressors or basis functions that are used in the series form $s$, $R_{c,s,i} := \left( f_c(X_i,\theta_c,h_c) - W_{s,i}' \beta_{1,s} \right)$ is the approximation error, and $\eta_{c,s,i} := e_{c,i} + R_{c,s,i}$ is the composite error term.
For a given series form $s$ and number of series terms $m_{n,s}(a_n)$, the estimate of $\beta_{1,s}$ is \[ \tilde{\beta}_{1,s} = (W_{s}' W_{s})^{-1} W_{s}' Y, \] where $W_{s} = (W_{s,1}, ..., W_{s,n})'$, $Y = (Y_1, ..., Y_n)'$.
Each semiparametric model $c$ can be written in a series form $s$ defined by the series terms used $W_{s}^{m_{n,s}}(x)$ and the number of series terms $m_{n,s}(a_n)$ as a function of $a_n$. However, not all series forms $s$ provide a good approximation of the model $c$. Among all series forms $s \in \mathcal{S}$, I need to define the ones that correspond to the semiparametric model $c$. Let \[ \mathcal{S}^*(c) = \{ s \in \mathcal{S}: \sup_{x \in \mathcal{X}}{| f(x,\theta_{c},h_{c}) - W_s^{m_{n,s}}(x)' \beta_{1,s} |} \to 0 \quad \text{as} \quad a_n \to \infty \} \] be all series forms $s$ that can approximate a given semiparametric model $c$.
Next, I consider all correctly specified models $c \in \mathcal{C}^*$ and define all series forms that can approximate at least one correctly specified model. Namely, let \[ \mathcal{S}_u^* = \cup_{c \in \mathcal{C}^*}{ \mathcal{S}^*(c) } \] be all “correctly specified” series forms. In other words, $\mathcal{S}_u^*$ includes all series forms $s$ that can approximate some correctly specified model $c$ as the sample size grows.
As I will show, the model selection procedures I propose will select the most parsimonious member of $\mathcal{S}_u^*$, i.e. the series form $s \in \mathcal{S}_u^*$ with the smallest $m_{n,s}(a_n)$ for every fixed $a_n$.
Next, I introduce additional series terms $T_{s,i} := T_s^{r_{n,s}}(X_i)$ and let $P_i := P^{k_{n,s}}(X_i) := (W_{s,i}', T_{s,i}')'$, so that $P_i$ can approximate a fully nonparametric model as $a_n \to \infty$. Note, however, that $a_n$ will be treated as fixed by the model selection procedure, so that it is possible to compare the complexity of different models $m_{n,s}(a_n)$. Here $m_{n,s}(a_n)$ is the number of series terms in the series form $s$ and $r_{n,s}(a_n)$ is the number of restrictions that the series form $s$ imposes on a fully nonparametric model for a given $a_n$. The model selection procedure is based on the LM type quadratic form, as in korolev_2018:\footnote{For any vector $V_i$, let $V = (V_1, ..., V_n)'$ be the matrix that stacks all observations together.} \[ \xi_s = \tilde{\varepsilon}_s' P (\tilde{\sigma}_s^2 P'P)^{-1} P' \tilde{\varepsilon}_s, \] where $\tilde{\varepsilon}_s = Y - W_s \tilde{\beta}_{1,s}$ are the semiparametric residuals for the series form $s$ and $\tilde{\sigma}_s^2 = \tilde{\varepsilon}_s' \tilde{\varepsilon}_s/n$. A heteroskedasticity robust version of the LM-BIC procedure is based on \[ \xi_{HC,s} = \tilde{\varepsilon}_s' \tilde{T}_s (\tilde{T}_s' \tilde{\Sigma}_s \tilde{T}_s)^{-1} \tilde{T}_s' \tilde{\varepsilon}_s, \] where $\tilde{\Sigma}_s = diag(\tilde{\varepsilon}_{s,i}^2)$ and $\tilde{T}_s = M_{W_s} T_s = (I - W_s (W_s' W_s)^{-1} W_s) T_s$ are the residuals from the regression of each element of $T_{s,i}$ on $W_{s,i}$.
This form is analogous to the parametric LM test statistic or overidentifying restrictions test statistic, but the dimensionality of $P$, $k_n$, is allowed to grow with $n$ to accommodate semi- and nonparametric models. The model selection criterion selects the model with the lowest value of the LM test statistic but additionally rewards parsimonious models that impose more restrictions: \[ MSC(s) = \frac{\xi_s - r_{n,s} \kappa_n}{\sqrt{2 r_{n,s}}}, \] where $r_{n,s}$ is the number of restrictions the series form $s$ imposes on the fully nonparametric model and $\kappa_n \to \infty$ as $n \to \infty$. The heteroskedastiticy robust model selection criterion is \[ MSC_{HC}(s) = \frac{\xi_{HC,s} - r_{n,s} \kappa_n}{\sqrt{2 r_{n,s}}} \]
I impose the following assumptions that are similar to the ones in korolev_2018.
Next, I define the error term and impose two moment conditions.
The following assumption deals with the behavior of the approximating series functions. From now on, let $|| A || = [tr(A'A)]^{1/2}$ be the Euclidian norm of a matrix $A$.
The following assumption states that series methods should be able to approximate unknown functions sufficiently well as the number of series terms grows.
Then I can use the following result.
These assumptions will allow me to use the results from korolev_2018, e.g. Theorems 1, 2, and 3. Then I obtain the following result.
This theorem makes two statements. First, the selected series form $\hat{s}$ approximates a semiparametric model $c$ that is correctly specified. Second, the selected series form $\hat{s}$ has the lowest dimension among all series forms that can approximate correctly specified models.
A similar result can be obtained for the heteroskedasticity robust model selection procedure:
In this and next sections I study testing procedures that sequentially apply the LM type specification test from korolev_2018 with adjusted critical values to different models. The upward testing (UT) procedure considered in this section starts from the most restrictive (e.g. fully parametric) model and carries out tests with progressively smaller $r_{n,s}$, the number of restrictions, until it finds a model that is not rejected.
Let \[ t_{s,m_{n,s}} = \frac{\xi_{s,m_{n,s}} - r_{n,s}}{\sqrt{2 r_{n,s}}} \] be the value of the LM type $t$ test statistic for model $s$ when we use $m_{n,s}(a_n)$ series terms. Let $\gamma_n > 0$ denote the critical value for the UT test. Define $\hat{m}_{n}^{UT}$ such that \[ \min_{s \in \mathcal{S}: m_{n,s} = \hat{m}_{n}^{UT}}{t_{s,m_{n,s}}} \leq \gamma_n \quad \text{and} \quad \min_{s \in \mathcal{S}: m_{n,s} < \hat{m}_{n}^{UT}}{t_{s,m_{n,s}}} > \gamma_n \]
Let \[ \hat{s}^{UT} = \operatorname*{arg\,min}_{s \in \mathcal{S}: m_{n,s} = \hat{m}_{n}^{UT}}{t_{s,m_{n,s}} } \]
In other words, $\hat{m}_{n}^{UT}$ is such that the null is rejected for all series forms with $m_{n,s} < \hat{m}_{n}^{UT}$ but is not rejected for at least one series form with $m_{n,s} = \hat{m}_{n}^{UT}$. $\hat{s}^{UT}$ is such that it minimizes the LM type test statistic among all series forms $s$ with $m_{n,s} = \hat{m}_{n}^{UT}$.
The downward testing (DT) procedure considered in this sections starts from the least restrictive (e.g. fully nonparametric) model and carries out tests with progressively larger $r_{n,s}$, the number of restrictions, until it finds a model that is rejected.
Define \[ \hat{m}_{n}^{DT} = \min_{m \in \mathcal{M}_0^{**}}{m}, \text{ where } \mathcal{M}_0^{**} = \{ m^{**} : \min_{s \in \mathcal{S}: m_{n,s} =m^{***}}{t_{s,m_{n,s}}} \leq \gamma_n \text{ for all } m^{***} \geq m^{**} \} \] and \[ \hat{s}^{DT} = \operatorname*{arg\,min}_{s \in \mathcal{S}: m_{n,s} = \hat{m}_{n}^{DT}}{t_{s,m_{n,s}} } \]
In other words, $\mathcal{M}_0^{**}$ is such that for each $m^{**} \in \mathcal{M}_0^{**}$ and each $m^{***} \geq m^{**}$ there is a series form $s$ with $m_{n,s} = m^{***}$ which is not rejected. $\hat{m}_{n}^{DT}$ is the smallest element of that set. $\hat{s}^{DT}$ is such that it minimizes the LM type test statistic among all series forms $s$ with $m_{n,s} = \hat{m}_{n}^{DT}$.
I need to make as additional assumption to ensure that the DT procedure is consistent.
Intuitively, because the DT procedure starts with the largest model and tries smaller and smaller $m_{n,s}$, it will only arrive at the smallest correctly specified model if there are no “gaps” in the series form space. In other words, there is no $m_{n,s}$ that is larger than the smallest correctly specified one for which there is no correctly specified model.
I study the finite sample performance of the LM-BIC and UT model selection procedures in the treatment effects estimation setting. I skip the DT procedure because it requires an additional condition. Suppose that the researcher observes an outcome variable $Y_i$, a treatment dummy variable $D_i$, and two control variables $X_i$ and $Z_i$. The researcher wants to estimate the treatment effects parameter $\beta$ in the model: \[ Y_i = D_i \beta + g(X_i, Z_i) + \varepsilon_i, \quad E[\varepsilon_i | D_i, X_i, Z_i] = 0 \]
The function $g(X_i, Z_i)$ is unknown, and the researcher wants to obtain a consistent and efficient estimate of $\beta$. She wants to choose among five possible models:
I generate I consider five data generating processes, each of which corresponds to one of the models above:
In order to use the LM-BIC model selection procedure to choose a model, the researcher has to estimate every model by series methods, replacing all unknown functions with their finite series expansions. In this section, I use the LM-BIC criterion with $\kappa_n = \ln{n}$ and the upward testing procedure with $\gamma_n = 0.05 \sqrt{n}$ to select the best model. I consider two sample sizes, $n = 1,000$ and $n=5,000$. I use power series and use $a_n = 1.5 n^{0.15}$ series terms to approximate each one-dimensional unknown function. For the two-dimensional function, I use $a_n^2$ series terms.
Tables (ref)--(ref) show the simulated selection probabilities for each of the five data generating processes and the means and mean squared errors of the estimates of $\beta$ in different models. As we can see, the LM-BIC procedure does very well with the sample size $n=5,000$, selecting the “correct” model (i.e. the most parsimonious of all correctly specified models) at least 99% of the time under all five DGPs. The UT procedure does not perform as well, selecting the right model only 83% of the time under the semiparametric additive DGP. Both procedures have hard time selecting the correct model under the additive DGP with $n=1,000$, selecting the semiparametric additive model less than 25% of the time. The UT procedure generally selects the right model less frequently than the LM-BIC procedure, but there is one exception. Under the nonparametric DGP with $n=1,000$, the LM-BIC procedure selects the right model 81% of the time, while the UT procedure always selects it.
Given these simulated model selection probabilities, it is not surprising that post-selection estimates of $\beta$ based on the LM-BIC procedure have lower MSE than the ones based on the UT procedure. Moreover, the MSE of the estimates of $\beta$ based on the LM-BIC is very close to that of the oracle estimator, especially with the sample size $n=5,000$. However, under the nonparametric DGP with $n=1,000$, the post-selection estimate of $\beta$ based on the UT procedure has noticeably lower MSE than the one based on the LM-BIC.
Overall, these results suggest that in most scenarios, the LM-BIC procedure slightly outperforms the UT procedure, but both seem to have good finite sample properties.
In this section, I apply the proposed model selection method to the Canadian household gasoline demand dataset from yatchew_no_2001. They estimate gasoline demand ($y$), measured as the logarithm of the total distance driven in a given month, as a function of the logarithm of the gasoline price ($PRICE$), the logarithm of the household income ($INCOME$), the logarithm of the age of the primary driver of the car ($AGE$), and other variables ($z$), which include the logarithm of the number of drivers in the household ($DRIVERS$), the logarithm of the household size ($HHSIZE$), an urban dummy, a dummy for singles under 35 years old, and monthly dummies.
yatchew_no_2001 used several parametric and semiparametric specifications to estimate gasoline demand. The inadequacy of parametric models and importance of semiparametric ones in gasoline demand estimation was first pointed out by hausman_newey_1995 and schmalensee_stoker_1999. yatchew_no_2001 follow these papers in using semiparametric specifications for gasoline demand. In korolev_2018, I apply the semiparametric LM type specification test to one of their models, which is nonparametric in $AGE$ but is parametric in all other variables, and find no evidence against it.
In this paper, I use a different approach. I consider seven candidate models:
I estimate each of these seven models using series methods, replacing all unknown functions with their finite series expansions, and compute the value of the model selection criterion as in Equation (ref), both for the homoskedastic and heteroskedastic versions of the LM type test statistic. Following korolev_2018, to construct the regressors $P^{k_n}$ used to evaluate the test statistic, I use $a_n = 3$ power series terms in $AGE$, $PRICE$, and $INCOME$, $j_n = 2$ power series terms in $DRIVERS$ and $HHSIZE$, and the set of dummies discussed above. I then use pairwise interactions (tensor products) of univariate power series, and add all possible three, four, and five element interactions between $AGE$, $PRICE$, $INCOME$, $DRIVERS$, and $HHSIZE$, without using higher powers in these interaction terms to avoid multicollinearity. Table (ref) presents the results.
As we can see, both the homoskedastic and heteroskedastic versions of the LM-BIC model selection criterion choose the sepimarametric model (ref), which is nonparametric in $AGE$ but parametric in all other variables. This is in line with the conclusions of yatchew_no_2001, who conclude that it is $AGE$ that nonlinearly affects gasoline demand, and of korolev_2018, who finds no evidence against model (ref) when tested against model (ref).
However, if I use the UT or DT procedure with $\gamma_n = 0.05 \sqrt{n} = 3.95$ assuming that the errors are homoskedastic, then I end up selecting the fully parametric model. In contrast, if I assume that the errors are heteroskedastic, then the parametric model is rejected, and the semiparametric model (ref) is selected.
This example illustrates a couple of potential issues with the proposed model selection procedures. First, even though asymptotically all procedures are consistent, in finite samples they may select different models. More specifically, if one assumes that the errors are homoskedastic, the parametric model ends up being selected by the UT and DT procedures, despite being rejected by the LM type specification test and not being selected by the LM-BIC procedure.
Second, the critical value $\gamma_n$ is somewhat arbitrary, and the UT and DT procedures may be very sensitive to the choice of critical values. If I chose a slightly higher $\gamma_n$, the parametric model would be selected by the UT and DT procedures even in the heteroskedastic case. If I chose a slightly lower value of $\gamma_n$, the parametric model would not be selected by these procedures even in the homoskedastic case.
While these two issues are important, my model selection procedure is not the only one affected by them. In fact, many existing model selection procedures are heavily dependent on tuning parameters choice, and developing ways to choose tuning parameters optimally in model selection may be an important avenue for future research.
In this paper, I develop a new model selection procedure for parametric, semiparametric, and nonparametric models. It combines the semiparametric LM type test from korolev_2018 and the BIC model selection procedure. The LM-BIC procedure aims to select the model with the lowest value of the semiparametric LM type test statistic but additionally penalizes complicated models. I prove that the resulting procedure is consistent, in the sense that it selects the lowest dimensional correctly specified model. I also develop an upward testing and downward testing procedures based on the semiparametric LM type specification test and prove their consistency. The proposed LM-BIC and UT procedures perform well in simulations.
In future work, it would be interesting to study how to select the number of terms in series expansions that are used to estimate different models and how to optimally choose $\kappa_n$, the penalty parameter for the LM-BIC, and $\gamma_n$, the critical value for the UT and DT procedures. It would also be important to develop methods for inference on post-selection estimates.
\setcounter{theorem}{0}
\setcounter{lemma}{0}
\setcounter{assumption}{0}
\setcounter{remark}{0}