A multivariate extension of the Misspecification-Resistant Information Criterion

A multivariate extension of the Misspecification-Resistant Information Criterion.

\def\spacingset#1{ {#1}} \spacingset{1}

\if00 \fi

\if10 {

} \fi

{\it Keywords:} information criteria, model selection,multivariate time series, Mean Square Prediction Error.

\spacingset{1.45}

Introduction

The model selection step is a fundamental task in statistical modelling and its implementation typically depends upon the objective of the exercise. In the time series framework the focus is on either forecasting future values or describing/controlling the process that has generated the data (DGP). A good model selection criterion must feature a good ability to identify the model with the “best” fit to future values, in a specified sense. In particular, in the parametric time series framework, we can identify two main properties. The first one is consistency, i.e., the ability to select the true DGP with probability one as the sample size diverges. This assumes that a true model exists and is among the set of candidate models. If either the set of candidate models does not contain the true DGP, or, for some reason, a true model cannot be postulated, then a selection criterion should be asymptotically efficient, for instance, in the mean square sense, i.e. it minimizes the mean squared prediction error as the sample size diverges. Starting from the seminal work of Akaike, AKA1973 a plethora of model selection criteria has been proposed. These include Akaike's AIC AKA1973, AKA1974, Schwarz's Bayesian Information Criterion (BIC) SCH1978, and Rissanen's Minimum Description Length (MDL) RIS1978. Such criteria paved the way for various extensions dealing with different unsolved issues. For instance, the AIC is efficient but not consistent (i.e. it leads to select overfitting models), whereas the BIC is consistent but not efficient, see HSU2019 for a discussion. A recent development for model selection in possibly misspecified parametric time series models in the fixed-dimensionality setting is given by the Misspecification-Resistant Information Criterion (hereafter $\operatorname{MRIC}$) HSU2019. Fixed-dimensionality means that the number of observations increases to infinity while the number of ‘true’ parameters is finite. In this respect, the MRIC provides a solution to the original research question of Akaike: it enjoys both consistency, in case the true model is included as a candidate, and asymptotic efficiency when a true model either cannot be assumed or is not included. Moreover, when the number of variables in the model grows with the sample size, the $\operatorname{MRIC}$ can achieve asymptotic efficiency, without the need for additional criteria. Finally, in the high-dimensional setting, the $\operatorname{MRIC}$ can be used together with appropriate model selection criteria to identify the best predictive models. The $\operatorname{MRIC}$ is based upon the additive decomposition of the mean squared prediction error in a term that depends upon the misspecification level and a term that measures the sampling variability of the predictor. The idea is to select the model with smallest variability among those that minimize the misspecification index. The appealing properties of the $\operatorname{MRIC}$ make it an ideal tool for omnibus time series model selection but, to date, only the univariate response case has been studied HSU2019. In this work we extend the $\operatorname{MRIC}$ to multivariate time series with a single regressor as to obtain the vectorial MRIC (hereafter $\operatorname{VMRIC}$). As it will be clear, such an extension does not easily derive from the univariate case since it requires dealing with the dependence structure within the components of the vector of forecasting error and hence relies upon random matrix theory. Such multivariate extension can be used in all those models where many time series depend upon a single regressor, like for instance, in econometrics, where many interest rates depend upon a single macroeconomic indicator, such as inflation. Other possible applications include dimension reduction and hedging, which is intimately connected to the problem of model selection BES16. The rest of the paper is organized as follows: in Section (ref) we introduce the notation and in Section (ref) summarize the available results for the univariate case; in Section (ref) we extend the MRIC approach to multivariate time series with a single regressor. In particular, in Section (ref) we obtain the asymptotic decomposition of the Mean Squared Prediction Error (hereafter MSPE) matrix into two parts: the first one is linked to the goodness of fit of the model and the second one depends upon the prediction variance. In Section (ref) we present the $\operatorname{VMRIC}$ and derive a consistent estimator for it, whereas in Section (ref), we prove the asymptotic efficiency of the $\operatorname{VMRIC}$. Section (ref) presents an example to assess the effect of misspecification in the $\operatorname{VMRIC}$ framework. All the proofs are detailed in Section (ref). (ref) contains auxiliary technical lemmas.

Notation and preliminaries

For each $t$, let $\{\mathbf{x}_t\}$ and $\{\mathbf{y}_t\}$, with $\mathbf{x}_t=(x_{t,1},\dots,x_{t,m})^\top$ and $\mathbf{y}_t=(y_{t,1},\dots,y_{t,w})^\top$, be two weakly stationary stochastic processes defined over the probability space $\left(\Omega, \mathcal{F}, \mathbb{P}\right)$. When $m=1$ ($w=1$, respectively) we write $x_t$ ($y_t$). Given a vector $\mathbf{v}$ and a matrix $\mathbf{M}$, we use $\left\lVert\mathbf{v}\right\rVert$ and $\left\lVert\mathbf{M}\right\rVert$ to refer to the $\mathcal{L}_2$ vectorial norm and the matrix norm induced by the Euclidean norm, respectively. We write $o(1)$ ($o_p(1)$) to indicate a sequence that converges (in probability) to zero and $O(1)$ ($O_p(1)$) to indicate a sequence that is bounded (in probability). Moreover, let $\{c_n\}$ be a sequence of scalar random variables whereas $\{\mathbf{v}_n\}$ and $\{\mathbf{M}_n\}$ are sequences of random vectors and random matrices, respectively. We adopt the following notation: $\mathbf{v}_n =o_p(c_n)$ if $\left\lVert\mathbf{v}_n\right\rVert/c_n=o_p(1)$; $\mathbf{v}_n=O_p(c_n)$, if $\left\lVert\mathbf{v}_n\right\rVert/c_n=O_p(1)$, $\mathbf{M}_n =o_p(c_n)$ if $\left\lVert\mathbf{M}_n\right\rVert/c_n=o_p(1)$; $\mathbf{M}_n = O_p(c_n)$ if $\left\lVert\mathbf{M}_n\right\rVert/c_n=O_p(1)$. For further details on matrix algebra see SEB2008, HOR2013, ODE2018, for multivariate time series see REI1993, LUT2005, TSA2013, and for asymptotic tools for vector and matrices, see JIA2010. Let $\{(\mathbf{x}_t,\mathbf{y}_t),t \in \{1,\dots,n\}\}$ be the observed sample, and divide the interval $[1,n]$ into the training set $[1,N]$ and the test set $[N+1,N+h]$, with $h$ being the forecasting horizon. Note that $\mathbf{x}_t$ can contain both endogenous and exogenous variables, therefore, Model ((ref)) encompasses many different models including, inter alia, VAR and VARX models. We denote $\bar{\mathbf{x}}=n^{-1}\sum_{t=1}^{n}\mathbf{x}_t$ and $\bar{\mathbf{y}}=n^{-1}\sum_{t=1}^{n}\mathbf{y}_t$, i.e. the two sample means. Without loss of generality assume $\operatorname{E}[\mathbf{x}_t]=\operatorname{E}[\mathbf{y}_t]=\boldsymbol 0$. In order to forecast $\mathbf{y}_{n+h}$, $h\geq 1$, we adopt the following $h$-step ahead forecasting Model:

where $\mathbf{B}_h$ is a $(w\times m)$ matrix and $\boldsymbol{\varepsilon}_t^{(h)}$ is the vector containing the $w$ $h$-step ahead forecast errors; as before, if $w=1$ we write $\varepsilon_t^{(h)}$.

Define

Then, the ordinary least squares estimator (hereafter OLS) of $\mathbf{B}_h$ results:

When $w=1$, $\mathbf{R}$ and $\mathbf{B}$ become $R$ and $\boldsymbol\beta$, respectively. The prediction of $\mathbf{y}_{n+h}$, $h\geq 1$, is given by

and the corresponding Mean Squared Prediction Error matrix is

The MRIC for parametric univariate time series models

In HSU2019, the authors focused on the case $w=1$ and $m\geq1$. Under appropriate conditions, they obtained the following asymptotic decomposition of $\operatorname{MSPE}$:

where $\mathbf{C}_{h,s} = \operatorname{E}\left[\mathbf{x}_1\mathbf{x}_{1+s}^\top \varepsilon_1^{(h)} \varepsilon_{1+s}^{(h)}\right]$, $s\geq0$, is the cross-covariance matrix between the regressors and the $h$-step ahead prediction error at lag $s$.

Based upon the above decomposition, the $\operatorname{MRIC}$ is defined as follows:

with $\hat{\operatorname{MI}}_h$ and $\hat{\operatorname{VI}}_h$ being the estimators of $\operatorname{MI}_h$ and $\operatorname{VI}_h$ respectively, i.e.: \[ \hat{\operatorname{MI}}_h = N^{-1}\sum_{t=1}^{N}\left(\hat{\varepsilon}_t^{(h)}\right)^2,\quad \hat{\operatorname{VI}}_h = \operatorname{tr}\left(\hat R^{-1}\hat{\mathbf{C}}_{h,0}\right) + 2 \sum_{s=1}^{h-1} \operatorname{tr}\left(\hat R^{-1} \hat{\mathbf{C}}_{h,s}\right), \] where $\hat{\mathbf{C}}_{h,s} = (N-s)^{-1} \sum_{t=1}^{N-s}\mathbf{x}_t\mathbf{x}_{t+s}^\top \hat{\varepsilon}_t^{(h)}\hat{\varepsilon}_{t+s}^{(h)}$ and $\hat{\varepsilon}_t^{(h)}=y_{t+h}-\hat{\boldsymbol{\beta}}_n(h)\mathbf{x}_t$ is the estimated forecast error; $\alpha_n$ is a penalization term sequence such that, as $n$ increases:

It is shown that $\hat{\operatorname{MI}}_h$ and $\hat{\operatorname{VI}}_h$ are consistent estimators of ${\operatorname{MI}}_h$ and ${\operatorname{VI}}_h$, moreover the asymptotic efficiency of the $\operatorname{MRIC}$ is proved. By minimizing this criterion, the model which minimizes $\operatorname{VI}$ among those with minimum $\operatorname{MI}$ is selected. Among other features, the $\operatorname{MRIC}$ is particularly helpful in situations where competing models present the same goodness-of-fit and the same number of parameters.

A multivariate extension of the MRIC framework

In this section we extend the $\operatorname{MRIC}$ approach to the case where the response is a multivariate time series ($w\geq2$) and the predictor is univariate ($m=1$), for a generic $h$-step ahead forecast. Hence, Model ((ref)) reduces to $\mathbf{y}_{t+h}=\boldsymbol{\beta}_h x_t+ \boldsymbol{\varepsilon}_t^{(h)}$, namely:

Asymptotic decomposition of the MSPE matrix

We extend the asymptotic representation of the $\operatorname{MSPE}_h$ defined in ((ref)) which is the key step to derive the $\operatorname{VMRIC}$ in this multivariate framework. We rely upon the following assumptions, which are the natural multivariate extensions of those in HSU2019.

VMRIC and its consistent estimation

In this section we introduce the $\operatorname{VMRIC}$. Let $\{\alpha_n\}$ be the penalization term sequence defined as in Eq. ((ref)).

The $\operatorname{VMRIC}$ can be estimated via the method of moments as to obtain:

and $\hat{\mathbf{C}}_{h,s} = (N-s)^{-1} \sum_{t=1}^{N-s} x_t x_{t+s} \hat{\boldsymbol{\varepsilon}}_t \hat{\boldsymbol{\varepsilon}}_{t+s}^\top$, and $\hat{\boldsymbol{\varepsilon}}_t=\mathbf{y}_{t+h}-\hat{\boldsymbol{\beta}}_n(h) x_t$ is the estimated forecast error vector. In Theorem (ref) we prove that $\hat{\operatorname{\textbf{MI}}}_h$ and $\hat{\operatorname{\textbf{VI}}}_h$ are consistent estimators of ${\operatorname{\textbf{MI}}}_h$ and ${\operatorname{\textbf{VI}}}_h$, respectively. Theorem (ref) relies upon the following assumptions, that are less restrictive with respect to (C1) -- (C6). For further discussions on the assumptions see HSU2019.

Asymptotic efficiency

In this section we prove the asymptotic efficiency of the $\operatorname{VMRIC}$ in the fixed dimensionality framework. To this end, let $\mathcal{M}$ be the set of $K$ candidate models; each model is indicated either by $\ell$ or $\kappa$, $1\leq \ell,\kappa\leq K$. Define the subsets $M_1$ and $M_2$ as follows:

In short, for a given forecast horizon $h$, $M_1$ contains the models with the minimum $\operatorname{\textbf{MI}}_h$ whereas in $M_2$ we are minimizing $\operatorname{\textbf{VI}}_h$ among the candidates models in $M_1$. The definition of efficiency used in our framework is the same as that of HSU2019:

The $\operatorname{VMRIC}$ selects the model with the smallest variability index among those that achieve the best goodness of fit. Hence, the selected model $\hat\ell_h$ is such that:

In the next Theorem we show that the $\operatorname{VMRIC}$ is an asymptotic efficient model selection criterion in the sense of Definition (ref).

Example: a misspecified bivariate AR(2) model

The aim of this section is twofold. First, we assess the goodness of the theoretical derivations and the finite sample behaviour of the method of moments estimator for the $\operatorname{VMRIC}$. Second, we show that in presence of misspecification the $\operatorname{VMRIC}$ leads to selecting the best predictive model (i.e. is asymptotically efficient) whereas both the $\operatorname*{AIC}$ and the $\operatorname*{BIC}$ fail to do so. In order to achieve the goals we consider a bivariate AR(2) DGP and use two misspecified predictive models for it: in Model 1 there is one omitted lagged predictor, whereas Model 2 uses only one non-informative predictor. We derive theoretically the Mean Square Prediction Error matrix and the $\operatorname{VMRIC}$ for both models and these show that Model 1 is a better predictive model over Model 2. Based on this, we assess the ability of the $\operatorname{VMRIC}$, and of the multivariate versions of the $\operatorname*{AIC}$ and $\operatorname*{BIC}$ to select the best model (Model 1) in finite samples and for different parameterizations. We start by providing the definition of misspecification. Consider an increasing sequence of $\sigma$-fields, $\left\{ \mathcal{G}_t \right\}$ such that $\sigma\left( \mathbf{x}_s, s\leq t \right) \subseteq \mathcal{G}_t \subseteq \mathcal{F} $, where $\left\{ \mathbf{x}_t \right\}$ is an $m$-dimensional weakly stationary process defined over the probability space $\left(\Omega, \mathcal{F}, \mathbb{P} \right)$.

Consider the following DGP :

where $\boldsymbol{a}\neq \boldsymbol{0}$, $\left\{ \boldsymbol{\varepsilon}_t \right\}$ is a sequence of independent and identically distributed (hereafter i.i.d.) bivariate random vectors with $E\left[ \boldsymbol{\varepsilon}_1 \right] = \boldsymbol{0} $, $E\left[ \boldsymbol{\varepsilon}_1 \boldsymbol{\varepsilon}_1^{\top} \right] >\boldsymbol{0} $ and $w_t$ is the following scalar AR(2) process:

where $\phi_1 \phi_2 \neq 0$ , $\left\{ \delta_t \right\}$ a sequence of i.i.d. random variables independent of $\left\{ \boldsymbol{\varepsilon}_t \right\}$ such that $$E\left[ \delta_1 \right] = 0\quad\text{and}\quad E\left[ \delta_1^2 \right] = 1 - \phi_2^2 - \left\{ \phi_1^2 \frac{1+\phi_2}{1-\phi_2} \right\}.$$ Hence, we obtain $E\left[ w_t^2 \right] \equiv \gamma_w(0) = 1$, where $\gamma_w(j) = E\left[ w_t w_{t+j} \right]$ is the $j$-th lag autocovariance of $w$. We consider the correctly specified $2$-step ahead forecasting model:

where $\boldsymbol{\varepsilon}_t^{*(2)} = \boldsymbol{\varepsilon}_{t+2} + \boldsymbol{a} \delta_{t+1}$. It can be easily proved that $E\left[ \boldsymbol{\varepsilon}_t^{*(2)} w_{t-j} \right] = \boldsymbol{0}$ for $j \geq 0$. Now, consider the following misspecified model, Model 1:

The forecasting error results:

We compute the theoretical value of the $\operatorname{VMRIC}$ by using Eq. ((ref)). After some routine algebra, we get:

which highlights how the variance-covariance matrix of the $2$-step ahead forecast vector is equal to the {DGP}'s variance-covariance plus a bias term that depends upon the misspecification considered. Now we focus on the variability index $\operatorname{\textbf{VI}}$. We get

and

where

Following Eq. ((ref)), the results from Eq. ((ref)), ((ref)), and ((ref)), deliver the $\operatorname{VMRIC}$ for this case. Now we consider a second misspecified model, Model 2:

where $z_t$ is a weakly stationary linear AR(1) process independent of $w_t$:

with $\psi_1 \in (-1, 1)$, and $\left\{ \upsilon_t \right\}$ is a sequence of i.i.d. random variables independent of both the error terms $\left\{ \delta_t\right\}$ and $\left\{\boldsymbol{\varepsilon}_{t}\right\}$ such that $E[\upsilon_t] = 0$ and $E[\upsilon_t^2] = 1 - \psi_1^2$, delivering $E[z_t] = 0$ and $E[z_t^2]=1$. Thus, $z_t$ is uncorrelated with both $w_t$ and $\mathbf{y}_t$, therefore $\boldsymbol{\rho}=\boldsymbol{0}$. The forecasting error in this case results $\boldsymbol{\eta}_t^{(2)} = \boldsymbol{a} w_{t+1} + \boldsymbol{\varepsilon}_{t+2}$. Following similar arguments as above we obtain ${\operatorname{\textbf{MI}}}$ and ${\operatorname{\textbf{VI}}}$ for Model 2:

As mentioned above, Model 1 is misspecified since it omits the lagged predictor $w_{t-1}$, while Model 2 only includes the non-informative predictor $z_t$.

Finite sample performance

First, we compare the above theoretical derivations with their sample counterpart. We consider three different parameterizations, presented in Table (ref). Also, $\alpha_n = n^\alpha$ with $\alpha=0.85$. Note that, in order for Eq. ((ref)) to hold, $\alpha$ must range in $(0, 1)$. Further experiments showed that results are fairly robust if reasonable values of $\alpha$ are selected. For an empirical method to determine it, see hsu2019_supp. We take the following variance/covariance matrix for the innovations: $$E[\boldsymbol{\varepsilon}_t\boldsymbol{\varepsilon}_t^{\top}] = \left[

\right].$$ \par We compute both the {$\operatorname{VMRIC}$} for Model 1 and Model 2, and estimate the $\hat{\operatorname{VMRIC}}$ and $\hat{\operatorname{VMRIC}}$ on a large sample of $n=10^6$ observations. The results are shown in Table~\ref{tab:2} for the two models, where the theoretical $\operatorname{VMRIC}$ (rows 1 and 3) is compared with the estimated one (rows 2 and 4). The results seem to confirm the consistency of the estimator shown in Eq.~(\ref{eqn:VMRIC_hat}). Clearly, the $\operatorname{VMRIC}$ of Model 1 is consistently smaller than that of Model 2 and indicates its superior predictive capability.

The finite sample behaviour of the method of moments estimator of the $\operatorname{VMRIC}$ can be further appreciated in Table (ref) where we show their bias and Mean Squared Error (MSE). The results are based upon 1000 Monte Carlo replications and seem to indicate a rate of convergence of the order of $n^{-1}$. In Table (ref), we show the percentages of correct model selection by the $\operatorname{VMRIC}$, compared with the multivariate version of the $\operatorname*{AIC}$ and $\operatorname*{BIC}$ for the three parameterizations of Table (ref). For a sample size of $n=100$, both the $\operatorname*{AIC}$ and $\operatorname*{BIC}$ select the best predictive model in about 50% of the cases and relying upon them is tantamount to tossing a fair coin. In such a case, the $\operatorname{VMRIC}$ selects the correct model in about $80\%$ of the cases and reaches $100\%$ for $n=1000$. On the contrary, for Case 3, both the $\operatorname*{AIC}$ and $\operatorname*{BIC}$ cannot go above $64\%$ for a sample size as large as $n=10000$ observations and this is a general indication of their lack of asymptotic efficiency.

Proofs

In this section we detail the proofs of the three theorems. Hereafter all the derivations hold for any fixed $h\geq 1$; for the sake of presentation we write $ \boldsymbol{\varepsilon}_{t}$ instead of $ \boldsymbol{\varepsilon}_{t}^{(h)}$. Remember that $\{l_n\}$ indicates an increasing sequence of positive integers such that:

and define $a = n-l_n-h$ and $b=n-l_n-h+1$.

Proof of Theorem (ref)

The proof of Theorem (ref) relies upon four propositions.

Proof of Theorem (ref)

We prove that:

Since $$\left( \boldsymbol{\hat{\beta}}-\boldsymbol{\beta}\right) = \hat{R}^{-1} \left( N^{-1} \sum_{t=1}^{N} x_t \boldsymbol{y}_{t+h} \right) - \boldsymbol{\beta} = \hat{R}^{-1} \left(N^{-1} \sum_{t=1}^{N} x_t \boldsymbol{\varepsilon}_t \right),$$ routine algebra implies that:

By applying Propositions (ref) -- Propositions (ref), we have:

The proof is completed upon noting that $(1)=(\ref{eq:main1})$ and $(Q)=(\ref{eq:main2})$.

Proof of Theorem (ref)

We start proving that

Note that

hence, it holds that $\hat{\operatorname{\textbf{MI}}}_h - \operatorname{\textbf{MI}}_h$ equals

Assumption (A1) implies that $(\ref{eq:mi_A})=O_p(n^{-1/2})$ whereas, by combining Assumptions (A3) and (A4) with the non-singularity of $R$ and H\"older's inequality, it can be shown that $(\ref{eq:mi_B})=O_p(n^{-1})$ and hence the proof of ((ref)) is complete. Next, we prove that

It suffices to show that

It holds that $\hat{\mathbf{C}}_{h,s}$ is equal to

We prove that ((ref)) is $o_p(1)$ componentwise. To this end consider: $$\operatorname{E}\left[\left( N-s\right)^{-1}\abs*{\sum_{t=1}^{N-s} x_t^2 x_{t+s} \varepsilon_{t+s,i}}\right],$$ with $\varepsilon_{t+s,i}$ being the $i$-th component of the vector $\boldsymbol{\varepsilon}_{t+s}$. The triangular inequality and H\"older's inequality imply that:

Since $ \hat{\boldsymbol{\beta}}_n(h) - \boldsymbol{\beta}_h = \hat{R}^{-1} \left( N^{-1} \sum_{j=1}^{N} x_j \boldsymbol{\varepsilon}_{j,h} \right)$, by combining Assumptions (A3), (A4) and (A5) with Chebyshev's inequality we obtain that ((ref)) is $o_p(1)$. Similarly, we can verify that ((ref)) and ((ref)) are $o_p(1)$. Lastly, Condition (A2) implies that $(\ref{Cs_A}) = \mathbf{C}_{h,s}+ o_p (1)$, hence ((ref)) is verified and the whole proof is complete.

Proof of Theorem (ref)

By Theorem (ref) the $\operatorname{VMRIC}_h$ defined in ((ref)) can be written as:

Therefore,

and hence

Now, consider two models $\ell_1$ and $\ell_2$ in the candidates set $J_{\ell_1}, J_{\ell_2} \in M_1$ such that $\operatorname{VI}_h(\ell_1)\neq \operatorname{VI}_h(\ell_2)$. We show that

By defining $\operatorname{\textbf{MI}}^*_h$ to be the minimum value of $\operatorname{\textbf{MI}}_h$ over the family of candidate models, we have:

Therefore, for sufficiently large $n$, it holds that:

Thus

and ((ref)) is verified and implies that

This completes the proof.

Acknowledgments

Greta Goracci acknowledges the support of Libera Università di Bolzano, Grant WW201L (ESAMD).