Unlocking the Regression Space

Unlocking the Regression Space

\thispagestyle{empty}

Introduction

Regression analysis is the cornerstone of statistical theory and practice. Ordinary least squares (OLS) has been applied, within various regression contexts, to build an extensive toolkit, for the exploration of economic and financial datasets. The basic theory underlying OLS estimation and inference in regression models has been largely established for over half of a century (see e.g. lai1982). The problem of robust estimation has long been a focus of empirical work in economics, beginning with the seminal work by white1980. Its importance is well understood in applied econometrics. At the same time, several important concerns have been raised by applied researchers. angrist2010 noted that “leamer1983 diagnosed his contemporaries’ empirical work as suffering from a distressing lack of robustness to changes in key assumptions”, and leamer2010 later reflected that “sooner or later, someone articulates the concerns that gnaw away in each of us and asks if the Assumptions are valid.” Similarly, karmakar2022 observed, that the assumption of parameter constancy, or “stationarity is often an oversimplified assumption that ignores systematic deviations of parameters from constancy". Clearly, this concern extends beyond parameter stability to encompass the stability of regressors, regression noise, and the underlying modelling assumptions.

In this paper, we focus on the inherent capacity of regression modelling to accommodate the effects of structural change in settings with both fixed and time-varying parameters. Many such structural changes influence not only the model parameters but also the regression space itself. This space comprises both the regressors and regression noise, and improper treatment of these components may result in incorrect inferences, misinterpretations, and forecasting distortions. We therefore examine which specifications of regression space can flexibly account for structural change while still enabling estimation of both fixed and time-varying regression parameters, the construction of confidence intervals, and the computation of standard errors.

Among recent developments, wu2007, boldea2012, and others, have proposed advanced theoretical methods for the estimation of the fixed parameters, while jansson2018, jochmans2019 developed procedures to estimate both fixed parameters and standard errors in regression models with an increasing number of covariates and heteroscedasticity. Meanwhile, degui2020, sun_hong2021 and linton2019 introduced new modelling frameworks that explicitly account for structural change. A common response to concerns about heteroskedasticity in the recent literature is the use of heteroscedasticity-robust variance and standard error estimators for linear regression models, see eicker1963, white1980, MacKinnon(2012) and jansson2018, among others.

There is also a sizeable and growing literature on the estimation of time-varying coefficient regression models, including works of fan1999, vogt2012, among others. This literature further explores tests for different types of parameter variation, see e.g. bai1998, zhangwu2012, zhangwu2015, HKW2024. In addition, specification tests and tests for parameter instability have received significant attention, with important contributions by hansen2000, georgiev2018, hidalgo2019, boldea2019, hong2023, and others.

The modelling of deterministic, smooth parameter evolution has a long history in statistics. Early examples include linear processes with time-varying spectral densities, introduced by pri1965. This framework is essentially nonparameteric and it has been further developed by rob1989, rob1991, dah1997, dahlhaus2019, dahlhaus2023, some of whom refer to these processes as {\it locally stationary}. The estimation of time-varying parameters, as well as fixed parameters under heteroskedasticity in time series models, has been studied in dahlhausG1998, xu_phillips2008, dgr2020, among other. Nonlinear time-varying time series models have also been developed by doukhan2008, bardet2009, vogt2012 and karmakar2022. Despite these advances, such approaches have not been not been widely adopted in applied economics, where random coefficient models remain more prevalent.

Various methods have been proposed over the years to identify and handle structural change. Early contributions assumed that changes were deterministic, rare, and abrupt. Testing for parameter breaks dates back to the pioneering work by cho1960, with further contributions by bro1974, plo1992, among others. More recent approaches allow for random evolution of parameters, where changes may be discrete, as in Markov Switching models by ham1989 or threshold models by ton1990, or continuous as in smooth transition models by ter1998, or those driven by unobservable shocks, as in random coefficient models by nyb1989. For example, cog2005 use random coefficient models to study stochastic volatility, while pri2005 examines whether changes in parameters or in the variance of shocks - policy-induced or otherwise - contributed to the period of macroeconomic calmness known as the “Great Moderation" after 1985. In these frameworks, parameters typically evolve as random walks or autoregressive processes.

Building on this literature, gky2014, gky2018, dgk2022, and others have developed a theoretical time series framework for random coefficient models and their estimation using kernel-based methods, which performs well in finite samples. These methods are computationally simple and straightforward to implement in applied research. For example, cgk2022 demonstrated the empirical prevalence of persistent volatility, suggesting that GARCH-type volatility structures may be less common than previously thought. Nevertheless, a full treatment of estimation and inference within a general regression framework has, surprisingly, not yet been provided.

In this paper, we provide a rigorous validation of the asymptotic normality of the feasible $t$-statistic for the estimation of both fixed and time-varying parameters in linear regression models within an extended regresion space of regressors and regression disturbances. Our main objective is to describe, in transparent terms, the extended regression space under which such normality is preserved. The class of admissible regressors and regression noises is broad. Regressors are obtained by rescaling and shifting stationary short-memory sequences, while regression errors are generated by arbitrary rescaling of a stationary martingale difference sequence. The restrictions imposed on the scale factors and mean processes are weak, allowing these to be either deterministic or stochastic, and to vary over time, possibly abruptly or through non-stationary (e.g. unit-root) dynamics. Some assumptions on the scale factors are necessary and resemble the Lindeberg condition in the classical Lindeberg–Feller central limit theorem. Importantly, the robust feasible $t$-statistic retains the same form and limiting distribution as in the standard setting. The infeasible robust standard errors coincide with the heteroskedasticity-consistent standard error estimator of white1980. Our assumptions do not rely on mixing or near-epoch dependence conditions, which prevail throughout the existing literature. Given the generality of the regression space, these assumptions typically require no additional empirical verification.

The estimation framework for fixed regression parameters is developed in Section (ref), which introduces the extended regression space, the underlying assumptions, and the main theoretical results. Section (ref) establishes the estimation theory for time-varying regression parameters within the same framework. The proofs highlight how the results for the fixed-parameter case naturally extend to time-varying settings, with only negligible additional terms.

Our results are complementary to existing frameworks. The novelty lies in providing a methodological foundation that confirms the validity of robust regression estimation in an extended regression space. The fundamental theory in this area traces back to lai1982, who studied regression models with heteroskedastic martingale difference noise under eigenvalue-based assumptions. Alternative methods, such as bootstrap procedures, see boldea2012; boldea2019, are widely used in regression analysis but may not be directly applicable to such a general class of regressors and regression noises. In contrast, we demonstrate that White-type standard errors remain applicable and computationally straightforward.

All theoretical results are supported by detailed, rigorous proofs. Monte Carlo simulations confirm that the proposed robust regression estimators perform well in finite samples. Overall, the framework developed in this paper is particularly suited to modelling economic and financial data, where heterogeneity, structural change, and dependence are inherent features.

The remainder of this paper is organised as follows. Section (ref) presents the regression setting with the extended regression space, accommodating heterogeneity and dependence, and outlines the theoretical results for infeasible and feasible $t$-statistics in the case of fixed parameters. Section (ref) extends the analysis to the time-varying regression parameters. Section (ref) addresses regression modelling with missing data patterns. Section (ref) illustrates the flexibility of our robust estimation method by its application to the estimation of an AR(p) model generated by a stationary martingale difference noise. Sections (ref) presents Monte Carlo simulation results. In Section (ref), we provide an empirical application of the robust regression framework to modelling asset returns. Finally, Section (ref) concludes. Proofs and additional simulations are provided in the Supplemental Material.

OLS estimation in general regression space

In this section, we focus on ordinary least squares (OLS) estimation in an environment that permits general heterogeneity in regression modelling. We analyze the model

where $\beta$ is a $p$-dimensional parameter vector, $z_t=(z_{1t}, ...., z_{pt})^\prime$ is a stochastic regressor and $u_t$ is an uncorrelated noise term. To include an intercept, the first component can be set as $z_{1t}=1$. We refer to the collection of $\{z_t, u_t\}$ jointly as “the regression space".

An applied researcher may want to work within a regression space that accommodates a wide range of regressors and regression noises, without being hindered by restrictive technical assumptions. Ideally, such a setting should permit regressors exhibiting non-stationarity and undefined generic structural change, while enabling estimation and inference under weak theoretical constraints that do not require empirical verification.

Our goal is to extend the OLS estimation procedure to a broad regression framework defined by baseline assumptions aligned with empirical research practice. These assumptions cover a wide variety of types of potentially non-stationary regression variables encountered in applied work. The framework achieves a level of generality comparable to that in glp2024, which addresses testing for absence of correlation and cross-correlation under general heterogeneity.

We begin with specifying the structure of an uncorrelated regression noise $u_t$. Suppose that

where $\{\varepsilon_t\}$ is a zero mean stationary uncorrelated martingale difference noise, and $\{h_t\}$ is a deterministic or stochastic scale factor independent of $\{\varepsilon_t\}$. The following assumption formalizes these conditions.

The information set $\mathcal{F}_t$ is generated by the past history $\mathcal{F}_t=\sigma(\varepsilon_s, \, s\le t)$ and possibly other variables.

A typical example of an m.d. noise in applied work is provided by the ARCH/GARCH family and the class of stochastic volatility processes. The specification ((ref)) therefore allows for conditional heteroskedasticity in $u_t$.

We next specify the regressors $z_t=(z_{1t}, ..., z_{pt})^\prime$ which form the key structural component of our regression space. For $k=1, ..., p$, the regressors can be written as

where $\eta_t=(\eta_{1t}, ..., \eta_{pt})^\prime$ is a stationary sequence, $g_t=(g_{1t},...., g_{pt})^\prime$ are deterministic or stochastic scale factors, and $\mu_t=(\mu_{1t}, ..., \mu_{pt})^\prime$ is a vector of deterministic or stochastic means. We assume that $\{\mu_t, g_t,h_t\}$ are independent of $\{\varepsilon_t, \,\eta_t\}$. To include an intercept in model ((ref)), we set

We further suppose that in ((ref)) $E\eta_{kt}=0$ except for the intercept ((ref)), where $\eta_{1t}=1$.

In summary, the admissible regressors $\{z_t\}$ in our setting are obtained by shifting and rescaling a short-memory stationary process $\{\eta_t\}$ by the mean process $\mu_t$ and the scale factor $g_t$: $$ z_t=\mu_t+I_{gt}\eta_t, \quad I_{gt}= {\rm diag}(g_{1t}, ..., g_{pt})^\prime. $$

The underlying stationary sequence $\{\eta_t\}$ is the fundamental component structuring regressors $z_t$. Estimation of the regression parameter $\beta$ requires only mild assumptions on $\{\mu_t,g_t\}$, and short-memory dependence assumption on $\eta_t$, satisfied by ARMA and related stationary time series models. This framework eliminates the need for additional empirical validation.

The novelty of this regression framework lies in the structural specification ((ref)), which accommodates regressors $z_t=(z_{1t}, ...., z_{pt})^\prime$ that may be deterministic or stochastic, and stationary or non-stationary. This flexibility arises from allowing a broad class of scale factors and mean processes $\{h_t, g_t, \mu_t\}$ which brings the OLS estimation closer to empirical practice.

The estimation framework also accommodates triangular arrays of means and scale factors: $\big(\mu_t, g_{t}, h_{t},\,\,\,$ $t=1,..., n\big)=\big(\mu_{nt}, g_{nt}, h_{nt}, \,\,\,t=1,..., n\big)$. Throughout the paper, we assume that these quantities may depend on the sample size $n$. For brevity of notation, the subscript $n$ is omitted.

The underlying stationary noise component $\eta_t$ in the regressors $z_t$ in ((ref)) is weakly exogenous with respect to the stationary m.d. noise $\varepsilon_t$ in $u_t = h_t \varepsilon_t$. The mean and scale factors $\{\mu_t,g_t\}$ are independent of $\{\varepsilon_t\}$, though they may be dependent on $\{h_t\}$. Overall, $\{\mu_t, g_t, h_t\}$ are mutually independent of $\{\eta_t, \varepsilon_t\}$, while potential dependence among $\{\mu_t\}$, $\{g_t\}$ and $\{h_t\}$ is unrestricted.

The processes $\mu_{kt}$ and $g_{kt}$ can be interpreted as conditional mean and variance, $\mu_{kt}=E[z_{kt}\, |\mathcal{F}_n^*]$, and $g_{kt}^2=\mathrm{var}(z_{kt}|\mathcal{F}_n^*) $ of $z_{kt}$, where $\mathcal{F}_n^*=\sigma\big(\mu_t, g_t,h_t, t=1, ..., n\big)$ denotes the information set generated by the means and scale factors.

Denote for $k=1, ..., p$,

We write $a_n\asymp_p b_n$ if $a_n=O_p(b_n)$ and $b_n=O_p(a_n)$.

Assumptions ((ref))–((ref)) impose only mild restrictions on the means $\mu_t$ and scale factors $g_t$. In particular, condition ((ref)) resembles the Lindeberg condition in the classical Lindeberg–Feller central limit theorem, as it excludes the possibility that the OLS estimation is dominated by a single extreme observation of $z_t$.

The first restriction on $g_{kt}$ in ((ref)) is necessary. For example, consider the regressor $z_t=g_{t}\eta_{t}, t=1, ..., n$, with scale factors $g_{1}=1$ and $g_{2}=g_{3}=...=g_{n}=0$, so that $z_{2}=z_{3}=...=z_{n}=0$. In this case, the OLS estimator of $\beta$ is inconsistent, and such a scale factor $g_t$ does not satisfy ((ref)).

The second condition ((ref)) ensures that OLS estimation is driven by the stochastic component $g_t\eta_t$ of the regressor $z_t$, rather than by deterministic or stochastic drift in $\mu_t$.

In the presence of an intercept, condition ((ref)) further implies that $\sum_{t=1}^n h_t^2\asymp_pn$, since $v_1^2\asymp_pv^2_{g1}$, $g_{1t}=1$, $v_{g1}^2=n$, and $v_{1}^2=\sum_{t=1}^n h_t^2$.

To estimate $\beta=(\beta_1, ..., \beta_p)^\prime$, we use the standard OLS estimator

computed from the sample $y_j, z_j, \,j=1, ..., n$.

\vskip.2cm \noindentConsistency. We first establish the consistency of the OLS estimator $\widehat \beta$.

\vskip.2cm This result implies that the $k$-th component $\widehat \beta_k$ of the OLS estimator is $v_k$-consistent, that is, $\widehat \beta_k -\beta_k = O_p(v_k^{-1})$. The convergence rate $v_k$ may deviate from the conventional $\sqrt{n}$ rate and may differ across components. From the definition of $v_k$ and $v_{gk}$, it follows that

\noindentAsymptotic normality. The asymptotic normality of an element $\widehat \beta_k$ of the OLS estimator, as well as the computation of its standard errors, requires additional assumptions on the scale factors and the stationary processes $\{\eta_t, \varepsilon_t\}$.

Assumption (ref) is not required when $\varepsilon_t$ is i.i.d. Together, Assumptions (ref) and (ref)(ii) exclude cases in which the mean process $\mu_t$ or a few extreme observations of $z_t$ or $u_t$, dominate the estimation of the regression parameter. Overall, these assumptions are mild. They accommodate both deterministic and stochastic means $\mu_t$ and scale factors $h_t, g_{t}$ that may change abruptly over time unlike other theoretically rigorous treatments which restrict structural change to be deterministic and smooth. This flexibility makes the framework particularly suitable for modelling financial data, as it allows for volatility jumps, commonly observed in empirical finance (see, e.g., eraker2003). In modern macroeconomic VAR models, the scale factor $h_t$ in the uncorrelated noise representation $u_t=h_t\varepsilon_t$ is typically assumed to be stochastic (see, e.g., chan2024, carriero2024), which our framework naturally encompasses.

Lemma (ref) below shows that Assumptions (ref) and (ref)(ii) holds for regressors $z_t$ and noises $u_t$ with bounded $4+\delta$ moments satisfying ((ref)). The following example provides additional sufficient conditions.

We now describe the infeasible standard errors $\sqrt {\omega_{kk}}$ using the notation:

where $\omega_{jk}$ denotes the $(j,k)$-th element of the matrix $\Omega_n$. The infeasible standard error of $\widehat \beta_k$ is defined as $\sqrt {\omega_{kk}}$, i.e., the square root of the corresponding diagonal element of $\Omega_n$.

The generality of our regression setting limits the multivariate asymptotic theory that can be established for $\widehat\beta_t$. While a full joint distribution of $\widehat\beta_t$ is not available, we can derive asymptotic normality for linear combinations $a^\prime \widehat \beta$ and then construct feasible inference for individual component $\beta_k$.

Existing literature typically imposes stronger assumptions on regressors and errors such as mixing regressors white2014, locally stationary regressors in zhangwu2012, or near-epoch dependent errors in boldea2012.

\vskip.2cm

Property ((ref)) is difficult to implement in practice because it requires estimation of the unknown matrices $D,\, \Omega_n$, except in the special case $a^\prime=(0, ...,1,...0)$ with only the $k$-th element nonzero. In this case, ((ref)) reduces to ((ref)), and the infeasible standard error $\sqrt{\omega_{kk}}$ can be consistently estimated by

The feasible standard error $\sqrt{\widehat \omega_{kk}}$ is the square root of the diagonal element $\widehat \omega_{kk}$ of $\widehat \Omega_n$.

\vskip.2cm This result is the main contribution of Section (ref). It enables straightforward computation of standard errors and the construction of confidence intervals for $\beta_k$ in the extended regression framework. Notably, the estimator $\widehat \Omega_n$ coincides with the heteroskedasticity-consistent standard error estimator of white1980.

\vskip.2cm

Corollary (ref) allows us to establish the asymptotic power and consistency of a test for testing the hypothesis \[ H_0: \beta_k = \beta_k^0, \quad {\rm vs.} \quad H_1: \beta_k \ne \beta_k^0, \] i.e., whether the $k$-th element of the regression parameter $\beta=(\beta_1,\ldots,\beta_p)^\prime$ is equal to a specific value $\beta_k^0$.

We conclude this section with a lemma that provides simple sufficient moment-type conditions for the validity of Assumptions (ref) and (ref)(ii). In particular, condition ((ref)) implies ((ref)).

The regular estimator of standard errors in OLS regression estimation is given by

Unlike the robust standard errors $\sqrt{\widehat\omega_{kk}}$, these conventional standard errors may produce coverage distortions, particularly when heteroskedasticity or heterogeneity in $g_t$, $h_t$, or $\mu_t$ is present, see Section (ref). This underscores the robustness and strong empirical performance of the normal approximation in ((ref)).

In this section, we have provided a rigorous validation of the asymptotic normality of feasible $t$-statistics for the components of the OLS estimator in linear regression models with general heterogeneity. The assumptions imposed are mild yet flexible, allowing a wide class of (possibly nonstationary) regressors and noise processes beyond those typically considered in the existing literature. Some conditions on scale factors are analogous to the Lindeberg condition and remain necessary. Our framework complements, rather than replaces, prior approaches; for instance, near-unit-root regressors georgiev2018 require a distinct theoretical treatment. Although bootstrap methods, see, e.g., boldea2012, boldea2019, are widely applied in regression analysis, they may not extend to the heterogeneous structures considered here. By contrast, we demonstrate that the heteroskedasticity-consistent standard errors of white1980 remain applicable and computationally straightforward.

\vskip.2cm In this paper we focus on the regression model ((ref)), where the regression noise $u_t$ in ((ref)) is uncorrelated. Extending the asymptotic theory to account for dependence in $u_t$ is a natural next step and is currently under consideration.

\vskip.2cm Detailed proofs of all results are provided in the Online Supplement.

Time-varying OLS estimation in extended regression space

This section demonstrates further advantages of the theory of regression estimation with a fixed parameter, developed in Section (ref). Thanks to the flexible setting, estimation of time-varying parameters naturally follows from our theory for fixed-parameter regression in the extended space, along with bounding of some negligible terms.

In the previous section, we discussed the estimation of the regression model ((ref)), $y_j=\beta^\prime z_j +u_j$, with a fixed parameter $\beta$. We now extend the model by allowing the regression parameter to vary over time. Specifically, we consider the model

where the regressors $z_j$ and the regression noise $u_j$, as defined in ((ref)) and ((ref)), remain unchanged. That is, they belong to the same regression space as in Section (ref).

The primary objective is to develop a point-wise estimation procedure for the path $\beta_1, ..., \beta_n$ of the time-varying parameter $\beta_j$ in model ((ref)), while preserving the same regression space introduced in Section (ref).

The literature on estimation of time-varying regression parameters $\beta_j$ is extensive. It primarily focuses on estimation and testing for parameter stability under relatively strong assumptions on the regressors and regression noise. For instance, regressors are assumed to be locally stationary in (vogt2012, model (3)), stationary and strongly mixing in (hong2023, Assumption A.1) and strictly stationary in (HKW2024, Assumption P$(d)$). It is clear that the class of regressors considered in our setting is broader, and they may be neither mixing nor stationary.

The objective of this section is to describe the extended regression space of regressors $z_t$ and disturbances $u_t$ that ensures the asymptotic normality of the feasible $t$-statistic estimating the components of the time-varying parameter $\beta_t$. We show that, as long as the regressors and the disturbance follow the structure $z_t=\mu_t+I_{gt}\eta_t$ and $u_t=h_t\varepsilon_t$, the class of admissible means $\mu_t$ and scale factors $g_t, h_t$ is very broad and characterized by weak restrictions that may not require empirical verification.

Further extensions of the regression space are possible. For example, the weakly exogenous component $\eta_t$ of the regressors $z_t$ in our paper is assumed to be a short-memory process. In contrast, HKW2024 demonstrate that estimation of the time-varying parameter $\beta_t$ also permits weakly exogenous, strictly stationary regressors $z_t$ that exhibit long-memory behavior.

While most assumptions on the regressors $z_j$ and regression noise $u_j$ remain unchanged from Section (ref), the estimator requires some modifications. Under an additional smoothness assumption on $\{\beta_j\}$, the time-varying OLS estimator $\widehat \beta_t$ of parameter $\beta_t$ at time $t$ is the standard OLS estimator for a fixed regression parameter, obtained by regressing $\widetilde y_j= b_{n,tj}^{1/2}y_j$ on $\widetilde z_j=b_{n,tj}^{1/2}z_j$:

The weights $b_{n, tj}$ are generated as follows:

where $H=H_n$ is a bandwidth parameter such that $H\rightarrow \infty$ and $H=o(n)$. The kernel function $K$ is bounded and there exist $a_0, \, \delta>0$ and $\theta>3$ such that

For example, ((ref)) is satisfied by functions $K(x)=I(x\in [0,1])$ and $K(x)=p(x)$ where $p(x)$ is the probability density function of the standard normal distribution.

We impose a smoothness assumption on the time-varying parameter $\beta_j$, which may be either deterministic or stochastic.

Next, we briefly outline how our asymptotic theory for the time-varying robust estimator builds on the results from Section (ref) on fixed-parameter regression estimation and the smoothness assumption ((ref)). To demonstrate this, we introduce the following regression model with a fixed parameter $\beta=\beta_t$:

Notice that the OLS estimator $\widehat \beta$ of the fixed parameter $\beta$ satisfies:

Since $\widetilde y_j =y_j^*+(\beta_j-\beta_t)^\prime \widetilde z_j$, the time-varying estimator $\widehat \beta_t $ given in ((ref)) satisfies:

Notice that $\widehat \beta -\beta$ in ((ref)) does not depend on $\beta$. Additionally, the regression space in estimation of the fixed parameter in Section (ref) permits rescaling, so premultiplying by the kernel weights $b_{n,tj}^{1/2}$ does not change the structure of regressors $\widetilde z_{j} =( \widetilde z_{1j}, ...,\widetilde z_{pj})^\prime$ and $\widetilde u_{j} $: they still satisfy the settings ((ref)) and ((ref)). Consequently, the model ((ref)) is covered by the regression model ((ref)) with a fixed parameter, and the asymptotic results for $\widehat \beta -\beta$ follow from Section (ref). The main technical task in this section is to show that the remainder term $R_t$ in ((ref)) is negligible, which follows from the smoothness assumption ((ref)).

\vskip.2cm The regressors $z_j$ and regression noise $u_j$ belong to the same regression space as defined in as in Section (ref). While the assumptions on the stationary process $\{\eta_j\}$ and the martingale difference noise $\{\varepsilon_j\}$ remain unchanged, for simplicity, we replace the previous conditions on the scale factors $g_j, h_j$ and the means $\mu_j$ with simple sufficient assumptions similar to those used in Lemma (ref). As before, the scale factors $\{h_j, g_j, \mu_j\}$ can be deterministic or stochastic, may vary with $n$, and are independent of $\{\eta_j, \varepsilon_j\}$.

Denote

It is straightforward to show that ((ref)) is valid if $g_{kt},h_t\ge c>0$ for all $t,n$.

To describe the infeasible standard errors $\sqrt{\omega_{kk,t}}$, we use:

where $\omega_{jk,t}$ denotes the $(j,k)$-th element of the matrix $\Omega_{nt}$. The infeasible standard error $\sqrt {\omega_{kk,t}}$ is defined by the diagonal element $ \omega_{kk,t}$ of the matrix $ \Omega_{nt}$.

The next theorem establishes the consistency rate and asymptotic normality property for the components of the time-varying OLS estimator $ \widehat \beta_t=(\widehat\beta_{1t}, ..., \widehat\beta_{pt})^\prime$, and allows for arrays of integers $t=t_n\in [1, ..., n]$, which may depend on $n$.

The consistency rate in ((ref)) is determined by the bandwidth parameter $H$ and the smoothness parameter $\gamma\in (0,1)$ in ((ref)). The condition $H=o(n^{2\gamma/(2\gamma+1)})$ ensures that in ((ref)) the bias term remains negligible.

As in the fixed-parameter case, for $(z_j, u_j)$ from the extended regression space, the asymptotic normality can be established in point-wise estimation for each individual component $\widehat \beta_{kt}$ of $\widehat \beta_{t}$.

The unknown standard error $\sqrt{\omega_{kk,t}}$ can be consistently estimated by:

The feasible standard error $\sqrt {\widehat \omega_{kk,t}}$ is defined by the diagonal element $\widehat \omega_{kk,t}$ of $\widehat \Omega_{nt}$.

Corollary (ref) allows us to establish the asymptotic power of the test of the hypothesis $$H_0: \beta_{kt}=\beta_{kt}^0,\quad {\rm vs.} \quad H_1: \beta_{kt} \ne \beta_{kt}^0, $$ based on the $t$-statistics $(\widehat \beta_{kt} -\beta_{kt}^0)/\sqrt{\widehat \omega_{kk,t}}$.

The estimator $\widehat \Omega_{nt}$ used to obtain robust standard errors in ((ref)) is a time-varying version of heteroskedasticity-consistent estimator of standard errors by white1980. Simulation results confirm that it does not produce coverage distortions in the estimation of $\beta_t$ under the settings considered in this section.

In conclusion, we provide examples of smoothly varying deterministic and stochastic parameters $\beta_t$ that satisfy Assumption (ref).

The above results are equipped with thorough and mathematically rigorous proofs, which can be found in the Online Supplement.

\vskip.2cm

The key new features in the estimation of time-varying parameter $\beta_t$ are similar to those highlighted in the estimation of the fixed parameter in Section (ref). Although the computation is straightforward, establishing the validity of the robust standard errors $\sqrt{\widehat \omega_{kk, t}}$ in the extended regression space of $(z_t, u_t)$ is challenging because the scale factors $h_t,g_t, \mu_t$ in model ((ref)) are unknown and potentially random, and highly general, while the asymptotic behaviour of the $\omega_{kk,t}$ may not be well-defined. The asymptotic normality of a single component of the estimator can still be established, even though a full multivariate asymptotic theory is not available. Unlike most existing literature, $\beta_t$ is permitted to evolve as a smoothly varying stochastic process.

Regression with missing data

In the previous sections, we showed that the extended regression space enables the estimation of both fixed and time-varying regression parameters. It offers several theoretical advantage, in particular, the ability to estimate regression models in the presence of missing data. Given the importance in empirical regression analysis in situations where some observations $y_t$ or regressors $z_t$ are missing, see, e.g., enders2022, we now present new and somewhat unexpected results on regression estimation with missing data. We show that the foundational assumptions underlying the constriction of regression space also allow us to accommodate an a broad range of missing data patterns.

In this section we suppose that instead of the full sample $(y_1, z_1), ..., (y_n,z_n)$, we observe a subsample

of dependent variable $y_t$ and regressor $z_t$. Our primary interest is to estimate both fixed and time-varying regression parameters using the subsample ((ref)).

To that end, we represent the observed data as partially observed sample

where $\tau_j$ is missing-data indicator. In ((ref)) it is defined as

We set $\tau_j=1$ if both $y_{j}$ and $z_{j}$ are observed, otherwise $\tau_j=0$. Throughout this section, $\tau_j$ is treated as a sequence of random or deterministic variables, allowing for regularly missing, block-wise missing, or randomly missing data patterns.

In order for the theoretical results of the previous section to apply, we impose the following assumptions on the missing data indicator $\tau_t$, the regressors $z_{kt}=\mu_{kt}+g_{kt}\eta_{kt}$ in ((ref)) and the regression noise $u_t=h_t\varepsilon_t$ in ((ref)).

\vskip.2cm {\bf Estimation of a fixed parameter}. Suppose that $y_t= \beta^\prime z_t+u_t$ follows the regression model ((ref)) with a fixed parameter $\beta$ as in Section (ref). Our primary interest is to estimate the parameter $\beta$ using subsample ((ref)). In view of ((ref)), we can write the partially observed regression model as

In ((ref)), the regressors $\widetilde z_t$ and the noise $\widetilde u_t$ can be represented as

They belong to the regression space described in ((ref)) and ((ref)). Therefore, parameter $\beta$ and the correspondent standard errors in model ((ref)) can be estimated using the OLS estimator $\widehat \beta$ and $\widehat \omega_{kk}$:

\vskip.4cm {\bf Estimation of a time-varying parameter}. Assume now that $y_t= \beta^\prime _tz_t+u_t $ follows the regression model ((ref)) with time-varying parameter $\beta_t$, where regressors $z_t$ and regression noise $u_t$ are as in ((ref)) and ((ref)). We are interested in estimating the parameter $\beta_t$ in the presence of missing data using the subsample ((ref)). Similarly to ((ref)), we base the estimation on the partially observed regression model with a time-varying parameter,

where regressors $\widetilde z_j$ and the noise $\widetilde u_j$ are defined as in ((ref)). They belong to the regression space described by ((ref)), and ((ref)) and thus results of Section (ref) on the estimation of time-varying parameter $\beta_j$ apply.

We show in the following theorem that under Assumptions (ref) and (ref), parameter $\beta_t$ and standard errors can be estimated point-wise at each time $t=1, ..., n$ provided that the missing data pattern satisfies the following condition:

This condition holds, for example, if $\tau_j=1$ for $|j-t|\le \epsilon H$ for some $\epsilon>0$.

The estimator $\widehat \beta_t$ and the estimator of the robust standard errors $\widehat \omega_{kk,t}$ given in ((ref)) and ((ref)) are defined as

Estimation of a stationary AR($p$) model with an m.d. noise

In this section we focus on another practical application of our regression framework developed in Section (ref). We show that it covers the estimation of parameters of a stationary AR($p$) model driven by a stationary martingale difference noise $\varepsilon_t$:

where parameters $\phi_0, ..., \phi_p$ are such that the model ((ref)) has a stationary solution. xu_phillips2008 developed estimation theory for AR$(p)$ model $y_t=\phi_0+\phi_1y_{t-1}+...+\phi_py_{t-p}+u_t$, when $u_t=h_t\varepsilon_t$ where $h_t$ is smoothly varying deterministic sequence and a m.d. sequence $\varepsilon_t$ has property $E[\varepsilon_t^2|{\cal F}_{t-1}]=1$ a.s. \,GTT2018 were among the first to analyze the distortions of standard errors caused by m.d. noise in estimation of ARMA models. This paper shows that the variance of the parameter vector $\phi$ converges to a well-defined limit; however, its complex structure complicates the estimation of the limiting variance and the corresponding standard errors in empirical applications. They restricted the estimation of standard errors to AR(1) and MA(1) models. In the case of AR$(p)$ model, using our method we are able to estimate standard errors for any $p$ without relying on asymptotic approximations which is the main novelty and contribution of this section. Notice that the model ((ref)) can be written as a special case of the regression model ((ref)),

Here, the parameter $\beta=(\beta_1, ..., \beta_{p+1})^\prime=(\phi_0, ...., \phi_p)^\prime$ is fixed, and the regressors $z_t=(z_{1t},z_{2t},...,z_{p+1,t})^ \prime=(1,y_{t-1},y_{t-2}, ..., y_{t-p})^\prime $ are stationary random variables. It is straightforward to verify that the regressors $$z_{kt}= \mu_{kt}+g_{kt}\eta_{kt}, \quad \mu_{kt}=E[y_{t-k}]=Ey_1, \quad g_{kt}=1, \quad \eta_{kt}=y_{t-k}-E[y_{t-k}]$$ for $k=2, ..., p+1$ satisfy the regression assumption ((ref)). In the theorem below, we assume that the standard stationarity conditions on parameters of the AR($p$) model ((ref)) are satisfied, see e.g. Theorem 3.1.1 in bd1991, which ensure the existence of a stationary solution

We assume that $\varepsilon_t$ satisfies Assumption (ref) and $\eta_t=(y_{t-1},y_{t-2}, ..., y_{t-p})^\prime $ satisfy Assumptions (ref) and (ref)(i). These assumptions impose only mild restrictions on the m.d. noise $\varepsilon_t$, and their validity can be verified for typical examples of uncorrelated m.d. noise, such as ARCH-type processes.

The OLS estimator $\widehat \beta $ of $\beta$ in regression model ((ref)) is defined as in ((ref)) and $ \widehat \omega_{kk}$ as in ((ref)).

\vskip.2cm The Monte Carlo results presented in Section (ref) demonstrate that the robust OLS estimation produces correct $95\%$ confidence intervals for $\beta_k$, whereas the standard OLS method exhibits coverage distortions, when the noise $\varepsilon_t$ is not i.i.d. This finding indicates that the robust OLS estimator has a broader range of applicability than merely addressing heteroscedasticity, and that it can also be effectively used in regression settings not covered by the standard OLS estimation and inference theory.

It is worth noting that the papers by doukhan2008, bardet2009 and karmakar2022 provide advanced theoretical results on the modelling and estimation of general nonlinear time-varying time series models; however, they address the linear AR$(p)$ model ((ref)) only in the trivial case of an i.i.d. noise $\varepsilon_t$.

Monte Carlo Simulations

In this section, we explore the finite sample performance of the robust and standard OLS estimation methods in regression settings, outlined in Sections (ref) and (ref). We examine the impact of time-varying deterministic and stochastic parameters, means, scale factors and heteroskedasticity of the regression noise on estimation. Comparison of simulation results for standard and robust estimation methods shows that, despite the generality of our regression setting, estimation based on the robust standard errors produces well-sized coverage intervals for fixed and time-varying regression parameters $\beta$ and $\beta_{t}$, while application of the standard confidence intervals leads to severe distortion of coverage rates.

Estimation of a fixed parameter

We generate arrays of samples of regression model with fixed parameter and an intercept:

We set the sample size to $n=1500$ and conduct $1000$ replications and set the nominal coverage probability at $0.95$. (Estimation results for $n=200, 800$ are available upon request). We also include a more complex example in the online supplement.

This model includes three parameters and three regressors. We set $z_{1t}=1$ and define

where $\xi_{2t}=\varepsilon_{t-1}$ and $\xi_{3t}=\varepsilon_{t-2}$. The stationary martingale difference noise $\varepsilon_t$ in $u_t$ is generated by a GARCH($1,1$) process

Models (ref) and (ref) are regression models with fixed parameters. Examples of plots of the simulated dependent variable, regressor and regression noise are shown in Figure (ref) and (ref) ($z_{2t}$ and $z_{3t}$ have similar patterns). To verify the validity of the asymptotic normal approximation of Corollary (ref) in finite samples, we compute empirical coverage rates (CP) for $95\%$ confidence intervals used in robust OLS estimation, for parameter $\beta$. For comparison, we compute the coverage rates CP$_{st}$ for standard confidence intervals based on the standard errors ((ref)) used in standard OLS estimation. The robust and standard OLS procedures share the same estimator $\widehat \beta$, and whence Bias, root mean square error (RMSE) and standard deviation (SD). Their confidence intervals differ because the variances (and standard errors) in their normal approximations are different.

Table (ref) reports estimation results for Model (ref) which contains determinist scale factors. It shows that coverage rate CP for robust confidence intervals is close to the nominal $95\%$, while the coverage rate CP$_{st}$ of the standard confidence intervals drops below $80\%$. The Bias, RMSE, and SD are small.

Table (ref) shows estimation results for Model (ref) which includes stochastic scale factors. It shows that the coverage rate CP for robust confidence intervals is close to the nominal $95\%$, whereas the standard estimation method produces coverage distortions for parameters $\beta_2$ and $\beta_3$.

To assess power, we vary $\beta_3$ in Model (ref) from $0$ to $0.5$ and record how often the test rejects $H_0: \beta_3=0$. Figure (ref) reports results for ROLS and OLS at sample sizes $n=200,\,800,\, 1500$. When $\beta_3=0$, ROLS achieves a good size close to the nominal $5\%$, while the size based on OLS results starts around $20\%$ and remains heavily oversized even as $n$ increases. For $\beta_3\neq 0$, power rises monotonically with $\beta_3$ for both methods. In Figure (ref), the blue solid lines represent power based on ROLS, and the red solid lines correspond to standard OLS. Considering the OLS estimation has large size distortion, we compute its adjusted power, shown by the red dotted lines. With small sample size $n=200$, OLS appears more powerful for $\beta_3 \le 0.2$, whereas ROLS catches up and achives good power when $\beta_3 \ge 0.3$. For $n=800$ and $1500$, both methods already achieve good power around $\beta_3=0.2$. Overall, ROLS provides reliable size and competitive power across different sample sizes. Similary results are observed for Model (ref).

Estimation of a time-varying parameter

In this section we examine the validity of the normal approximation for the estimator $\widehat \beta_t$, ((ref)), of time-varying parameter $\beta_t$, as established in Corollary (ref) of Section (ref). We replace the fixed regression parameter $\beta$ in the model ((ref)) by a time-varying parameter $ \beta_t=(\beta_{1t},\beta_{2t},\beta_{3t})^\prime$:

where $z_{1t}=1$ and $z_{2t}, z_{3t}$ are defined using $\mu_{2t},\mu_{3t}$ and $\eta_{2t}, \eta_{3t}$ as in ((ref)).

We consider two simulation models. Model (ref) assumes deterministic parameters and scale factors, while Model (ref) combines deterministic and stochastic parameters and scale factors.

where $\{\zeta_{j}\}, \{\nu_{j}\}$ are stationary ARFIMA$(0, d, 0)$ processes with memory parameter $d=0.4$.

We estimate $\beta_t$ using the estimator $\widehat \beta_t$, ((ref)), where the weights $b_{n,tj}= K(|t-j|/H) $ are computed with the Gaussian kernel function $K(x)= (2\pi )^{-1/2}\exp(-x^2/2)$ with bandwidth $H=n^h$, $h=0.4, 0.5, 0.6, 0.7$.

Figure (ref) displays parameter estimation results for a single simulation from Model (ref). It depicts the estimates $\widehat \beta_{k1}, ..., \widehat \beta_{kn}$ (red line) against the true parameters $\beta_{kt}$ (blue line), $k=1,2,3$ obtained with the bandwidth $H=n^{0.5}$, and their point-wise $95\%$ confidence intervals (grey dashed lines), computed using the robust standard errors. The robust time-varying confidence intervals cover the true parameters $\beta_{kt}$, $t=1,...,n$, for most of the time points.

Figure (ref) reports the point-wise empirical coverage rates (blue line) in time-varying robust estimation of parameters $\beta_{kt}, k=1,2,3$ which are close to the nominal $95\%$ for most of the time points. Figure (ref) shows the RMSE's for different choices of the bandwidth $H=n^h$, $h=0.4, 0.5, 0.6, 0.7$. As expected, the RMSE depends on the smoothness of the parameter $\beta_{kt}$ and often is minimized by moderately large values of $H$, for example, $H=n^{0.6}$.

Figure (ref) reports estimation results for a single simulation from Model (ref), and Figure (ref) displays point-wise empirical coverage rates for robust $95\% $ confidence intervals. For deterministic parameters $\beta_{1t}$ and $\beta_{2t}$, estimation quality is good and results are similar to those obtained for Model (ref). For the stochastic parameter $\beta_{3t}$, the robust point-wise confidence intervals cover the path of stochastic parameter $\beta_{3t}$ for most of the time points, see Figure (ref)(\subref{MIXE5estimator5b3}). Figure (ref)(\subref{MIXE5CP5b3}) shows that coverage rates of robust time-varying confidence intervals for $\beta_{3t}$ might be slightly affected by stochastic variation in the parameter and scale factors. Nevertheless, they are still satisfactory and reasonably close to the nominal $95\%$ coverage.

Estimation of regression parameter with missing data

To examine the impact of missing data on the robust and standard OLS estimation based on partially observed data $(y_{j_1},z_{j_1}), (y_{j_2},z_{j_2}), ...., (y_{j_N},z_{j_N}),$ we use two types of missing data patterns over the time period $1, ..., 1500$. \vskip.2cm {\it Type 1}. The block of data $j\in [650, \,850]$ is missing.

{\it Type 2}. $500$ single observations are missing at randomly selected times.

\vskip.2cm Tables (ref) and (ref) report robust and standard estimation results for Model (ref) with fixed parameter. Table (ref) shows that block missing data (Type 1) do not lead to noticeable changes in Bias, RMSE and SD, and the coverage rate for robust confidence intervals remains around $95\%$. At the same time, the coverage rate CP$_{st}$ of the standard confidence intervals is substantially distorted.

Table (ref) shows that randomly missing data do not affect the coverage rate of robust confidence intervals which remains to the nominal $95\%$, while the coverage rate of the standard confidence intervals drops to around $65\%$. This emphasises the flexibility of the robust OLS estimation of the fixed parameter in the presence of block or randomly missing data.

Figures (ref) -- (ref) report estimation results for Model (ref) with time-varying parameter $\beta_t$.

Figure (ref) shows the coverage rates in time-varying robust estimation with block missing data (Type 1, shaded region) for $t=1, ..., 1500$. The coverage is close to the nominal $95\%$, with some distortion for parameters $\beta_{1,t}$ and $\beta_{2,t}$ and a larger distortion for parameter $\beta_{3,t}$ within the shaded region. The distortion peaks at the centre of the block, as expected. Although the width of missing data block, $200$, exceeds the bandwidth $H=n^{0.5}=39$ used in estimating $\beta_t$, the coverage distortion seems to be offset by the smooth down-weighting of the data, and the performance of the robust time-varying OLS estimation exceeds expectations.

Figure (ref) reports the path of the estimator $\widehat\beta_{kt}$ and the point-wise robust confidence intervals, for a single simulation. The robust confidence intervals become wider in the shaded region, which likely explains the satisfactory coverage performance during that period.

Figure (ref) shows that randomly missing data (Type 2) do not distort the robust time-varying OLS estimation. For all three parameters and time periods $t$, the coverage rate is close to the nominal. Overall, robust estimation of time-varying parameter does not appear be affected by randomly missing data.

Estimation of a stationary AR($p$) model

We assess the performance of the robust and standard procedures in the case of a stationary AR($2$) model:

where $\varepsilon_t=e_te_{t-1}$, $e_t\sim i.i.d. \,\mathcal{N}(0,1)$ is a stationary martingale difference noise. The regressors $z_t=(z_{1,t},z_{2,t},z_{3,t})^\prime=(1,y_{t-1},y_{t-2})^\prime $ include an intercept and the two past lags of $y_t$. By Theorem (ref), the parameter $\beta$ can be estimated by using the robust estimation method.

Table (ref) shows that the coverage rate for the robust OLS estimation is close to the nominal $95\%$, while the standard OLS estimation exhibits extensive coverage distortion for $\beta_2$ and $\beta_3$.

Empirical experiment

In this section, we analyze the structure and dynamics of daily S$\&$P 500 log returns, $r_t$, from 02/01/1990 to 31/12/2019, (sample size $n=7558$). We employ robust regression estimation to assess whether the returns $r_t$ can be modelled using a time-varying regression model of the form

where $\{\varepsilon_t\}$ is an i.i.d.$(0,1)$ noise, and the time-varying mean and scale factor $\mu_t,h_t$ are independent of $\{\varepsilon_t\}$. Our objective is to estimate the time-varying mean $\mu_t$, the scale factor $h_t$, and to test for the absence of autocorrelation in the absolute residuals $|u_t|=h_t|\varepsilon_t|$, thereby assessing the fit of the model ((ref)) to the data.

It returns $r_t$ follows the model ((ref)) with i.i.d. noise $\varepsilon_t$, then the absolute residuals $|u_t|$'s are uncorrelated then for $t\ne s$:

Conversely, if the noise $\varepsilon_t$ exhibits ARCH effects (stationary conditional heteroskedasticity), the sequence $|u_t|$ becomes autocorrelated, and the null hypothesis of uncorrelated absolute residuals $|u_t|$ would be rejected.

We estimate the the time varying mean $\mu_{t}$ using the time-varying OLS estimator with bandwidths $H=n^{0.4}, n^{0.5}, ..., n^{0.7}$. Figure (ref)(\subref{ep:CI_u}) shows the estimated path of $\widehat \mu_{t}$ and the associated $95\%$ confidence intervals for bandwidth $H=n^{0.6}$ indicating that $\mu_{t}$ is very likely to change over time.

Assumption ((ref)) implies that

Therefore, $|\widehat u_t|=|r_t-\widehat \mu_t| \sim h_tE|\varepsilon_t|+h_t(|\varepsilon_t|-E|\varepsilon_t|)$ and thus $ y_t=|\widehat u_t|$ follows a time-varying regression model of the form

where $\beta_{1t}=h_tE|\varepsilon_t|$ represents a time-varying intercept, $ g_{t}=h_t $ denotes the scale factor, and $\eta_t=|\varepsilon_t|-E|\varepsilon_t|$ is an i.i.d. noise. Hence $\beta_{1t}$ can be consistently estimated using the time-varying OLS estimator $\widehat\beta_{1t}$. Figure (ref)(\subref{ep:CI_b}) displays the estimated path of $\widehat{\beta}_{1t}$ and the corresponding $95\%$ confidence intervals for $\beta_{1t}=h_tE|\varepsilon_t|$ with bandwidth $H=n^{0.6}$, revealing pronounced time variation in the scale factor $h_t$.

Figure (ref)(\subref{SP500ResCorr_u1_CH}) reports testing results for zero correlation at lags $k=1, ...,20$ in the residual sequence $\widehat{\widetilde{u}}_t=y_t-\widehat \beta_{1t}$. We employ the standard test and robust test procedures developed in glp2024. Given that the sample size is large ($n=7558$) and $\beta_{1t}$ is estimated non-parametrically with bandwidth $H=n^{0.6}$, we restrict the correlation analysis to the subsample $j\in[500,1000]$. Both tests provide no evidence of significant correlation within this subsample, suggesting that the model ((ref)) fits the returns $r_t$ well during this time period.

The same is not likely to be true if $r_t^*=r_t-\widehat \mu_t$ follows a GARCH(1,1) process, as confirmed by the following experiment. We fit a GARCH(1,1) model to the demeaned returns $r_t^*=r_t-\widehat \mu_t$,

We generate a simulated GARCH(1,1) sample $r^*_{g 1}, ...., r^*_{g n}$, apply the regression model ((ref)) to the absolute values $y_t^*=|r^*_{g t}|$, and compute the residuals, $\widehat{u}^*_t=y_t^*-\widehat \beta_{1t}$. Figure (ref)(\subref{SP500ResCorr_u2_CH}) shows that both standard and robust tests detect significant correlation in residuals $\widehat u^*_t$, confirming the presence of conditional heteroskedasticity in the simulated GARCH data.

Conclusion

The robust OLS and time-varying OLS estimation and inference methods developed in this paper offer considerable flexibility for modelling economic and financial data. They allow for general heterogeneity in regression components and for structural change of regression coefficients over time. Moreover, the generalization of the structure of regressors and error terms further expands the range of empirical settings to which robust OLS regression framework can be applied. In particular, the paper develops asymptotic theory for general regression models with stochastic regressors possibly including a time varying mean, and provides data-based robust standard errors that enable the construction of confidence intervals for regression parameters. The Monte Carlo analysis demonstrates the strong performance of the robust estimation approach under complex settings, and confirms the asymptotic normality property and consistency of the proposed estimators.

\setcounter{equation}{0} \numberwithin{equation}{section}

This Supplement provides proofs of the results given in the text of the main paper. It is organised as follows: Section (ref), (ref), (ref) provide proofs of the main theorems. Section (ref) contains auxiliary technical lemmas used in the proofs.

Formula numbering in this supplement includes the section number, e.g. $(8.1)$, and references to lemmas are signified as “Lemma 10.\#", e.g. Lemma 10.1. Theorem references to the main paper include section number and are signified, e.g. as Theorem $2.1$, while equation references do not include section number, e.g. $(1)$, $(2)$.

In the proofs, $C$ stands for a generic positive constant which may assume different values in different contexts.

Proofs of Theorems (ref) and (ref), Corollaries (ref) and (ref), and Lemma (ref)

{\bf Proof of Theorem (ref)}. Notice that in view of ((ref)),

Recall definition ((ref)) of $D$ and $D_g$. Then

since $DD_g^{-1}=O_p(1)$ by ((ref)) of Assumption (ref), $D^{-1}S_{zu}=O_p(1)$ by ((ref)) of Lemma (ref). Moreover, by ((ref)) and ((ref)), $$D_gS_{zz}^{-1}D_g=D_gE[S_{zz}\, |\mathcal{F}_n^*]^{-1}D_g+o_p(1)=O_p(1).$$ This completes the proof of the consistency claim ((ref)) of the theorem. $\Box$ \vskip.2cm Recall that for $p\times p$ symmetric matrices $A$, $B$ and a $p\times 1$ vector $b$ it holds:

where $||A||_{sp}$ denotes the spectral norm and $||A||$ the Euclidean norm of the matrix $A$.

Recall the definition of the information set $\mathcal{F}_n^*=\sigma\big(\mu_t, g_t,h_t,t=1, ..., n\big)$.

\vskip.2cm {\bf Proof of Theorem (ref)}. {\it Proof of ((ref))}. By ((ref)),

Moreover, by the same argument as in the proof of ((ref)),

Hence,

By ((ref)) of Lemma (ref),

This together with ((ref)) implies:

Write

To prove ((ref)), it remains to show that

Notice that $\{ \xi_t\} $ is an m.d. sequence with respect to the $\sigma$-field

$ \mathcal{F}_{n,t}=\sigma(\varepsilon_1, ..., \varepsilon_t; \,\,\mu_s, h_s,g_s, s=1, ..., n)$:

The latter follows noting that the variables $v_n^{-1},d_n,h_t$ are $\mathcal{F}_{n,t-1}$-measurable since they are function of $\mu_s, h_s,g_s, s=1, ..., n$. Similarly, since $\eta_t$'s are $\mathcal{F}_{n,t-1}$ measurable (see Assumption (ref)), the variables $z_t=\mu_t+I_{gt}\eta$ are also $\mathcal{F}_{n,t-1}$-measurable. Finally, by assumption, $\{\mu_s, h_s,g_s, s=1, ..., n\}$ and $\{\varepsilon_s, s=1, ..., n\}$ are mutually independent, and therefore $E[\varepsilon_t |\mathcal{F}_{n,t-1}]= E[\varepsilon_t |\mathcal{F}_{t-1}]=0$ by Assumption (ref). This shows that the conditional expectation property $E[\xi_t \, |\mathcal{F}_{n,t-1}]=0$ is preserved for $\xi_t$ and completes the argument showing that $\xi_t$ is a martingale difference sequence with respect to the $\sigma$-field $\mathcal{F}_{n,t-1}$.

Therefore, by Corollary 3.1 of hal1980, to prove ((ref)), it suffices to show that

Observe that (a) holds with a non-random limit $\eta^2=1$. Thus, the verification of the condition (3.21) of Corollary 3.1, that the $\sigma$-fields are nested, ${\cal F}_{n,t}\subset {\cal F}_{n+1,t}$ for $t=1, ..., n$ and $n \ge 1$, is unnecessary; see remark on page 59 in hal1980. To verify (a), notice that

Then, setting $S_{zzuu}^{(c)}=\sum_{t=1}^n z_tz^\prime_th_t^2E[\varepsilon_t^2\,|\mathcal{F}_{t-1}]$, we can write,

Recall that by ((ref)), $DE[S_{zz}|\mathcal{F}_n^*]^{-1}D=O_p(1)$. We show in ((ref)) of Lemma (ref) that $$ D^{-1}S_{zzuu}^{(c)} D^{-1}= D^{-1}E[S_{zzuu}|\mathcal{F}_n^*]D^{-1}+o_p(1). $$ Together with ((ref)), this implies

which proves (a).

Next we prove (b). We have

By definition of $d_n$, $||d_nD ||^2=||a^\prime D\,E[S_{zz}|\mathcal{F}_n^*]^{-1}D||^2.$ On the other hand, by ((ref)) of Corollary (ref), for any $a$,

where $b_n$ is $\mathcal{F}_n^*$ measurable, and, thus, also $\mathcal{F}_{n,t-1}$ measurable. Then,

Hence,

by ((ref)) of Lemma (ref). This completes the proof (b) and the claim ((ref)) of the theorem.

\vskip.2cm The claim ((ref)) follows from ((ref)) by setting $a=(a_1, ..., a_p)^\prime=(0, ...,0,1,0....)^\prime$ where $a_k=1$ and $a_j=0$ for $j \ne k$. Then $a^\prime D=v_k$ and $a^\prime D\Omega_n D a=v_k^2\omega_{kk}$, where $\omega_{kk}$ is the $(k,k)$-th diagonal element of $\Omega_n$. Then, $$ \frac{a^\prime D( \widehat \beta -\beta)}{\sqrt{a^\prime D\Omega_n D a }} =\frac{( \widehat \beta -\beta)}{\sqrt{\omega_{kk}}} \rightarrow _d \mathcal{N}(0, 1) $$ by ((ref)). This completes the proof of the theorem. $\Box$

\vskip.2cm

{\bf Proof of Corollary (ref)}. We will show that

which together with ((ref)) implies ((ref)):

To prove ((ref)), we will verify that

which implies the following property for diagonal elements: $$ v_k^2 \widehat \omega_{kk}=v_k^2 \omega_{kk}+o_p(1). $$ In ((ref)) of Lemma (ref) it is shown that

for any $a=(a_1, ...,a_p)^\prime$, $||a||=1$ where $b_n,\,b_{n2}>0$ do not depend on $a, n$ and $b_n^{-1}=O_p(1),$ $b_{n2}=O_p(1).$ Set $a=(0, ...,1,...0)^\prime$, where $a_j=0$ for $j \ne k$ and $a_k=1$. Then $a^\prime D\Omega_nD a =v_k^2 \omega_{kk}$, and by ((ref)), $v_k^2 \omega_{kk}\ge b_n>0$. This proves ((ref)):

In addition, the bounds ((ref)) imply that $\sqrt{ \omega_{kk}}\asymp_p v_k^{-1}$:

\vskip.2cm {\it Proof of ((ref))}. Set $V_n=DD_g^{-1}$. By ((ref)) of Assumption (ref), $V_n=O_p(1)$. We have

By ((ref)), ((ref)), ((ref)) and ((ref)) of Lemma (ref),

We will show that

This implies ((ref)):

\vskip.2cm {\it Proof of ((ref))}. By definition,

Notice that $$\sum_{t=1}^n||D^{-1}z_t||^2\le ||D^{-1}D_g||^2\sum_{t=1}^n||D_g^{-1}z_t||^2= O_p(1),$$ since $||D_gD^{-1}||=O_p(1)$ by assumption ((ref)) and $ \sum_{t=1}^n||D_g^{-1}z_t||^2=O_p(1)$ by ((ref)) of Lemma (ref). Hence, to verify ((ref)), it suffices to show that

Recall the equality $\widehat u_t^2-u_t^2=(\widehat u_t-u_t)^2+2(\widehat u_t-u_t)u_t$. Denote $q_n=||D(\beta -\widehat \beta)||$. Then,

Hence,

where $q_n=O_p(1)$ by Theorem (ref), and $$\max_{t=1,..., n} ||D^{-1}z_t||^2=o_p(1), \quad \max_{t=1,..., n} ||D^{-1}z_tu_t||=o_p(1)$$ by ((ref)) of Lemma (ref). This implies ((ref)) and completes the proof of the corollary. $\Box$

\vskip.2cm

{\bf Proof of Corollary (ref)}. Let $\beta_k$ be the true value of the $k$-th component of the parameter $\beta$, and suppose that $\beta_k\ne\beta_k^0$. Write

By ((ref)) of Corollary (ref), $t_{n,1} \rightarrow _d \mathcal{N}(0, 1)$ and $\sqrt{ \omega_{kk}}\asymp_pv_k^{-1}$. Hence, $$ t_{n,1}=O_p(1), \quad t_{n,2}\asymp_pv_k \rightarrow_p \infty. $$ Then, $ t_n=t_{n,1}+t_{n,2}=O_p(1)+t_{n,2}\asymp_pv_k \rightarrow_p \infty, $ which proves the claim of Corollary (ref). $\Box$

\vskip.4cm {\bf Proof of Lemma (ref)}. {\it Proof of ((ref))}. It suffices to show that

Notice also that $z^2_{kt}=\mu^2_{kt}+2\mu_{kt}g_{kt}\eta_{kt}+g^2_{kt}\eta_{kt}^2$,

In addition, by assumption ((ref)) of lemma, $v_{gk}^{-2}=(\sum_{t=1}^n g_{kt}^2)^{-1}=O_p(n^{-1})$. Thus,

We will show that $Ei_{n,1}=o(1)$ which implies ((ref)). Observe that for any $L\ge 1$, $$z^2_{kt}\le L+z^2_{kt}I(z^2_{kt}\ge L)\le L+L^{-1}z^4_{kt}.$$ By assumption ((ref)), $E[z^4_{kt}]\le c<\infty$ where $c$ does not depend on $t,n$. Hence,

which implies $i_n=o_p(1)$ and proves ((ref)).

\vskip.2cm {\it Proof of ((ref))}. It suffices to verify that

By assumption ((ref)) of lemma, $v_{k}^{-2}=O_p(n^{-1})$. This together with ((ref)) implies that

We will show that $Ei_{n,2}=o(1)$ which implies $i_{n,2}=o_p(1)$ and proves ((ref)).

Similarly as above, for any $L\ge 1$, setting $L_0= \log L$, for $\delta>0$ we obtain

By assumption ((ref)), $E[z^4_{kt}]\le c$ and there exists $\delta>0$ such that $E[|u_t|^{4+4\delta}]\le c$, where $c<\infty$ does not depend on $t,n$. Hence, $E[h^{4+4\delta}_{t}]=E[(E[u^2_{t}\, |\mathcal{F}_n^*])^{2+2\delta}]$ $\le E[|u_t|^{4+4\delta}]\le c$. Notice that $A_L\rightarrow 0$ as $L\rightarrow \infty$. Therefore, as $n, L\rightarrow\infty$,

which implies $i_n=o_p(1)$ and proves ((ref)).

\vskip.2cm {\it Proof of ((ref))}. By assumption ((ref)) of Lemma (ref), $E[z_{kt}^4]\le c$ and $E[u_{t}^4]\le c$ where $c<\infty$ does not depend on $t,k,n$. By ((ref)),

where $c<\infty$ does not depend on $t,n$. Hence,

By assumption $(\ref{e:lec1++})$, $n/v_k^2=O_p(1)$ and $ n/v_{gk}^2=O_p(1)$. Thus,

which proves ((ref)). This completes the proof of the lemma. $\Box$

Proofs of Theorem (ref) and Corollaries (ref) and (ref)

{\bf Proof of Theorem (ref)}. Recall the notation introduced in Section (ref). Set $$ \widetilde y_j= b_{n,tj}^{1/2}y_j, \quad \widetilde z_j=b_{n,tj}^{1/2}z_j,\quad \widetilde u_j=b_{n,tj}^{1/2}u_j. $$ Then we can write

Recall the estimator $\widehat \beta_t$ given in ((ref)). In Section (ref) we introduced an auxiliary regression model with a fixed parameter $\beta=\beta_t$:

Recall the OLS estimator $\widehat \beta$ of the fixed parameter $\beta$ in this model, given in ((ref)): $$ \widehat \beta =\big( \sum_{j=1}^n \widetilde z_j \widetilde z_j^\prime\big)^{-1} \big( \sum_{j=1}^n \widetilde z_j y_j^*\big)=\beta+\big( \sum_{j=1}^n \widetilde z_j \widetilde z_j^\prime\big)^{-1} \big( \sum_{j=1}^n \widetilde z_j \widetilde u_j\big). $$ In ((ref)) we showed that following relation:

The remainder $R_t=(R_{1t}, ...., R_{pt})^\prime$ arises due to time variation in the parameter $\beta_j$ and is negligible. We will obtain an upper bound for this term. The term $\widetilde \beta -\beta$ is the main component We will analyse it using the results of Section (ref). Overall, equation ((ref)) shows that properties of $\widehat \beta_t-\beta_t$ are determined by the properties of $\widehat \beta-\beta$, with an additional negligible term $R_t$.

First we will show that the components of $\widehat \beta-\beta=(\widetilde \beta_{1}- \beta_{1}, ..., \widetilde \beta_{p}- \beta_{p})^\prime $ and $R_t$ satisfy the following properties. For $k=1, ..., p$,

\vskip.2cm {\it Proof of ((ref))}. Recall that $\widetilde z_j=(\widetilde z_{1j}, ..., \widetilde z_{pj})^\prime$, and

By Lemma (ref), under assumptions of theorem, $ \widetilde\mu _{kj}$ and the scale factors $\{\widetilde g_{kj}, \widetilde h_j\}$ satisfy Assumptions (ref) and (ref)(ii). Thus, by Theorem (ref),

where $ v_k^2\equiv v_{kt}^2=\sum_{j=1}^n \widetilde g_{kj}^2\widetilde h_j^2=\sum_{j=1}^n b_{n,tj}g_{kj}^2 h_j^2$ and

Indeed, $v_{kt}^{-2}=O_p(H^{-1})$ by ((ref)) of Assumption (ref). On the other hand, ((ref)) implies that $Ev_{kt}^2\le \sum_{j=1}^n b_{n,tj}E[g_{kj}^2 h_j^2]\le c\sum_{j=1}^n b_{n,tj}=O(H)$, where the last relation easily follows using definition of $b_{n,tj}$ and ((ref)). Hence $v_{kt}^2=O_p(H)$, which proves ((ref)).

This complete the proof of the first claim in ((ref)), while the second claim holds by ((ref)) of Theorem (ref). The third claim holds since by ((ref)) of Corollary (ref) and ((ref)),

\vskip.2cm {\it Proof of ((ref))}. Write

We will show that

which implies $||R_t||\le ||S_{\tilde z\tilde z,t}^{-1}||\,||S_{\tilde z\tilde z\beta,t}||=O_p\big((H/n)^\gamma\big). $ Then, $|R_{kt}|\le ||R_t||=O_p\big((H/n)^\gamma\big)$ which proves ((ref)).

To verify ((ref)), recall notation of the $p\times p$ diagonal matrix

Notice that

because $||D_{\widetilde g}^{-1}||^2= \sum_{k=1}^p v_{\widetilde g k}^{-2}=O_p(H^{-1})$ by Assumption (ref). On the other hand,

$D_{\widetilde g}S_{\tilde z\tilde z,t}^{-1}D_{\widetilde g}=O_p(1)$ by ((ref)) and ((ref)) of Lemma (ref). This proves the first claim in ((ref)).

Next, bound

We have $||\widetilde z_j||^4=b_{n,tj}^2|| z_j||^4$. Recall that $E|| z_j||^4\le c$ by Assumption (ref), $E||\beta_j-\beta_t||^2\le c(|t-j|/n)^{2\gamma}$ by Assumption (ref), and it is trivial to show that under ((ref)), $$\sum_{j=1}^n b_{n,tj} (|t-j|/H)^\gamma=O(H).$$ This implies

which proves the second claim in ((ref)).

We now are ready to prove the claims ((ref)) and ((ref)) of the theorem. First, together with ((ref)), the properties ((ref)) and ((ref)) establish the consistency result ((ref)): $$ \widehat \beta_t-\beta_t= (\widetilde\beta-\beta)+R_t=O_p\big(H^{-1/2}+(H/n)^\gamma\big). $$ To prove the asymptotic normality property ((ref)), recall assumption $H=o(n^{2\gamma/(2\gamma+1)})$. Then

because by ((ref)) and ((ref)), $$\omega_{kk,t}^{-1/2}B_t= O_p\big(H^{1/2}\big)O_p\big((H/n)^\gamma\big) =O_p\big(H^{1/2}(H/n)^\gamma\big)=o_p(1)$$ under assumption $H=o(n^{2\gamma/(2\gamma+1)})$. Then,

by ((ref)) which proves the asymptotic normality property ((ref)) of the theorem. Noting that $\sqrt{\omega_{kk,t}}\asymp_p H^{-1/2}$, as shown in ((ref)), this completes the proof of the theorem. $\Box$

\vskip.2cm {\bf Proof of Corollary (ref)}. In the proof of Theorem (ref) we wrote the time-varying regression model as a regression model

with a fixed parameter $\beta=\beta_t$. We showed that the regressors $\widetilde z_j$ and the noise $\widetilde u_j$ satisfy assumptions of Theorem (ref) and that the contribution of the term $r_j$ is asymptotically negligible. That allowed us to establish the asymptotic normality property ((ref)) of Theorem (ref) for $\widehat \beta_{kt}$ using results of Section (ref).

Clearly, to prove Corollary (ref), it suffices to verify the second claim in ((ref)),

Proof of the corresponding result in the case of fixed parameter in Corollary (ref) shows that we need to verify the validity of ((ref)) for our regression model ((ref)), i.e. to show that

where $\widehat u_j=\widetilde y_j- \widehat \beta^\prime \widetilde z_j$, $\widehat \beta=\widehat \beta_t$, $D={\rm diag}(v_1, ...., v_k)^\prime$ and $v_k^2=\sum _{j=1}^n \widetilde g_{kj}^2\widetilde h_j^2$.

Set $\widehat u_j^*=(\beta_t-\widehat \beta_t)^\prime\widetilde z_j+\widetilde u_j$. Write

By ((ref)), $j_{n1}=D^{-1}S_{\widetilde z\widetilde z\widetilde u\widetilde u} D^{-1}+o_p(1).$ Hence, to prove ((ref)), we need to show that

By Assumption (ref), $||D^{-1}||=O_p(H^{-1/2})$. Hence,

We will show that $j_{n3}=o_p(1)$ which implies ((ref)). Notice that

Using the inequality $2|ab|\le a^2+b^2$, we can bound in ((ref)),

Next we evaluate $|r_j\widetilde u_j|$ in ((ref)). Let $L>1$ be large number. Then,

Hence,

$ |\widehat u_j^2-\widehat u_j^{*\, 2}|\le2 r_j^2+||\beta_t-\widehat \beta_t||^2||\widetilde z_j||^2+2 L^{-1} ||\widetilde z_j||\, |\widetilde u_j| +2 L||\widetilde z_j||^{-1}\, |\widetilde u_j|r_j^2. $

Since $r_j^2\le ||\beta_j-\beta_t||^2||\widetilde z_j||^2$, this yields

Recall that $\widetilde z_j=b_{n,tj}^{1/2}z_j$ and $\widetilde u_j=b_{n,tj}^{1/2}u_j$. Denote $\theta_j =2|| z_j||^4 +2||z_j||^3 |u_j|.$ Then,

Hence,

By ((ref)) of Theorem (ref), $||\beta_t-\widehat \beta_t||^2 =o_p(1)$, and $L^{-1}$ can be made arbitrarily small by selecting large $L$. We will show that

Combining this with ((ref)), we obtain $$ |j_{n3}|=Lo_p(1)+\big(o_p(1)+L^{-1}\big)O_p(1), $$ so that the right hand side can be made arbitrarily small by selecting a large enough $L$ and letting $n \rightarrow \infty$. This proves ((ref)).

To bound $Eq_{n1}$ observe that by Assumption (ref), $E ||\beta_t-\beta_j||^2\le C(|t-j|/n)^{2\gamma}$, where $0<\gamma\le 1$ and and recall ((ref)). Then,

when $H=o(n^{2\gamma/(2\gamma+1)})$. This proves ((ref)) for $Eq_{n1}$.

To bound $Eq_{n2}$ and $Eq_{n3}$, recall that by Assumption (ref), $ Ez_{kj}^4\le C$ and $Eu_{j}^4\le C$ which implies that $E\theta_j\le C$. Moreover, under ((ref)) it holds $H^{-1}\sum_{j=1}^nb_{n,tj}=O(1)$ and $b_{n,tj}^2\le Cb_{n,tj}$. Hence,

This completes the proof of ((ref)) and the corollary. $\Box$

\vskip.2cm {\bf Proof of Corollary (ref)}. Let $\beta_{kt}$ be the true value of the $k$-th component of the time-varying parameter $\beta_t$. Suppose that $|\beta_{kt}^0- \beta_{kt}|\ge a>0$ for $t=t_n \in [1,...,n]$ as $n \rightarrow \infty$. Write

By ((ref)) of Corollary (ref), $\tau_{n1, t} \rightarrow _d \mathcal{N}(0, 1)$ and $\sqrt{ \omega_{kk,t}}\asymp_p H^{-1/2}$. Hence, $$ \tau_{n1,t}=O_p(1), \quad \tau_{n2,t}\asymp_pH^{1/2} \rightarrow_p \infty. $$ Then, $ \tau_{n,t}=\tau_{n1,t}+\tau_{n2,t}=O_p(1)+\tau_{n2,t}\asymp_pH^{1/2} \rightarrow_p \infty, $ which proves the claim of the Corollary (ref). $\Box$

{\bf Proof of Lemma (ref)}. Notice that assumptions ((ref)) imply $\sum_{j=1}^nb_{n,tj}\asymp H$. Thus, the claim of Lemma (ref) follows using the same argument as in the proof of Lemma (ref). $\Box$

Proofs of Theorems (ref), (ref) and Theorem (ref)

{\bf Proof of Theorem (ref)}. Suppose that $y_t=\beta^\prime z_t+u_t$ follows the regression model ((ref)). In the presence of missing data, estimation of the parameter $\beta$ is based on a regression model with the fixed parameter ((ref)):

where the regressors $\widetilde z_t=(\widetilde z_{1t}, ..., \widetilde z_{pt})^\prime$ and the noise $\widetilde u_t$ take the form

and $\tau_t$ is the missing data indicator. Under Assumptions (ref) and (ref) of the theoren, $\{\widetilde \mu_{t}, \widetilde g_{t},\widetilde h_{t}\}$ are independent of $\{\varepsilon_t, \eta_t\}$. Therefore, $(z_t,u_t)$ belongs to the regression space described in ((ref)) and ((ref)) of Section (ref).

We estimate the fixed parameter $\beta$ using the estimator defined in ((ref)):

We will show that $(\widetilde z_t,\widetilde u_t)$ satisfy Assumptions (ref), (ref), (ref) and (ref) of Theorem (ref) of Section (ref). Then, the required result ((ref)) for $\widehat \beta$ of this theorem follows directly from the claims ((ref)) of Corollary (ref).

We split Assumptions (ref), (ref), (ref) and (ref) into two groups:

(a) Assumptions (ref), (ref), and (ref)(i), and

(b) Assumptions (ref) and (ref)(ii).

Assumptions (a) imposed on the stationary processes $ \eta_t, \varepsilon_t$ are part of Assumption (ref) of Theorem (ref).

It remains to show the validity of the assumptions in group (b), i.e. that the means $\widetilde \mu_t$ and the scales $\widetilde h_t, \widetilde g_t$ satisfy Assumptions (ref) and (ref)(ii). By Assumption (ref), we have $Ez_{kt}^4\le c$ and $ E u_{t}^4\le c$. Moreover, $g_{kt} \ge c_1>0$ and $h_{t} \ge c_1>0$ where $c, c_1>0$ do not depend on $k,t$ and $n$. For $k=1, ..., p$, define:

Notice that $$ \widetilde v_k^2\ge c_1^4\sum_{t=1}^n \tau_t=c_1^4 N, \quad \widetilde v_{gk}^2\ge c_1^2\sum_{t=1}^n \tau_t=c_1^2 N, $$ where $N$ is the size of the subsample ((ref)). By assumption of the theorem, $n/N=O_p(1)$. Thus,

which confirms the validity of Assumptions (ref) and (ref)(ii); see Lemma (ref). This completes the proof of the theorem. $\Box$

\vskip.2cm {\bf Proof of Theorem (ref)}. Now, suppose that $y_t=\beta^\prime_t z_t+u_t$ follows the regression model ((ref)) with a time-varying parameter $\beta_t$. In the presence of missing data, estimation of the time-varying parameter $\beta_t$ is based on a model ((ref)):

Here, the regressors $\widetilde z_t=(\widetilde z_{1t}, ..., \widetilde z_{pt})^\prime$ and the noise $\widetilde u_t$ are the same as in ((ref)). We showed in the proof of the Theorem (ref) that $(z_t,u_t)$ belongs to the regression space described in ((ref)) and ((ref)) of Section (ref).

The estimator of the time-varying parameter $\beta_t$ is given in ((ref)):

We will show that $(\widetilde z_t,\widetilde u_t)$ satisfy Assumptions (ref), (ref), (ref)(i), (ref) and (ref) of Theorem (ref). Then, the results ((ref)), ((ref)) and ((ref)) for $\widehat \beta_t$ of Theorem (ref) follow from the results ((ref)) of Theorem (ref) and ((ref)) of Corollary (ref).

Observe that Assumptions (ref), (ref), (ref)(i) on $\eta_t, \varepsilon_t$ are part of Assumption (ref) of this theorem, which also includes Assumption (ref) for $\beta_t$.

It remains to show that $\widetilde z_t, \widetilde u_t$ satisfy Assumption (ref). This requires to prove the validity of ((ref)) and ((ref)) under Assumption (ref) of this theorem, which we showed in the proof of Theorem (ref). $\Box$

\vskip.2cm {\bf Proof of Theorem (ref)}. We consider a stationary AR($p$) model ((ref)), $$ y_t=\phi_0+\phi_1y_{t-1}+...+\phi_py_{t-p}+\varepsilon_t, $$ where $\varepsilon_t$ is a stationary $m.d.$ sequence with respect to the information set ${\cal F}_t=\sigma(\varepsilon_s, s\le t)$. Write it as a regression model ((ref)),

with fixed parameter $\beta=(\beta_1, ..., \beta_{p+1})^\prime=(\phi_0, ...., \phi_p)^\prime$ and regressors $z_t=(z_{1t},z_{2t},...,z_{p+1,t})^ \prime=(1,y_{t-1},y_{t-2}, ..., y_{t-p})^\prime $. Under assumption ((ref)) of theorem, AR($p$) model has a stationary solution:

and regressors $$z_{kt}= \mu_{kt}+g_{kt}\eta_{kt}, \quad \mu_{kt}=E[y_{t-k}]=Ey_1, \quad g_{kt}=1, \quad \eta_{kt}=y_{t-k}-E[y_{t-k}], $$ for $k=2, ..., p+1$, satisfy regression assumption ((ref)). From ((ref)) it follows that the regressors $\eta_t=(\eta_{1t}, ...., \eta_{pt})^\prime =(y_{t-1},y_{t-2}, ..., y_{t-p})^\prime $ are $\mathcal{F}_{t-1}=\sigma(\varepsilon_s, s\le t-1)$ measurable. Moreover, under the assumptions of the theorem, $(\varepsilon_t, \eta_t)$ satisfy Assumptions (ref), (ref), (ref) and (ref) of Theorem (ref) in Section (ref). Finally, we show that $Ey_t^8\le C<\infty$. Recall that by the assumption of theorem, $\varepsilon_t$ is a stationary $m.d.$ sequence such that $E\varepsilon_t^8<\infty.$ It is known that if $E|\varepsilon_t|^p<\infty$, for some $p>2$, then $$ E\big|\sum_{j=0}^\infty a_j\varepsilon_{t-j}\big|^p\le C\big(\sum_{j=0}^\infty a_j^2\big)^{p/2}, $$ where $C<\infty$ does not depend on $n$; see e.g., Lemma 2.5.2 in GKS2012. Hence $E(y_t-\mu)^8<\infty$ and $E\eta_{kt}^8<\infty$ from $k=1, ..., p$.

Thus, regressors $z_t$ and regression noise $u_t=\varepsilon_t$ satisfy Assumptions (ref), (ref), (ref) and (ref) of Section (ref). Therefore, the robust OLS estimator $\widehat \beta$ of $\beta$ has properties derived in Corollary (ref) which implies Theorem (ref). $\Box$

Proofs of Section (ref): Auxiliary lemmas

This section contains auxiliary lemmas used in the proofs of the main results for Section (ref). For the ease of referencing, we include the statement of Lemma (ref)(i) established in glp2024Suppl.

{\bf Proof of Lemma (ref).} The claim (i) of Lemma (ref) was derived in (glp2024Suppl, Lemma A5). To prove (ii), denote $s_n=\sum_{t=1}^n|\beta_t|$. Then,

This implies

This completes the proof of ((ref)) and the lemma. $\Box$ \vskip.2cm Recall notation

Recall definition $\mathcal{F}_n^*=\sigma(\mu_t, g_t, t=1, ..., n)$ and $\mathcal{F}_{n,t-1}$ in ((ref)). Denote

\vskip.2cm Before the proof of lemma, we will state the following corollary. Denote

Notice that under ((ref)) of Assumption (ref),

\vskip.2cm {\bf Proof of Lemma (ref)(i)}. {\it Proof of ((ref))}. Set $I_{gt}={\rm diag}(g_{1t},..., g_{pt})$. By definition,

Then

where $ e_t=E[\widetilde z_t|\mathcal{F}^*_n]=I_{gt}E[\eta_t]$. Using ((ref)), we can write

We split the proof into two cases when regression model ((ref)) does not include intercept and when intercept is included.

Case 1 (no intercept): $e_t=I_{gt}E[\eta_t]=(0,...,0)^\prime$.

Case 2 (intercept included): $e_t=I_{gt}E[\eta_t]=I_{gt}(1,0,...,0)^\prime=(g_{1t},0,...,0)^\prime$, $g_{1t}=1$.

\vskip.2cm {\bf Case 1}. Let $e_t=0$. Then ((ref)) implies

In this instance, $$ E[ \widetilde z_t \widetilde z^\prime_t |\mathcal{F}^*_n]= I_{gt}E[\eta_t\eta_t^\prime]I_{gt} =I_{gt}\Sigma I_{gt}, $$ where $ E[\eta_t\eta_t^\prime]=\Sigma =(\sigma_{jk})_{j,k=1, ...,p}$. By Assumption (ref)(ii), the matrix $\Sigma$ is positive definite. Therefore, there exists $b>0$ such that for any $\alpha=(\alpha_{1}, ..., \alpha_p)^\prime $, $$ \alpha^\prime \Sigma \alpha \ge b||\alpha||^2. $$ Hence, setting $\gamma_{kt}=v_{gk}^{-1}g_{kt}$, we derive

since $\sum_{t=1}^n \gamma_{kt}^2=1$ and $||a||=1$. With ((ref)) this proves the first claim in ((ref)):

Matrix $W_{zz}$ is symmetric and, thus, it has real eigenvalues. The bound ((ref)) implies that the smallest eigenvalue of $W_{zz}$ has property $ \lambda_{ min}\ge b_n>0$. Therefore $W_{zz}$ is positive definite, and the largest eigenvalue $\theta_{\max}$ of $W_{zz}^{-1}$ has property $\theta_{\max}=\lambda_{min}^{-1}\le 1/b_n$, which implies that $||W_{zz}^{-1}||_{sp}\le 1/b_n$. This proves the second claim in ((ref)).

\vskip.2cm {\bf Case 2} (intercept included): $e_t=I_{gt}E[\eta_t]=I_{gt}(1,0,...,0)^\prime=(g_{1t},0,...,0)^\prime$. Recall that in presence of intercept, $g_{1t}=1$ and $\eta_{1t}=1$.

\vskip.2cm {\it Proof of ((ref))}. Set $a=(a_1, ..., a_p)^\prime$, $\widetilde a=(a_2, ..., a_p)^\prime$. Recall that

We will show that there exists $b>0$ such that for any $a$ and $n \ge 1$,

where $c_{*,n}$ is defined as in ((ref)). These bounds imply ((ref)). Indeed, suppose that $ ||\widetilde a||> (1-b)|a_1|/(2c_{*,n}^{1/2})$. By ((ref)), this is equivalent to

Then, by ((ref)),

On the other hand, if $ ||\widetilde a||\le (1-b)|a_1|/(2c_{*,n}^{1/2})$, then in ((ref)), $$a_1^2-2|a_1|\,||\widetilde a||c_{*,n}^{1/2}\ge a_1^2-(1-b)a_1^2= b\,a_1^2$$ which together with ((ref)) implies

Therefore,

This implies that there exists $c>0$ such that

where $ b_n^{-1}=c(1+c_{*,n})=O_p(1) $ by ((ref)). This verifies the first claim in ((ref)).

\vskip.2cm {\it Proof of ((ref))}. Below we will show that there exists $b>0$ such that

In addition, observe that in Case 2,

Then from ((ref)), using ((ref)) and ((ref)) we arrive at ((ref)):

{\it Proof of ((ref))}. By ((ref)) and ((ref)),

By Cauchy inequality and ((ref)),

Since $\mu_{1t}=0$, then $ | a^\prime D_g^{-1}\mu_t|\le ||\widetilde a||\,|| D_g^{-1}\mu_t||. $ Hence, using notation $c_{*,n}$ introduced in ((ref)), we obtain

which together with ((ref)) and ((ref)) proves ((ref)):

\vskip.2cm {\it Proof of ((ref))}. Recall, that in presence of intercept, $\eta_t=(1,\eta_{2t}, ..., \eta_{pt})^\prime$ and $E[\eta_{kt}] =0.$ Denote $\tilde \eta=(\eta_{2t}, ...., \eta_{pt})^\prime$ and $\widetilde \Sigma =E[\tilde \eta\tilde \eta^\prime]$. Then $$ E[ \widetilde z_t \widetilde z^\prime_t |\mathcal{F}^*_n]= I_{gt}E[\eta_t\eta_t^\prime]I_{gt} =I_{gt}{\rm diag}(1, \widetilde\Sigma )I_{gt}={\rm diag}\big(g_{1t}^2, \tilde I_{gt}\widetilde\Sigma\tilde I_{gt}\big), $$ where ${\rm diag}(1, \widetilde\Sigma)$ is a block diagonal matrix and $\widetilde I_{gt}={\rm diag}(g_{2t}, ..., g_{pt})$. By assumption, the matrix $\widetilde \Sigma$ is positive definite. Denote $\widetilde D_g={\rm diag}(v_{g2}, ..., v_{gp})$. Then,

Observe that $i_{n,1}=a_1^2$ since $v_{g1}^{-2}\sum_{t=1}^n g^2_{1t}=1$. Recall that $||\widetilde a||\le 1$. Hence, by ((ref)),

for some $b>0$ which does not depend on $n$ and $a$. This implies ((ref)).

Summarizing, note that by ((ref)) and ((ref)),

where $c>0$ does not depend on $n$. Notice that $b_n^{-1}\le c(1+c_{*,n})=O_p(1)$ by ((ref)). This proves the first claim in ((ref)).

Proof of the second claim in ((ref)) is the same as in Case 1.

\vskip.2cm {\it Proof of ((ref))}. Observe that

by ((ref)) of Lemma (ref). This proves ((ref)). \vskip.2cm

{\bf Proof of ((ref)), ((ref)), ((ref)) and ((ref))}. Denote by $\delta_{jk}$ the $jk$-th element of the matrix

To prove ((ref)), it remains to show that

\vskip.2cm {\bf Case 1}: $e_t=0$. Then, by ((ref)), we have $$ z_tz^\prime_t-E[z_tz^\prime_t|\mathcal{F}_n^*]=I_{gt}(\eta_t\eta_t^\prime-E[\eta_t\eta_t^\prime])I_{gt} +\mu_t\eta_t^\prime I_{gt}+I_{gt}\eta_t\mu_t^\prime. $$ Therefore, setting $\gamma_{jt}=v_{gj}^{-1}g_{jt}$, we can write

By assumption, sequences $\{w_{1t}= \eta_{jt}\eta_{kt}-E[\eta_{jt}\eta_{kt}]\}$, $\{w_{2t}= \eta_{kt}\}$ and $\{w_{3t}= \eta_{jt}\}$ are covariance stationary short memory sequences with zero mean, and the weights $\{b_{1t}= \gamma_{jt}\gamma_{kt}\}$ are independent of $\{w_{1t}\}$, $\{b_{2t}=v_{gj}^{-1}\mu_{jt}\gamma_{kt}\}$ are independent of $\{w_{2t}\}$ and $\{b_{3t}= v_{gk}^{-1}\mu_{kt}\gamma_{jt}\}$ are independent of $\{w_{3t}\}$, Thus, applying Lemma (ref) to $S_{n,i}, \,i=1,2,3$, we obtain

Denote $r_{jn}=\max_{t=1,...,n}\gamma_{jt}^2$. Then,

Notice that $\sum_{t=1}^n\gamma_{kt}^2=1$. Observe that $r_{jn}=o_p(1)$ by ((ref)) and $v^{-2}_{gj}\sum_{t=1}^n\mu_{jt}^2=O_p(1)$ by ((ref)) of Assumption (ref). This implies $\delta_{jk}^2=o_p(1)$ which proves ((ref)).

\vskip.2cm {\bf Case 2}. Let $e_t=(1, 0,...,0)^\prime$.

To prove ((ref)), it suffices to show that $\delta_{jk}$, $j,k=1, ..., p$ in ((ref)) have property ((ref)): $\delta_{jk}=o_p(1)$. Recall that in presence of intercept we have $z_t=(1, z_{2t}, ..., z_{pt})^\prime$.

First, observe that for $j,k=2, ...,p$, $\delta_{jk}$ are the same as in ((ref)) and whence $\delta_{jk}=o_p(1)$ by ((ref)). Second, $\delta_{11}=0$ since $z_{1t}=1$. Finally, for $k=2, ..., p$, we have

Then,

By assumption, $\{\eta_{kt}\}$ is a covariance stationary short memory sequence with $E[\eta_{kt}]=0$, and $\{\eta_{kt}\}$ and $\{\gamma_{kt}\}$ are mutually independent. Therefore, by Lemma (ref), $$ \delta_{1k}=n^{-1/2} O_p\Bigl(( \sum_{t=1}^n\gamma_{kt}^2)^{1/2} \Bigr)=n^{-1/2}O_p(1)=o_p(1) $$ which proves ((ref)). This completes the proof of ((ref)) in Case 2.

\vskip.2cm {\it Proof of ((ref))}. It follows using the same argument as in Case 1.

\vskip.2cm {\it Proof of ((ref))}. To prove that $D^{-1}S_{zu}=O_p(1)$, write

It suffices to show that

We have

By Assumptions, (ref) and (ref), the sequences $\{w_{1t}=\varepsilon_t\}$, $\{w_{2t}=\eta_{kt}\varepsilon_t\}$ are covariance stationary short memory sequences with zero mean, the weights $\{b_{1t}=v_{k}^{-1}\mu_{kt}h_t\}$ are independent of $\{w_{1t}\}$, and $\{b_{2t}=v_{k}^{-1}g_{kt}h_t\}$ are independent of $\{w_{2t}\}$.

Thus, applying Lemma (ref) to each of the sum $S_{n,1}, S_{n,2}$, we obtain

Notice that,

by ((ref)) of Assumption (ref) which proves ((ref)).

\vskip.2cm {\it Proof of ((ref))}. Observe that by ((ref)) and ((ref)) of Lemma (ref), $D_g^{-1}(\sum_{t=1}^nz_tz_t^\prime) D_g^{-1}=O_p(1)$. Therefore, $$\sum_{t=1}^n||D_g^{-1}z_t||^2={\rm trace}\Big(D_g^{-1}(\sum_{t=1}^nz_tz_t^\prime) D_g^{-1}\Big)=O_p(1).$$ This proves ((ref)) and completes the proof of the part (i) of the lemma.

\vskip.2cm {\bf Proof of Lemma (ref) (ii)}. {\it Proof of ((ref))}. We can write

Let $\delta>0$ be a small number which will be selected below. Then,

Thus,

We will show that there exist $b_n>0$ and $\delta=\delta_n>0$ such that $b_n^{-1}=O_p(1)$, $\delta_n^{-1}=O_p(1)$ and for any $a=(a_1, ..., a_p)^\prime$, $||a||=1$ and $n\ge 1$,

Using these bounds in ((ref)), we obtain

\vskip.2cm First we prove ((ref)). Setting

we can write $$ q_{1,n}=\sum_{t=1}^n a^\prime D_{g^*}^{-1}E[ Z_t Z^\prime_t|\mathcal{F}^*_n]D_{g^*}^{-1}a= a^\prime \, W_{ZZ} \, a. $$ Observe that the variables $Z_t=\mu_t^*+I_{g^*t}\eta_t$ satisfy assumptions of Lemma (ref)(i). Hence by ((ref)),

where $c>0$ does not depend on $n$. Notice that $b_n^{-1}\le c(1+c_{**,n})=O_p(1)$ by ((ref)). This proves ((ref)).

To prove ((ref)), recall that $||a||=1$. Bound $$ q_{n,2}\le ||a||^2q_{n,2}^*=q_{n,2}^*, \quad q_{n,2}^*= \sum_{t=1}^n E[||D^{-1}z_th_t||^2I(\varepsilon_t^2< \delta)|\mathcal{F}^*_n\big]. $$ In ((ref)) of Lemma (ref) we show that $q_{n,2}^*\le c_1(1+c_{**,n}) \delta ^{1/4}$, where $c_1>0$ does not depend on $n$ and $c_{**,n}$ is defined in ((ref)). Thus, selecting

we obtain $ q_{n,2}\le c_1(1+c_{**,n}) \delta_n ^{1/4}= b_n/2, $ which proves the bound ((ref)). Notice that $\delta_n \le (2cc_1)^{-4}$ can be made small by selecting large $c$ in ((ref)).

In turn, by ((ref)),

where $b_n$ is defined in ((ref)). This implies

for some $c>0$ which does not depend on $n$. Notice that $b_n^*$ is $\mathcal{F}^*_n$ measurable, and $(b_n^*)^{-1}\le c(1+c_{**,n})^{9}=O_p(1)$ by ((ref)). This proves the first claim in ((ref)). The second claim follows using the same argument as in the proof of ((ref)).

\vskip.2cm {\it Proof of ((ref))}. Observe that

by ((ref)) of Lemma (ref) which implies ((ref)).

\vskip.2cm {\it Proof of ((ref))}. Write $D\Omega_n D= W_{zz}^{-1}W_{zzuu}W_{zz}^{-1}$, $(D\Omega _nD)^{-1}= W_{zz}W_{zzuu}^{-1}W_{zz}$. By ((ref)), ((ref)), ((ref)) and ((ref)),

We will show that

Since $b_n^{-1}=b_{n5}=O_p(1)$ this proves the first claim in ((ref)). To verify ((ref)), notice that the smallest eigenvalue $\lambda_{min}$ of the matrix $D\Omega D$ and the largest eigenvalue $\theta_{max}$ of the inverse matrix $(D\Omega_n D)^{-1}$ are related by the equality $\theta_{max}=\lambda_{min}^{-1}$. By ((ref)), $\theta_{max}\le b_{n5}$. Thus, for $||a||=1$, $$ a^\prime D\Omega_n D a\ge \lambda_{min}=\theta_{max}^{-1}\ge b_n:= b_{n5}^{-1}, $$ where $b_n^{-1}=b_{n5}=O_p(1)$ which proves ((ref)). Finally, by ((ref)), for $||a||=1$, $a^\prime D\Omega_n D a\le ||D\Omega_n D||_{sp}\le b_{n4}=O_p(1)$ which proves the second bound in ((ref)).

\vskip.2cm {\bf Proof of ((ref)), ((ref)) and ((ref))}. Write

To prove ((ref)), it suffices to verify that

Recall that $z_t=\mu_t+\widetilde z_t$ and $u_t=h_t\varepsilon_t$, where $E\varepsilon_t^2=1$. Hence,

By ((ref)),

Then,

Therefore, setting $\gamma_{jt}=v_j^{-1}g_{jt}h_t$, it follows that

To prove ((ref)), it suffices to show that

By Assumption (ref), $\{\eta_{jt}\eta_{kt}\varepsilon_t^2\}$, $\{\eta_{kt} \varepsilon_t^2\}$ and $\{\varepsilon_t^2\}$ are covariance stationary short memory zero mean sequences, and these sequences are mutually independent of the weights $\{\gamma_{jt}\gamma_{kt}\}$, $\{v_j^{-1}\mu_{jt}h_t\gamma_{kt}\}$ and $\{(v_j^{-1}\mu_{jt}h_t)(v_k^{-1}\mu_{kt}h_t)\}$. Moreover, definition of $v_k$ and $\gamma_{kt}$ and ((ref)) of Assumption (ref) imply that $$ \sum_{t=1}^n \gamma^2_{kt}=1, \quad v_k^{-2}\sum_{t=1}^n\mu_{kt}^2h^2_t=O_p(1) $$ and by ((ref)) of Assumption (ref), $$ \max_{t=1, ..., n}\gamma^2_{kt}=o_p(1), \quad v_k^{-2}\max_{t=1, ..., n}\mu_{kt}^2h^2_t=o_p(1). $$ Thus, ((ref)) follows by using Lemma (ref) and applying a similar argument as in the proof of ((ref)). This completes the proof of ((ref)).

The claim ((ref)) follows using ((ref)) and property $W_{zzuu}^{-1}=O_p(1)$ of ((ref)):

\vskip.2cm {\it Proof of ((ref))}. Write

By ((ref)), $D^{-1}S_{zzuu}D^{-1}= W_{zzuu}+o_p(1).$ We will show that

which together with ((ref)) implies ((ref)): $ D^{-1}S_{zzuu}^{(c)}D^{-1}=W_{zzuu}+o_p(1).$ We have, $u_t^2-E[u_t^2|\mathcal{F}_{n,t-1}]=h_t^2(\varepsilon_t^2-\sigma_t^2)$, where $\sigma_t^2=E[\varepsilon_t^2\,|\mathcal{F}_{t-1}]$. Write

Then ((ref)) follows if we show that

We have $z_t=\mu_t+\widetilde z_t$ and $u_t=h_t\varepsilon_t$. So,

Hence, denoting $\gamma_{jt}=v_j^{-1}g_{jt}h_t$, we obtain

Observe, that sequences $\{w_{1t}=\eta_{jt}\eta_{kt}(\varepsilon_t^2-\sigma_t^2)\}$, $\{w_{2t}=\eta_{kt}(\varepsilon_t^2-\sigma_t^2)\}$, $\{w_{3t}=\eta_{jt}(\varepsilon_t^2-\sigma_t^2)\}$, $\{w_{4t}=\varepsilon_t^2-\sigma_t^2\}$ are sequences of uncorrelated random variables with zero mean and constant variance. For example, by assumption, $\eta_{jt}\eta_{kt}$ are $\mathcal{F}_{t-1}$ measurable. Then, for $t\ge s$,

Then using the same argument as in the proof of ((ref)) it follows

which proves ((ref)) and completes the proof of ((ref)).

This completes the proof of the part (ii) and of the lemma. $\Box$

\vskip.2cm {\bf Proof of Corollary (ref)}. The claim ((ref)) is shown in ((ref)), and the claim ((ref)) is shown in ((ref)). $\Box$

\vskip.2cm

{\bf Proof of Lemma (ref)}. {\it Proof of ((ref))}. Denote

By ((ref)),

By Assumption (ref)(i) and Assumption (ref)(i),

$ E[\theta_{1t} \ |\mathcal{F}^*_n]=E[\theta_{1t}]=E[\theta_{11}], \quad E[\theta_{2t} \ |\mathcal{F}^*_n]=E[\theta_{2t}]=E[\theta_{21}]. $

This implies

Notice that

by definition ((ref)) of $c_{*,n}$ and $c_{**,n}$ and because

Moreover, $c_{*,n}=O_p(1)$, $c_{**,n}=O_p(1)$ by ((ref)). Clearly, ((ref)) and ((ref)) prove ((ref)).

\vskip.2cm {\it Proof of ((ref))}. Denote

$\theta_{2t}(\delta)= I(\varepsilon_t^2< \delta)+||\eta_t||^2I(\varepsilon_t^2< \delta). $

Recall, that by assumption, $\varepsilon_t$ is a stationary sequence, and by Assumption (ref)(i), $E[||\eta_t||^4]=E[||\eta_1||^4]$. Then,

We will show that for sufficiently small $\delta>0$,

$ \mbox{$E[I(\varepsilon_1^2< \delta)]\le C\delta^{1/2}.$} $

Indeed, by Assumption (ref), the variable $\varepsilon_1$ has probability distribution density $f(x)$ and $f(x)\le c<\infty$ when $|x|\le x_0$ for some $x_0>0$. Without restriction of generality assume that $\delta\le x_0$. Then, $$ \mbox{$E[I(\varepsilon_1^2< \delta)]=\int I(|x|\le \delta^{1/2})f(x)dx \le c\int I(|x|\le \delta^{1/2})dx\le C\delta^{1/2}.$} $$ Therefore, $E[\theta_{2t}(\delta)]\le C\delta^{1/4}$, and as in ((ref)), we obtain

which proves ((ref)).

\vskip.2cm {\it Proof of ((ref))}. We will prove the first claim (the proof of the second claim is similar). By ((ref)), $ ||D^{-1}z_tu_t||^2\le2 b_{2t}\theta_{2t}. $ Let $K>0$ be a large number. Then, $\theta_{2t}\le K+ \theta_{2t}I(\theta_{2t}\ge K)$. Therefore,

By ((ref)) of Assumption (ref) and ((ref)),

Since $\{b_t\}$ and $\{\theta_{2t}\}$ are mutually independent, then by ((ref)) of Lemma (ref),

We will show that

where $\Delta_{K}\rightarrow 0$, $K \rightarrow \infty$ and $\Delta_{K}$ does not depend on $n$. Together with ((ref)) this implies

Next we prove ((ref)). Set $L=K^{1/4}$. Then, letting $\varepsilon_{L,t}^{2+}=\varepsilon_t^2I(\varepsilon_t^2>L)$, we obtain

since, as $K\rightarrow \infty$, $L^2/K=K^{-1/2} \rightarrow 0$, $E[(\varepsilon_{L,1}^{2+})^2] \rightarrow 0$ and $E[||\eta_1||^4<\infty$. This implies ((ref)).

\vskip.2cm {\it Proof of ((ref))}. Denote by $i_n$ the left hand side of ((ref)). By ((ref)), $||D^{-1}z_tu_t||^2\le 2 b_{2t}\theta_{2t}.$ Let $K>0$ be a large number. Then,

Observe, that $b_n^{-1} b_{2t}$ is $\mathcal{F}_{n,t-1}$ measurable. Then,

$ i_n\le \epsilon_n^{-1}K^2(2b_n^{-1})^2 \sum_{t=1}^nb_{2t}^2+ 2b_n^{-1}\sum_{t=1}^n b_{2t}\theta_{2t}I(\theta_{2t}> K). $

Together with ((ref)), ((ref)) and ((ref)), this implies:

This proves ((ref)) and completes the proof of the lemma. $\Box$

Additional Monte Carlo simulation results

In this section, we further evaluate the finite-sample performance of our robust OLS estimation method using two examples of regression models with fixed parameters, where the regressors $z_t$ and regression noise $u_t$ exhibit complex, non-standard structures.

\vskip.2cm {\bf Example 1}. As in the Monte Carlo section of the main paper, we generate arrays of samples from a regression model with a fixed parameter and an intercept, using a sample size of $n = 1500$ and $1000$ replications. We first consider the following model:

We specify the scale factor $h_t$ in the regression noise $u_t=h_t\varepsilon_t$ as a deterministic trend $h_t=0.4(t/n)$, and a stationary martingale difference noise $\varepsilon_t$ is generated from a GARCH($1,1$) process

Define the regressors as $z_{1t}=1$ and $z_{kt}=\mu_{kt}+g_{kt}\eta_{kt}$ for $k=2,3$, where

Figure (ref) displays plots of a sample of variables $y_t$, $z_t$, and $u_t$ for $t = 1, \dots, 1500$ generated by Model ((ref))-((ref)), which exhibit clear patterns of non-stationary behavior. The Monte Carlo simulation results for sample size $n = 1500$ based on 1000 replications are reported in the Table (ref). Since the regressors $z_t$ and regression noise $u_t$ in this model satisfy the assumptions of Corollary (ref), as expected, the Monte Carlo simulation results confirm excellent performance of the robust OLS estimator. In particular, the empirical coverage of the $95\%$ confidence intervals is close to the nominal $95\%$, whereas the standard OLS estimator exhibits significant coverage distortions.

\vskip.2cm {\bf Example 2}. Next, we provide an example of a regression model in which the components $\beta_1, \beta_2, \beta_3$ of the fixed regression parameter are estimated at different rates. Consider regression model ((ref)) with $\varepsilon_t$, $\eta_{2t},\eta_{3t}$ defined as in Example 1. Set $h_t \equiv 1$, and let the means $\mu_{kt}$ and scale factors $g_{kt}$, $k=2,3$ be defined as follows:

This model satisfies the assumptions of Corollary (ref) (see also Remark (ref) in the main paper). Therefore, the corresponding $t$-statistics for $k=1,2,3$ have the following property:

where, the robust standard errors $\sqrt{ \widehat \omega_{kk}}$ are inversely proportional to the consistency rate $$ v_k= \big(\sum_{j=1}^n g_{jt}^2\big)^{1/2}. $$ In this model, the intercept $\beta_1$ associated with the regressor $z_{1t}=1$ is estimated at the consistency rate $v_1=\sqrt n$; the parameter $\beta_2$ linked with the regressor $z_{2t}$ (with $g_{2t}=t$) at the rate $v_2\sim n^{3/2}$, and the parameter $\beta_3$ linked with the regressor $z_{3t}$ (with $g_{3t}=t^\gamma$) at the rate $v_3\sim n^{\gamma+1/2}$. The rate $v_3$ is super-fast, $n$, when $\gamma=1/2$; standard, $n^{1/2}$, when $\gamma=0$; super-slow, $n^{1/4}$, when $\gamma=-1/4$; and logarithmic, $\log n$, when $\gamma=-1/2$. Monte Carlo results reported in Table (ref) confirm the validity of the normal approximation ((ref)) in finite samples ($n=1500$, based on $1000$ replications). In particular, the coverage of the robust $95\%$ confidence intervals is close to the nominal level for all three parameters $\beta_1, \beta_2, \beta_t$ and for all values of $\gamma$ considered in the construction of the regressor $z_{3t}$. In contrast, the coverage rates based on the standard OLS method exhibit noticeable distortions, especially for $\beta_{2t}$ and $\beta_{3t}$.

As expected, smaller values of $\gamma$ are associated with slower consistency rates $v_3$, wider confidence intervals, and larger standard deviations for the estimator of $\beta_3$.

\vskip.2cm Table (ref) reports the estimation results for the parameters $\beta_1, \beta_2, \beta_3$ for sample sizes $n=200, 800, 1500, 3000$, when the regressor $z_t$ is generated with $\gamma=-1/4$ and $\beta_3$ is estimated with the super-slow rate $v_3=n^{1/4}$. The coverage rates for the robust OLS method are close to the nominal level in all cases. As expected, as $n$ increases, the standard errors of all three parameter estimates decrease; however, for $\beta_3$, which is estimated with the super-slow rate $ n^{1/4}$, the reduction in the standard deviation is relatively slow.

\vskip.2cm Figure (ref) displays plots of a single sample of the variables $y_t$ and $z_{3t}$ for $t = 1, \dots, 1500$ generated by Model ((ref)) for $\gamma=1/2,0,-1/4,-1/2$. These samples exhibit clear patterns of non-stationary behavior.