High-dimensional mixed-frequency IV regression

Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.

76,029 characters · 12 sections · 63 citation commands

Rendered from LaTeX for readability, not typeset faithfully. Citation keys are highlighted; maths is left as source; figures, tables and equation environments are summarised rather than reproduced; unrecognised commands are greyed out so nothing is silently dropped. Email addresses are removed.

abstractThis paper introduces a high-dimensional linear IV regression for the data sampled at mixed frequencies. We show that the high-dimensional slope parameter of a high-frequency covariate can be identified and accurately estimated leveraging on a low-frequency instrumental variable. The distinguishing feature of the model is that it allows handing high-dimensional datasets without imposing the approximate sparsity restrictions. We propose a Tikhonov-regularized estimator and derive the convergence rate of its mean-integrated squared error for time series data. The estimator has a closed-form expression that is easy to compute and demonstrates excellent performance in our Monte Carlo experiments. We estimate the real-time price elasticity of supply on the Australian electricity spot market. Our estimates suggest that the supply is relatively inelastic and that its elasticity is heterogeneous throughout the day.

{ Keywords: high-dimensional IV regression, mixed-frequency data, identification, Tikhonov regularization, continuum of moment conditions, real-time price elasticities. \\ JEL Classifications: C14, C22, C26, C58}

\thispagestyle{empty}

\setcounter{page}{1}

Introduction

The technological progress over the past decades has made it possible to generate, to collect, and to store new intraday high-frequency time series datasets that are widely available along with the "old" low-frequency data. Indeed, the economic activity occurs in real time and the economic and financial transactions are frequently recorded instantaneously, while the traditional time series data are available at a quarterly, monthly, or sometimes daily frequencies. Ignoring the high-frequency nature of the data leads to the loss of the information through the temporal aggregation and makes it impossible to quantify the economic activity in real time. At the same time, combining the low and the high-frequency datasets allows obtaining more refined measures of the economic activity that can be used subsequently to inform market participants and to guide policies.

In this paper, we introduce a novel high-dimensional mixed-frequency instrumental variable (IV) regression suitable for the datasets recorded at different frequencies. The model connects a low-frequency dependent variable to endogenous covariates sampled from a continuous-time stochastic process. Alternatively, the regressor might be sampled from a continuous-space stochastic process encountered in the spatial data analysis or any other stochastic process indexed by the continuum. This leads to the high-dimensional IV regression with a large number of endogenous regressors.

The high-dimensional mixed-frequency IV regression features several remarkable properties. First, we show that it is possible to identify and to estimate accurately the high-dimensional slope parameter leveraging on a low-frequency instrumental variable. In contrast, the point identification in the (high-dimensional) linear IV regression typically relies on the order condition postulating that the number of instrumental variables should be at least as large as the number of endogenous regressors. Second, the mixed frequency IV regression can handle arbitrary large number of endogenous covariates relatively to the sample size without relying on approximate sparsity condition and restrictive tail conditions. Such a remarkable property is possible due to the continuous-time structure of the regressor and the slope parameter. Continuous-time structures is one of the "blessings of dimensionality" according to donoho2000high.\footnote{The continuum modeling, the concentration of measure phenomenon, and the dimension asymptotics are the three "blessings of dimensionality", according to donoho2000high.} These properties distinguish our model from the ridge IV regression, cf., carrasco2012regularization or the high-dimensional IV regression of belloni2012sparse.

The high-dimensional mixed-frequency IV regression is an example of ill-posed inverse problem in the sense that the map from the distribution of the data to the slope parameter is not continuous. As a result, we need to introduce some amount of regularization\footnote{The concept of regularization originates from the mathematical literature on ill-posed inverse problems, cf., tikhonov1943stability and tikhonov1963regularization, see carrasco2007linear for a review and further references in econometrics.} to smooth out the discontinuities and to obtain a consistent estimator. In this paper, we focus on the Tikhonov regularization and establish its statistical properties with weakly dependent data. The estimation accuracy of the continuous-time slope parameter depends both on its regularity as well as on the regularity of a certain integral operator.

Our empirical application extends the classical IV estimation of the supply and the demand equations, cf., wright1928tariff, to the real-time spot markets. We collect a new dataset using publicly available data and estimate the real-time price elasticity of supply in the Australian electricity spot market. To that end, we leverage on the daily temperature as an instrumental variable that shifts the demand curve and is exogenous for supply shocks. The temperature is a valid instrumental variable since the electricity demand increases in hot and cold times due to cooling and heating needs. Our empirical results reveal that while the supply of the electricity is relatively inelastic, its elasticity is heterogeneous across the day, peaking around 6 pm and dropping subsequently to its lowest value around 4 am.

\paragraph{Contribution and related literature.} Our paper connects several strands of the literature. First, following ghysels2004midas, ghysels2007midas, and andreou2010regression, there is an increasing interest in using datasets sampled at different frequencies in the empirical practice. Most of this literature, with a notable exception for ghysels2010midas and khalaf2017dynamic, is largely focused on the forecasting problem with mixed-frequency data and does not consider the structural econometric modeling with the instrumental variable approach. The mixed-frequency data typically lead to high-dimensional problems and the dimensionality is controlled using tightly parametrized weight functions, see also foroni2015unrestricted for the unrestricted mixed-frequency data models. Our paper has the following features: 1) we introduce a novel instrumental variable regression suitable for the data sampled at mixed frequencies and the structural econometric modeling; 2) we do not rely on a particular parameterization of the weight function; 3) our high-frequency data are generated from the endogenous continuous-time stochastic process and we study the in-fill asymptotics.

Second, we build on insights from literature on the Tikhonov regularization of ill-posed inverse problems in econometrics, see carrasco2007linear, gagliardini2012tikhonov, and carrasco2013asymptotic for comprehensive surveys, and the functional linear IV regression, see florens2015instrumental, benatia2017functional, and babii2020a. In contrast to this literature, we show that it is possible to achieve identification and to estimate accurately the slope parameter relying on a single instrumental measured at a low-frequency only. The structure of our model is also qualitatively different and leads to the conditional expectation operator that was not previously encountered in the ill-posed inverse problems literature.

Lastly, following the influential work of belloni2011lasso, belloni2012sparse, and belloni2014inference, there is an increasing interest in the estimation and inference with high-dimensional datasets in econometrics.\footnote{The literature on approximately sparse econometric models is vast, see belloni2018high for an excellent introduction and further references.} In particular, belloni2012sparse propose to use the LASSO to address the problem of many instruments and the nonparametric series estimation of the optimal instrument. Our mixed-frequency IV regression is qualitatively different from the above models and does not impose the approximate sparsity on the high-dimensional slope coefficients, see babii2019estimation for a comprehensive treatment of approximately sparse mixed-frequency time series regressions. The problem of the optimal instrument is more challenging in our nonparametric setting and is left for future research, see florens2015there for some steps in this direction.

The paper is organized as follows. In section (ref), we present the mixed-frequency IV regression, illustrate several economic examples, and discuss the main identification issues. In section (ref), we present the Tikhonov-regularized estimator and derive its statistical properties for the weakly dependent time series data. All technical details appear in the appendix. We report on a Monte Carlo study in section (ref) which provides further insights about the validity of asymptotic analysis in finite samples typically encountered in empirical applications. Section (ref) presents an empirical application to the estimation of real-time supply elasticities. Lastly, section (ref) concludes.

Mixed-frequency IV regression

The purpose of this section is to introduce the mixed-frequency IV regression and to discuss our identification and estimation strategies.

The model

Econometrician observes $\{(Y_t,Z_t(s_j),W_t):\; t=1,\dots,T,j=0,1,\dots,m \}$, where $Y_t\in\ensuremath{\mathbf{R}}$ is a low-frequency dependent variable, $Z_t(s_j)$ is a realization of a real-valued continuous-time stochastic process $Z_t = \left\{Z_t(s):\;s\in S\subset\ensuremath{\mathbf{R}}^d\right\}$, and $W_t\in\ensuremath{\mathbf{R}}^q$ is a (vector of) low-frequency instrumental variables.\footnote{If $d=1$, then $S\subset \ensuremath{\mathbf{R}}$ can be interpreted as a time index. More generally, if $d=2$, then $S\subset\ensuremath{\mathbf{R}}^2$ can be a geographical location (spatial process), and if $d=3$, then $S\subset\ensuremath{\mathbf{R}}^3$ can denote both the space and the time dimension (spatio-temporal process). Regardless of the dimension $d$, we always refer to $Z_t$ as a continuous-time stochastic process.} The number of high-frequency observations $m$ is left unrestricted and can (potentially) be much larger than the sample size $T$. The mixed-frequency IV regression is described as

equation*[equation* omitted — 140 chars of source]

Note that since the regressor is sampled from a high-frequency covariate $Z_t$,

Discretizing the continuous-time equation, we obtain

equation*[equation* omitted — 143 chars of source]

It is worth stressing that the discretization of the continuous-time model leads to a consistent definition of regression slopes across different frequencies, cf., sims1971discrete and geweke1978temporal. In contrast, the naive discrete-time regression equation does not impose any normalization and the magnitude of the slope parameter is different across different frequencies.

The following three examples provide several empirical settings where our mixed-frequency IV regression model could be useful.

example[Real-time price elasticities] Spot markets operate in real-time with commodities traded for immediate delivery. The mixed-frequency IV regression can be used to estimate the real-time elasticities of supply/demand, which is a continuous-time extension of the classical linear IV regression, cf., wright1928tariff. In our empirical application, $Y_t$ is the quantity sold at the spot market on a day $t$ and $Z_{t}(s)$ is the equilibrium market price on a day $t$ at time $s$. The market equilibrium leads to the endogeneity problem. Using daily temperatures as a demand shifter, we can identify the real-time price elasticity $\beta$ of the electricity supply.

example[Intraday liquidity] In the equilibrium of a seminal kyle1985continuous model, $Y_t$ is a daily price change of an asset $t$, $Z_t(s)$ is an order flow imbalance on a day $t$ at time $s$, and $1/\beta$ is a liquidity parameter. The liquidity parameter quantifies the sensitivity of the market price to the imbalance between the supply and the demand. Endogeneity comes from the strategic behavior of informed traders who are likely to distribute orders over time to minimize the impact on prices and the market equilibrium.

example[Measurement errors] Classical measurement errors in the high-frequency regressor sampled from a continuous-time stochastic process also lead to the endogeneity problem. Such measurement errors are especially pronounced in the high-frequency intraday financial data contaminated by the market microstructure noise, see zhang2005tale and hansen2006realized.

Identification

To simplify the notation, in this section, we suppress the dependence of $(Y_t,Z_t,W_t)$ on $t$ and write $(Y,Z,W)$, which is well-justified under stationarity. The mixed-frequency IV regression becomes\footnote{Alternatively, if we start from the linear model $Y=\Phi(Z)+U$, where $\Phi:L_2(S)\to \ensuremath{\mathbf{R}}$ is a continuous linear functional, then by the Riesz representation theorem, we can always write $\Phi(Z) = \langle \beta,Z\rangle$ for a unique slope parameter $\beta\in L_2(S)$. Here and later, $L_2(S)$ denotes the set of real functions on $S$, square-integrable with respect to the Lebesgue measure and the natural inner product $\langle.,.\rangle$, see Appendix for more details on the notation.}

equation*[equation* omitted — 110 chars of source]

The identification in the linear IV regression relies on the uncorrelatedness between the instrumental variable and the unobservables, i.e., $\ensuremath{\mathds{E}}[UW]=0$, and the rank condition. The rank condition requires in turn that the number of the instrumental variable matches the dimension of the endogenous covariate. In our settings, the endogenous covariate is a high-dimensional realization of a continuous-time stochastic process, which requires in turn a high-dimensional instrumental variable. Given that the instrumental variable has to be exogenous to the system, this imposes a strong requirement on the instrumental variable.

In contrast, our identification strategy relies on the mean independence exogeneity condition, $\ensuremath{\mathds{E}}[U|W]=0$. Assuming that the order of the integration can be interchanged, the exogeneity leads to

equation[equation omitted — 178 chars of source]

where $L:L_2(S)\to L_2(W)$ is an integral operator mapping the unknown slope parameter $\beta$ to the conditional mean function $h$.\footnote{For a random variable $W$, we denote $L_2(W) = \{f:\; \ensuremath{\mathds{E}}|f(W)|^2<\infty \}$ with some abuse of notation.} Eq. (ref) is an example of the Fredholm integral equation of type I solving, which is typically known to be ill-posed in the sense that the inverse map from $h$ to $\beta$ is discontinuous, see carrasco2007linear.

Our identification strategy relies on the linear completeness property of the distribution of $(Z,W)$. We say that the stochastic process $Z\in L_2(S)$ is linearly complete\footnote{The linear completeness condition is significantly weaker than the nonlinear completeness condition typically used in the nonparametric IV literature, cf., babii2017completeness.} for $W\in\ensuremath{\mathbf{R}}^q$ if for all $b\in L_2(S)$ with $\ensuremath{\mathds{E}}|\langle Z,b\rangle|<\infty$, we have

equation*[equation* omitted — 93 chars of source]

assumptionThe stochastic process $Z$ is linearly complete for $W$.

The linear completeness is a generalization of the rank condition imposed in the finite-dimensional linear IV regression and requires that the operator $L$ is injective. Consider another injective operator $M:L_2(W)\to L_2(S)$ such that $(M\varphi)(u) = \ensuremath{\mathds{E}}[\varphi(W)\Psi(u,W)]$ for some square-integrable function of the instrumental variable $\Psi$. Applying $M$ to both sides of Eq. (ref) leads to

equation[equation omitted — 176 chars of source]

where $r=Mh$ and $K=ML$ is a new operator. It is more convenient to estimate the slope parameter $\beta$ using the continuum of moment restrictions in Eq. (ref), since it does not involve conditional expectations, nonparametric estimation of which involves additional tuning parameters.\footnote{The problem of estimating a finite-dimensional parameter using a continuum of moment conditions is addressed, e.g., in carrasco2000generalization and carrasco2007efficient.} At the same time, Eq. (ref) has the same identifying power as Eq. (ref) provided that the operator $M$ is injective. A large class of instrument functions $\Psi$ that ensure injectivity of $M$ is characterized in stinchcombe1998consistent.\footnote{See also an earlier work of bierens1982consistent who develops consistent specification tests and the work of dominguez2004consistent and lavergne2013smooth who develop estimators of finite-dimensional parameters based on the Bierens-type trick.} Our default recommendation is the logistic CDF, $\Psi(u,W) = \frac{1}{1 + e^{-u^\top W}}$, which real-valued and bounded.

Tikhonov regularization

In this section, we introduce the Tikhonov-regularized estimator of the slope parameter $\beta$ and study its statistical properties with time series data.

Estimator

Our objective is to estimate the slope parameter $\beta$ using the continuum of moment conditions in Eq. (ref), which requires inverting the operator $K$. Note that the integral operator $K$ has the kernel function $k(s,u)\triangleq \ensuremath{\mathds{E}}[Z(s)\Psi(u,W)]$, which is typically square-integrable. Consequently, the operator $K$ is compact and its generalized inverse is not continuous, see carrasco2007linear. The operator inversion problem is amplified by the fact that $r$ and $K$ are unobserved and have to be estimated from the data. In this paper, we focus on the Tikhonov-regularized estimator of $\beta$.

Let $(Y_t,Z_t,W_t)_{t=1}^T$ be a stationary sample. The operator $K$ and the function $r$ are estimated using sample means

equation*[equation* omitted — 239 chars of source]

The Tikhonov-regularized estimator solves the following penalized least-squares problem

equation*[equation* omitted — 111 chars of source]

where $\alpha>0$ is a tuning parameter controlling the amount of the regularization and $\|.\|$ is the natural norm on the relevant $L_2$ space. The estimator has a well-known closed-form expression, which resembles the expression of the finite-dimensional ridge regression estimator\footnote{It is well-known that the compact self-adjoint operator has a countable, decreasing to zero sequence of eigenvalues. Tikhonov regularization stabilizes the spectrum of the generalized inverse of the operator $\hat K^*\hat K$, replacing its eigenvalues $\frac{1}{\hat\lambda_j}$ by $\frac{1}{\alpha + \hat\lambda_j}$, see carrasco2007linear for more details.}

equation[equation omitted — 96 chars of source]

where $\hat K^*$ is the adjoint operator to $\hat K$. To compute the adjoint operator, note that for every $\psi\in L_2$, by Fubini's theorem

equation*[equation* omitted — 331 chars of source]

Therefore, the adjoint operator is

equation*[equation* omitted — 86 chars of source]

Statistical properties

To investigate the statistical properties of $\hat\beta$, we introduce several weak-dependence conditions on the underlying stochastic processes. The following definition generalizes the notion of the covariance stationarity to function-valued stochastic processes, see bosq2012linear for a comprehensive introduction to the statistical theory of stochastic processes in Hilbert and Banach spaces.

definitionThe $L_2(S)$-valued stochastic process $(X_t)_{t\in\ensuremath{\mathbf{Z}}}$ is covariance stationary if \begin{enumerate} • the second moment exists: $\sup_{t\in\ensuremath{\mathbf{Z}}}\ensuremath{\mathds{E}}\|X_t\|^2<\infty$; • the mean function is constant over time: $\ensuremath{\mathds{E}}[X_t(s)] = \mu(s),\forall s\in S$ and $\forall t\in\ensuremath{\mathbf{Z}}$; • the autocovariance function depends only on the distance between observations: $\forall s,u\in S$ and $\forall h,k\in\mathbf{Z}$ \begin{equation*} \begin{aligned} \gamma_{h,k}(s,u) & = \ensuremath{\mathds{E}}[(X_{h}(s) - \mu(s))(X_{k}(u) - \mu(u))] \\ & = \ensuremath{\mathds{E}}[(X_{|h-k|}(s) - \mu(s))(X_0(u)-\mu(u))], \\ & \triangleq \gamma_{h-k}(s,u). \end{aligned} \end{equation*} \end{enumerate}

We also need a notion of the absolute summability of the autocovariance function for $L_2(S)$-valued stochastic processes.

definitionThe $L_2(S)$-valued covariance stationary process $(X_t)_{t\in\ensuremath{\mathbf{Z}}}$ has the absolutely summable autocovariance function $\gamma_h$ if \begin{equation*} \sum_{h\in \ensuremath{\mathbf{Z}}}\|\gamma_h\|_1 <\infty, \end{equation*} where $\|\gamma_h\|_1 = \int_S|\gamma(s,s)|\ensuremath{\mathrm{d}} s$ denotes the $L_1$ norm on the diagonal of $S\times S$.

The following assumption restricts the dependence structure of the process.

assumption$\{u\mapsto Y_t\Psi(u,W_t):\; t\in\mathbf{Z}\}$ and $\{(s,u)\mapsto Z_t(s)\Psi(u,W_t):\; t\in\mathbf{Z}\}$ are covariance stationary $L_2$-valued stochastic processes with absolutely summable autocovariance functions.

Assumption (i) is a relatively mild condition and is satisfied, in particular, when $(Y_t,Z_t,W_t)_{t\in\mathbf{Z}}$ is strictly stationary. The absolute summability of autocovariances is also a relatively mild condition that is typically assumed in the time series analysis. It is worth stressing that the stationarity is imposed on entire trajectories of the processes over $t\in\ensuremath{\mathbf{Z}}$. At the same time, on a fixed day $t\in\ensuremath{\mathbf{Z}}$, the intraday observations $Z_t(s)$ for $s\in S$ can be non-stationary.

Since the mixed-frequency IV regression model is ill-posed, we also need to quantify the degree of ill-posedness of the operator $K$ and the regularity of the slope parameter $\beta$. The following conditions serve this purpose.

assumptionThe slope parameter $\beta$ belongs to the class \begin{equation*} \mathcal{F}(\gamma,R) = \left\{b \in L_2(S):\; b = (K^*K)^{\gamma}\psi,\;\|\psi\|^2\leq R\right\} \end{equation*} for some $\gamma\in(0,1]$ and $R>0$.

To appreciate this condition, note that if $\beta = (K^*K)^{\gamma}\psi$, then $\psi = (K^*K)^{-\gamma}\beta$. Let $(\sigma_j,\beta_j,\psi_j)_{j=1}^\infty$ be the singular values decomposition of the compact linear operator $K$, see carrasco2007linear. Then $\beta = \sum_{j=1}^\infty\langle\beta,\beta_j\rangle\beta_j$ and by the Parseval's identity

equation*[equation* omitted — 107 chars of source]

Therefore, $\beta = (K^*K)^\gamma \psi$ and $\|\psi\|^2\leq R$ in Assumption (ref) restrict the regularity of the slope parameter $\beta$ as measured by how fast the Fourier coefficients $(\langle\beta,\beta_j\rangle)_{j=1}^\infty$ decrease to zero relatively to the smoothing properties of the operator $K$ as measured by how fast the singular values $(\sigma_j)_{j=1}^\infty$ decrease to zero and the regularity parameter $\gamma>0$.

The following result provides statistical guarantees on the estimation accuracy for the Tikhonov-regularized estimator in the mean-integrated squared error.

theoremSuppose that Assumptions (ref), (ref), and (ref) are satisfied. Then \begin{equation*} \ensuremath{\mathds{E}}\left\|\hat \beta - \beta\right\|^2 \leq C\left(\frac{1}{\alpha T} + \frac{\alpha^{2\gamma} + \alpha^{2\gamma\wedge 1}}{\alpha T} + \alpha^{2\gamma}\right), \end{equation*} where the constant $C$ can be found in Eq. (ref).

Consequently, if the regularization parameter $\alpha$ tends to zero, we obtain

equation*[equation* omitted — 120 chars of source]

The two terms are balanced for $\alpha\sim T^{-\frac{1}{2\gamma + 1}}$, in which case the convergence rate of the integrated MSE is $O\left(T^{-\frac{2\gamma}{2\gamma + 1}}\right)$. The uniform inference for the Tikhonov-regularized estimator is also possible, cf., babii2020a. Lastly, one could also consider regularization with Sobolev norm penalty and/or more general spectral regularization schemes, see carrasco2007linear, carrasco2013asymptotic, babii2017completeness, and babii2020b.

Infill asymptotics

So far we have assumed that the trajectory of the stochastic process $\{Z_t(s):\; t=1,\dots,T,s\in S\}$ is completely observed. In this section, we relax this requirement and investigate the case when we only observe $\{Z_t(s_j):\; t=1,\dots,T,\;j=1,\dots,m\}$, i.e., realizations of the process at discrete time points $s_j\in S,j=1,\dots,m$. For simplicity of presentation, suppose that $S=[0,1]$ and that $0=s_0\leq s_1< s_2 <\dots< s_m=1$.

Then the operator

equation*[equation* omitted — 180 chars of source]

is not accessible in practice the continuous-time stochastic process $Z_t$ is only partially observed. Instead, we observe its discrete-time approximation for every $\phi\in C[0,1]$

equation*[equation* omitted — 80 chars of source]

with $\delta_j = s_j - s_{j-1}$. Let $\Delta_m \triangleq \max_{1\leq j\leq m}\delta_j$ and let $\hat\beta_m$ be the solution to

equation*[equation* omitted — 76 chars of source]

For the in-fill asymptotics, we need additionally the following assumption.

assumption(i) The process $Z$ has trajectories in the H\"{o}lder class $C_L^\kappa[0,1]$ for some $\kappa\in(0,1)$ and $L\in(0,\infty)$; (ii) $\sup_{w}\|\Psi(.,w)\|^2\leq \bar\Psi<\infty$; (iii) $\Delta_m = O(\alpha^{1/\kappa}/T^{1/2\kappa})$ as $\alpha\to 0$ and $T\to\infty$.

Assumption (ref) (i) is satisfied, e.g., for the Brownian motion on $[0,1]$ with $\kappa<1/2$. (ii) is satisfied, e.g., for uniformly bounded instrument functions on compact intervals. (iii) imposes restrictions on the in-fill asymptotics. In the special case of the uniform spacing, it reduces to the condition $m^{-\kappa} = O(\alpha/T^2)$. In other words, the number of regressors should increase sufficiently fast. It is worth stressing that the number of regressors $m$ can be much larger than the sample size $T$ and can increase even faster than exponentially.

The following result shows that the integrated MSE can converge at the same rate as if we observed the process, cf., Theorem (ref).

theoremSuppose that Assumptions (ref), (ref), (ref), and (ref) are satisfied. Then \begin{equation*} \ensuremath{\mathds{E}}\left\|\hat \beta_m - \beta\right\|^2 \leq O\left(\frac{1}{\alpha T} + \alpha^{2\gamma}\right) \end{equation*} for some constants $C<\infty$.

Monte Carlo experiments

In this section, we discuss the numerical implementation of our high-dimensional mixed-frequency IV estimator and study its behavior in finite samples with Monte Carlo experiments.

We use the logistic CDF, $\Psi(u,W) = \frac{1}{1+\exp(-u^\top W)}$, as an instrument function.\footnote{This function fits our assumptions since it is uniformly bounded and real-valued, unlike some other choices, cf., bierens1982consistent and stinchcombe1998consistent. At the same time, we find in Monte Carlo experiments that it works significantly better than, e.g., $\Psi(s,w)=\ensuremath{\mathds{1}}\{w\leq s \}$.} We rewrite Eq. (ref) as $\alpha \hat{\beta} + \hat{K}^*\hat{K}\hat\beta = \hat{K}^*\hat r$ and discretize it with the Riemann sum on a grid of uniformly spaced points $j/m,j=1,\dots,m$. The discretized equation is

equation*[equation* omitted — 217 chars of source]

where $\boldsymbol{\hat\beta}= (\hat{\beta}(j/m))_{1\leq j\leq m}, \mathbf{Z} = (Z_t(j/m))_{\substack{1\leq t\leq T \\ 1\leq j\leq m}},\boldsymbol{\Psi} = (\Psi(j/m,W_t))_{\substack{1\leq j\leq m \\ 1\leq t\leq T}},\mathbf{y} = (Y_t)_{1\leq t\leq T}$ and $\mathbf{I}_T$ is a $T\times T$ identity matrix. Then we compute the estimator as

equation*[equation* omitted — 225 chars of source]

There are $5,000$ replications in each Monte Carlo experiment. We generate samples of $(Y_t,Z_t,W_t)_{t=1}^T$ of size $T\in\{100,500,1000\}$ as follows

equation*[equation* omitted — 357 chars of source]

where $\{B_t(s): s\in[0,1],t=1,\dots,T \}$ are independent Brownian motion, generated independently of all other variables and initiated at i.i.d. random draws from $U(-1/2,1/2)$. The parameter $\sigma\in\{0.5,1\}$ represents the noise level. We consider two slope parameters $\beta(s)=-10\exp(s)$ and $\beta(s)=10s$ with $s\in[0,1]$. All continuous-time quantities are discretized at $200$ equidistant points.

The integrated bias, variance, and MSE are approximated by the Riemann sum on a grid of $100$ equidistant points in $[0,1]$. Table (ref) and Table (ref) present the results of our Monte Carlo experiments for two different population slope parameters. The mixed-frequency IV estimator behaves according to our asymptotic results. We can see the bias/variance trade-off -- as the regularization parameter $\alpha$ tends to zero, the bias decreases while the variance increases. The optimal choice of the regularization parameter should balance the two. The estimator performs better when the sample size increases and the noise level decreases. We can also see that the linear slope parameter is estimated more accurately. Figure (ref) and Figure (ref) summarize graphically the outcome of Monte Carlo experiments for $\alpha = 10^{-6}$. The shaded gray area represents the pointwise $95\%$ confidence interval across $5,000$ replications. Overall, the mixed-frequency IV estimator demonstrates excellent performance across different specifications.

It is worth stressing that since the stochastic $Z$ is observed at $m=200$ time points, the number of endogenous regressors exceeds the sample size when $T=100$. In this case, the conventional IV estimator does not exist. At the same time, the naive generalization of the ridge regression and the LASSO are also not appropriate in our setting. The ridge regression would typically require $m/T\to 0$, cf., carrasco2007linear. The LASSO would require the approximate sparsity, somewhat stronger weak dependence conditions, and $m^{1/\kappa}/T^{1-1/\kappa}\to 0$, where $\kappa$ measures tails and weak dependence, cf., babii2019estimation.

table[table omitted — 1,467 chars of source]

table[table omitted — 1,457 chars of source]

figure[figure omitted — 977 chars of source]

figure[figure omitted — 970 chars of source]

Real time elasticity of electricity supply

At the beginning of the 90s, electricity markets around the world were vertically integrated industries with prices set by regulators. Over the last 30 years, major countries experienced deregulation. Today, electricity is often sold at competitive spot markets where prices are determined according to the laws of supply and demand. Elasticities of supply and demand summarize the behavior of energy producers and consumers, inform market participants, and play an important role in the policy design, forecasting, and energy planning. The real-time elasticity of supply contains a piece of important information on seller's response to the intraday price fluctuations.\footnote{While there is an extensive literature on forecasting with intraday electricity data, see, e.g., aneiros2013functional and references therein, the structural econometric analysis of the real-time electricity data received less attention, see benatia2017functional and benatia2018functional for notable exceptions. The latter paper studies the multi-unit electricity auction in New York and estimates the firm-level market power. It is also worth mentioning that the real-time price elasticities of demand have been previously estimated in patrick2001estimating and lijesen2007real relying on a different econometric methodology.}

Most of the electricity in Australia is generated, sold, and bought at the National Electricity Market (NEM), which is one of the largest interconnected electricity systems in the world. The NEM started operating as a wholesale spot market in December 1998. It supplies about 200 terawatt-hours of electricity to around 9 million customers each year reaching \$16.6 billion of trades in 2016-2017. The supply and the demand come from over 100 competitive generators and retailers participating in the market and are matched instantaneously in real time through a centrally coordinated dispatch process. Generators offer to supply a fixed amount of electricity at a specific time in the future and can resubmit subsequently the offered amount and price if needed. The Australian Energy Market Operator (AMEO) decides which generators will produce electricity to meet the demand in the most cost-efficient way.

We construct a new dataset using publicly available data from the AEMO and the Australian Bureau of Meteorology for the New South Wales in 1999-2018. The central pieces of the dataset are the daily aggregate quantities of the electricity sold at the spot market, intraday high-frequency prices measured each half an hour, and the average daily temperatures. The high-dimensional mixed-frequency IV regression model is

equation*[equation* omitted — 137 chars of source]

where $Q_t$ is the quantity sold on a day $t$, $P_t(s)$ is the price at time $s$ on a day $t$, and $W_t$ is an instrumental variable. To estimate the supply elasticity, we use the average daily temperature. Since the observed prices are measured with half an hour intervals, the regression equation is discretized as

equation*[equation* omitted — 79 chars of source]

where $s_j = 0.5j$ with $j=1,2,\dots,48$.

Figure (ref) displays the histogram of the natural logarithm of equilibrium quantities and the boxplot with equilibrium prices plotted against the hour. Figure (ref) displays the histogram of the temperature and the scatterplot with quantities plotted against the temperature. Marginal distributions seem to be well-behaved. The price series seems to be not stationary during the day with the median price peaking in the evening and plummeting during the night. There is also more volatility in the price in the evening. Note that such our assumptions do not rule out intraday nonstationarities. Figure (ref) (b) illustrates that the quantity sold is driven by heating and cooling demands. It is worth stressing that the temperature series is available at the daily frequency which is not allowed in florens2015instrumental. At the same time, our mixed-frequency IV regression model allows to instrument intraday prices with the daily temperature series.

figure[figure omitted — 385 chars of source]

figure[figure omitted — 350 chars of source]

To compute the estimator, we estimate the regularization parameter using a method similar to the one used in feve2010practice. The method consists of minimizing the approximately scaled $L_2$ norm of the residual of the inverse problem

equation*[equation* omitted — 81 chars of source]

where $\hat\beta_\alpha = (\alpha I + \hat K^*\hat K)^{-1}\hat K^*\hat g$. In our case, the minimum is reached at $\alpha^*=2.16\times 10^{-3}$ as be seen from Figure (ref), panel (a).

figure[figure omitted — 385 chars of source]

Figure (ref), panel (b) displays the estimated intraday elasticity of supply using our high-dimensional mixed-frequency IV regression. We find that depending on the hour, the price elasticity of supply ranges between 0.135 and 0.165. The elasticity is the highest in the evening, around 6 pm and the lowest during the night. The supply appears to have the real-time price elasticity of a similar order of magnitude as the demand.\footnote{patrick2001estimating find real-time demand elasticities between 0 and -0.27 for 5 industrial sectors in UK.} The relatively inelastic supply may probably be attributed to the fact that the market participants are allowed to hedge financial risks and the difficulty to adjust the electricity production in real time.

Conclusions

This paper introduces a novel high-dimensional mixed-frequency IV regression and contributes to the growing literature on high-dimensional and mixed-frequency data. We show that the slope parameter of the high-dimensional endogenous regressor can be identified and accurately estimated leveraging on an instrumental variable observed at a low-frequency only.

We characterize the identifying condition in the model and study the statistical properties of the Tikhonov-regularized estimator with time series data. The mixed-frequency IV estimator has a closed-form expression and is easy and fast to compute numerically. Our statistical analysis does not restrict the number of high-frequency observations of the process and can handle the number of covariates increasing with the sample size even faster than exponentially.

In our empirical application, we estimate the real-time price elasticity of supply at the Australian electricity spot market. We find that the supply is relatively inelastic and that its elasticity is heterogeneous throughout the day. To conclude, we note that our identification strategy with a low-frequency IV can also be applied to the instrumental variable model of benatia2017functional with a high-frequency dependent variable.

thebibliography{42} \expandafter\ifx\csname urlstyle\endcsname\relax \else \fi \bibitem[Andreou et al.(2010)Andreou, Ghysels, and Kourtellos]{andreou2010regression} Elena Andreou, Eric Ghysels, and Andros Kourtellos. \newblock Regression models with mixed sampling frequencies. \newblock Journal of Econometrics, 158\penalty0 (2):\penalty0 246--261, 2010. \bibitem[Aneiros P{\'e}rez et al.(2013)Aneiros P{\'e}rez, Vilar-Fern{\'a}ndez, Cao Abad, and Mu{\ n}oz San Roque]{aneiros2013functional} Germ{\'a}n Aneiros, Juan M. Vilar, Ricardo Cao, and Antonio Mu{\ n}oz San Roque. \newblock Functional prediction for the residual demand in electricity spot markets. \newblock IEEE Transactions on Power Systems, 28\penalty0 (4):\penalty0 4201--4208, 2013. \bibitem[Babii(2020a)]{babii2020a} Andrii Babii. \newblock Honest confidence sets in nonparametric iv regression and other ill-posed models. \newblock Econometric Theory (forthcoming), 2020. \bibitem[Babii(2020b)]{babii2020b} Andrii Babii. \newblock Are unobservables separable? \newblock UNC Working paper, 2020. \bibitem[Babii and Florens(2018)]{babii2017completeness} Andrii Babii and Jean-Pierre Florens. \newblock Is completeness necessary? Estimation and inference in non-identified models. \newblock UNC Working Paper, 2018. \bibitem[Babii et al.(2019)Babii, Ghysels, and Striaukas]{babii2019estimation} Andrii Babii, Eric Ghysels, and Jonas Striaukas. \newblock Estimation and HAC-based inference for machine learning time series regressions. \newblock UNC Working Paper, 2019. \bibitem[Belloni et al.(2011)Belloni, Chernozhukov, and Hansen]{belloni2011lasso} Alexandre Belloni, Victor Chernozhukov, and Christian Hansen. \newblock Lasso methods for gaussian instrumental variables models. \newblock \emph{MIT Department of Economics Working Paper}, 2011. \bibitem[Belloni et al.(2012)Belloni, Chen, Chernozhukov, and Hansen]{belloni2012sparse} Alexandre Belloni, Daniel Chen, Victor Chernozhukov, and Christian Hansen. \newblock Sparse models and methods for optimal instruments with an application to eminent domain. \newblock \emph{Econometrica}, 80\penalty0 (6):\penalty0 2369--2429, 2012. \bibitem[Belloni et al.(2014)Belloni, Chernozhukov, and Hansen]{belloni2014inference} Alexandre Belloni, Victor Chernozhukov, and Christian Hansen. \newblock Inference on treatment effects after selection among high-dimensional controls. \newblock \emph{The Review of Economic Studies}, 81\penalty0 (2):\penalty0 608--650, 2014. \bibitem[Belloni et al.(2018)Belloni, Chernozhukov, Chetverikov, Hansen, and Kato]{belloni2018high} Alexandre Belloni, Victor Chernozhukov, Denis Chetverikov, Christian Hansen, and Kengo Kato. \newblock High-dimensional econometrics and regularized gmm. \newblock \emph{arXiv preprint arXiv:1806.01888}, 2018. \bibitem[Benatia(2018)]{benatia2018functional} David Benatia. \newblock Functional econometrics of multi-unit auctions: an application to the New York electricity market. \newblock CREST Working Paper, 2018. \bibitem[Benatia et al.(2017)Benatia, Carrasco, and Florens]{benatia2017functional} David Benatia, Marine Carrasco, and Jean-Pierre Florens. \newblock Functional linear regression with functional response. \newblock \emph{Journal of Econometrics}, 201\penalty0 (2):\penalty0 269--291, 2017. \bibitem[Bierens(1982)]{bierens1982consistent} Herman J Bierens. \newblock Consistent model specification tests. \newblock \emph{Journal of Econometrics}, 20\penalty0 (1):\penalty0 105--134, 1982. \bibitem[Bosq(2012)]{bosq2012linear} Denis Bosq. \newblock {Linear processes in function spaces: theory and applications}, volume 149. \newblock \emph{Springer Science & Business Media}, 2000. \bibitem[Carrasco and Florens(2000)]{carrasco2000generalization} M. Carrasco and J.P. Florens. \newblock Generalization of GMM to a continuum of moment conditions. \newblock \emph{Econometric Theory}, 16\penalty0 (06):\penalty0 797--834, 2000. \bibitem[Carrasco(2012)]{carrasco2012regularization} Marine Carrasco. \newblock A regularization approach to the many instruments problem. \newblock \emph{Journal of Econometrics}, 170\penalty0 (2):\penalty0 383--398, 2012. \bibitem[Carrasco et al.(2007{a})Carrasco, Chernov, Florens, and Ghysels]{carrasco2007efficient} Marine Carrasco, Mikhail Chernov, Jean-Pierre Florens, and Eric Ghysels. \newblock Efficient estimation of general dynamic models with a continuum of moment conditions. \newblock \emph{Journal of Econometrics}, 140\penalty0 (2):\penalty0 529--573, 2007. \bibitem[Carrasco et al.(2007{b})Carrasco, Florens, and Renault]{carrasco2007linear} Marine Carrasco, Jean-Pierre Florens, and Eric Renault. \newblock Linear inverse problems in structural econometrics estimation based on spectral decomposition and regularization. \newblock \emph{Handbook of Econometrics, Vol. 6B}, 5633--5751, 2007. \bibitem[Carrasco et al.(2014)Carrasco, Florens, and Renault]{carrasco2013asymptotic} Marine Carrasco, Jean-Pierre Florens, and Eric Renault. \newblock Asymptotic normal inference in linear inverse problems. \newblock \emph{The Oxford Handbook of Applied Nonparametric and Semiparametric Econometrics and Statistics}, 2014. \bibitem[Dominguez and Lobato(2004)]{dominguez2004consistent} Manuel A. Dominguez and Ignacio N. Lobato. \newblock Consistent estimation of models defined by conditional moment restrictions. \newblock \emph{Econometrica}, 72\penalty0 (5):\penalty0 1601--1615, 2004. \bibitem[Donoho(2000)]{donoho2000high} David Donoho. \newblock High-dimensional data analysis: The curses and blessings of dimensionality. \newblock \emph{AMS math challenges lecture}, 2000. \bibitem[F{\`e}ve and Florens(2010)]{feve2010practice} Fr{\'e}d{\'e}rique F{\`e}ve and Jean-Pierre Florens. \newblock The practice of non-parametric estimation by solving inverse problems: the example of transformation models. \newblock \emph{The Econometrics Journal}, 13\penalty0 (3):\penalty0 S1--S27, 2010. \bibitem[Florens and Sokullu(2018)]{florens2015there} Jean-Pierre Florens and Senay Sokullu. \newblock Is there an optimal weighting for linear inverse problems? \newblock \emph{University of Bristol Working Papers}, 2018. \bibitem[Florens and Van Bellegem(2015)]{florens2015instrumental} Jean-Pierre Florens and S{\'e}bastien Van Bellegem. \newblock Instrumental variable estimation in functional linear models. \newblock \emph{Journal of Econometrics}, 186\penalty0 (2):\penalty0 465--476, 2015. \bibitem[Foroni et al.(2015)Foroni, Marcellino, and Schumacher]{foroni2015unrestricted} Claudia Foroni, Massimiliano Marcellino, and Christian Schumacher. \newblock Unrestricted mixed data sampling (MIDAS): MIDAS regressions with unrestricted lag polynomials. \newblock \emph{Journal of the Royal Statistical Society: Series A (Statistics in Society)}, 178\penalty0 (1):\penalty0 57--82, 2015. \bibitem[Gagliardini and Scaillet(2012)]{gagliardini2012tikhonov} Patrick Gagliardini and Olivier Scaillet. \newblock Tikhonov regularization for nonparametric instrumental variable estimators. \newblock \emph{Journal of Econometrics}, 167\penalty0 (1):\penalty0 61--75, 2012. \bibitem[Geweke(1978)]{geweke1978temporal} John Geweke. \newblock Temporal aggregation in the multiple regression model. \newblock \emph{Econometrica}, 46\penalty0 (3):\penalty0 643--661, 1978. \bibitem[Ghysels and Wright(2010)]{ghysels2010midas} Eric Ghysels and Jonathan H Wright. \newblock MIDAS instruments. \newblock UNC Working Paper, 2010. \bibitem[Ghysels et al.(2004)Ghysels, Santa-Clara, and Valkanov]{ghysels2004midas} Eric Ghysels, Pedro Santa-Clara, and Rossen Valkanov. \newblock The MIDAS touch: Mixed data sampling regression models. \newblock UNC Working Paper, 2004. \bibitem[Ghysels et al.(2007)Ghysels, Sinko, and Valkanov]{ghysels2007midas} Eric Ghysels, Arthur Sinko, and Rossen Valkanov. \newblock MIDAS regressions: Further results and new directions. \newblock \emph{Econometric Reviews}, 26\penalty0 (1):\penalty0 53--90, 2007. \bibitem[Hansen and Lunde(2006)]{hansen2006realized} Peter R Hansen and Asger Lunde. \newblock Realized variance and market microstructure noise. \newblock \emph{Journal of Business & Economic Statistics}, 24\penalty0 (2):\penalty0 127--161, 2006. \bibitem[Khalaf et al.(2017)Khalaf, Kichian, Saunders, and Voia]{khalaf2017dynamic} Lynda Khalaf, Maral Kichian, Charles Saunders, and Marcel Voia. \newblock Dynamic panels with MIDAS covariates: Nonlinearity, estimation and fit. \newblock \emph{Technical Report}, 2017. \bibitem[Kyle(1985)]{kyle1985continuous} Albert S. Kyle. \newblock Continuous auctions and insider trading. \newblock \emph{Econometrica}, 53\penalty0 (6):\penalty0 1315--1335, 1985. \bibitem[Lavergne and Patilea(2013)]{lavergne2013smooth} Pascal Lavergne and Valentin Patilea. \newblock Smooth minimum distance estimation and testing with conditional estimating equations: uniform in bandwidth theory. \newblock \emph{Journal of Econometrics}, 177\penalty0 (1):\penalty0 47--59, 2013. \bibitem[Lijesen(2007)]{lijesen2007real} Mark G. Lijesen. \newblock The real-time price elasticity of electricity. \newblock \emph{Energy Economics}, 29\penalty0 (2):\penalty0 249--258, 2007. \bibitem[Patrick and Wolak(2001)]{patrick2001estimating} Robert H. Patrick and Frank A. Wolak. \newblock Estimating the customer-level demand for electricity under real-time market prices. \newblock Technical report, National Bureau of Economic Research, 2001. \bibitem[Sims(1971)]{sims1971discrete} Christopher A. Sims. \newblock Discrete approximations to continuous time distributed lags in econometrics. \newblock \emph{Econometrica}, 39\penalty0 (3):\penalty0 545--563, 1971. \bibitem[Stinchcombe and White(1998)]{stinchcombe1998consistent} Maxwell B. Stinchcombe and Halbert White. \newblock Consistent specification testing with nuisance parameters present only under the alternative. \newblock \emph{Econometric Theory}, 14\penalty0 (03):\penalty0 295--325, 1998. \bibitem[Tikhonov(1963b)]{tikhonov1963regularization} Andrei N. Tikhonov. \newblock On the regularization of ill-posed problems (in Russian). \newblock In \emph{Doklady Akademii Nauk SSSR}, 153\penalty0 (1):\penalty0 49--52, 1963. \bibitem[Tikhonov(1963a)]{tikhonov1943stability} Andrey N. Tikhonov. \newblock On the solution of ill-posed problems and the method of regularization. \newblock In \emph{Doklady Akademii Nauk SSSR}, 151\penalty0 (3):\penalty0 501--504, 1963. \bibitem[Wright(1928)]{wright1928tariff} Philip G. Wright. \newblock Tariff on animal and vegetable oils. \newblock \emph{Macmillan Company}, New York, 1928. \bibitem[Zhang et al.(2005)Zhang, Mykland, and A{\"\i}t-Sahalia]{zhang2005tale} Lan Zhang, Per A Mykland, and Yacine A{\"\i}t-Sahalia. \newblock A tale of two time scales: determining integrated volatility with noisy high-frequency data. \newblock \emph{Journal of the American Statistical Association}, 100\penalty0 (472), 2005.

\setcounter{page}{1} \setcounter{section}{0} \setcounter{equation}{0} \setcounter{table}{0} \setcounter{figure}{0}

center[center omitted — 40 chars of source]

Proofs

\paragraph{Notation:} We use $L_2(S)$ to denote the space of functions on $S\subset \ensuremath{\mathbf{R}}^d$, square-integrable with respect to the Lebesgue measure. We endow the space $L_2(S)$ with the natural inner product $\langle \beta,\gamma\rangle=\int_{ S} \beta(s)\gamma(s)\ensuremath{\mathrm{d}} s$ and the norm $\|\beta\|=\sqrt{\langle\beta,\beta\rangle}$ for all $\beta,\gamma\in L_2(S)$. Any vector $a\in\ensuremath{\mathbf{R}}^m$ should be considered as a column-vector and can be written as $a=(a_j)_{1\leq j\leq m}$. For a bounded linear operator $K:\mathcal{E}\to\mathcal{F}$ between the two Hilbert spaces $\mathcal{E}$ and $\mathcal{F}$, let $\|K\|_{\infty} = \sup_{\|\phi\|\leq 1}\|K\phi\|$ denote its operator norm. Let $\sigma(K^*K)$ denote the spectrum of the corresponding self-adjoint operator $K^*K$. The $m\times T$ matrix $A$ is written by enumerating all its elements $A=(A_{j,t})_{\substack{1\leq j\leq m\\1\leq t\leq T}}$. If $m=T$, then we simply write $A = (A_{ij})_{1\leq i,j\leq m,}$. We use

equation*[equation* omitted — 291 chars of source]

to denote the space of H\"{o}lder continuous functions with common parameters $\kappa,L>0$. Lastly, for $a,b\in\ensuremath{\mathbf{R}}$, put $a\vee b = \max\{a,b\}$.

To prove Theorem (ref), we need two auxiliary lemmas. The first lemma bounds the expected norm of the sample mean of a covariance stationary zero-mean $L_2(S)$-valued stochastic process $(X_t)_{t\in\ensuremath{\mathbf{Z}}}$ by the norm of its auto-covariance function $\gamma_h$.

lemmaSuppose that $(X_t)_{t\in\ensuremath{\mathbf{Z}}}$ is a zero-mean covariance stationary process in $L_2(S)$ with absolutely summable autocovariance function \begin{equation*} \sum_{h\in\mathbf{Z}} \|\gamma_h\|_1 < \infty, \end{equation*} where $\|\gamma_h\|_1=\int_S|\gamma_h(s,s)|\ensuremath{\mathrm{d}} s$. Then \begin{equation*} \ensuremath{\mathds{E}}\left\|\frac{1}{T}\sum_{t=1}^TX_t\right\|^2 \leq \frac{1}{T}\sum_{h\in\mathbf{Z}} \|\gamma_h\|_1. \end{equation*}

proofWe have \begin{equation*} \begin{aligned} \ensuremath{\mathds{E}}\left\|\frac{1}{T}\sum_{t=1}^TX_t\right\|^2 & = \frac{1}{T^2}\ensuremath{\mathds{E}}\left\langle\sum_{t=1}^TX_t,\sum_{k=1}^TX_k\right\rangle \\ & = \frac{1}{T^2}\sum_{t,k=1}^T\int_S\ensuremath{\mathds{E}}[X_t(s)X_k(s)]\ensuremath{\mathrm{d}} s \\ & = \frac{1}{T^2}\sum_{t,k=1}^T\int_S\gamma_{t-k}(s)\ensuremath{\mathrm{d}} s \\ & = \frac{1}{T}\sum_{|h|<T}\frac{T-|h|}{T}\int_S\gamma_h(s,s)\ensuremath{\mathrm{d}} s \\ & \leq \frac{1}{T}\sum_{h\in\mathbf{Z}}\int_S\left|\gamma_h(s,s)\right|\ensuremath{\mathrm{d}} s, \end{aligned} \end{equation*} where the second line follows by the bilinearity of the inner product and Fubini's theorem and the third under the covariance stationarity.

The following lemma allows controlling estimation errors appearing in the proof of Theorem (ref) in terms of more primitive quantities.

lemmaSuppose that $\hat k,k,\hat r,r,\beta$ are square-integrable. Then \begin{equation*} \ensuremath{\mathds{E}}\left\|\hat K - K\right\|^2_\infty \leq \ensuremath{\mathds{E}}\left\|\hat k - k\right\|^2 \end{equation*} and \begin{equation*} \ensuremath{\mathds{E}}\left\|\hat r - \hat K\beta\right\|^2 \leq 2\ensuremath{\mathds{E}}\left\|\hat r - r\right\|^2 + 2\|\beta\|^2\ensuremath{\mathds{E}}\left\|\hat k - k\right\|^2. \end{equation*}

proofBy the definition of the operator norm and the Cauchy-Schwartz inequality \begin{equation} \begin{aligned} \ensuremath{\mathds{E}}\left\|\hat K - K\right\|^2_\infty & = \ensuremath{\mathds{E}}\left[\sup_{\|\phi\|\leq1}\left\|\hat K\phi - K\phi\right\|^2\right] \\ & = \ensuremath{\mathds{E}}\left[\sup_{\|\phi\|\leq1}\int\left|\int\phi(s)\left(\hat k(s,u) - k(s,u)\right)\ensuremath{\mathrm{d}} s\right|^2\ensuremath{\mathrm{d}} u\right] \\ & \leq \ensuremath{\mathds{E}}\left\|\hat k - k\right\|^2. \end{aligned} \end{equation} For the second part, use $r=K\beta$, $\|a+b\|^2\leq 2\|a\|^2+2\|b\|^2$, $\|K\beta\| \leq\|K\|_\infty\|\beta\|$, and the estimate in Eq. (ref) \begin{equation*} \begin{aligned} \ensuremath{\mathds{E}}\left\|\hat r - \hat K\beta\right\|^2 & \leq 2\ensuremath{\mathds{E}}\left\|\hat r - r\right\|^2 + 2\ensuremath{\mathds{E}}\|\hat K\beta - K\beta\|^2 \\ & \leq 2\ensuremath{\mathds{E}}\left\|\hat r - r\right\|^2 + 2\|\beta\|^2\ensuremath{\mathds{E}}\left\|\hat k - k\right\|^2. \end{aligned} \end{equation*}

proof[Proof of Theorem (ref)] The proof is based on the following decomposition \begin{equation*} \hat \beta - \beta = R_1 + R_2 + R_3 + R_4 \end{equation*} with \begin{equation*} \begin{aligned} R_1 & = (\alpha I + \hat K^*\hat K)^{-1}\hat K^*(\hat r - \hat K\beta) \\ R_2 & = \alpha(\alpha I + \hat K^*\hat K)^{-1}\hat K^*(\hat K - K)(\alpha I + K^* K)^{-1}\beta \\ R_3 & = \alpha(\alpha I + \hat K^*\hat K)^{-1}(\hat K^* - K^*)K(\alpha I + K^* K)^{-1}\beta \\ R_4 & = (\alpha I + K^*K)^{-1}K^*K\beta- \beta. \end{aligned} \end{equation*} To see that this decomposition holds, note that \begin{equation*} \begin{aligned} R_2 + R_3 & = \alpha(\alpha I + \hat K^*\hat K)^{-1}\left[\hat K^*\hat K - K^*K\right](\alpha I + K^* K)^{-1}\beta \\ & = \alpha(\alpha I + \hat K^*\hat K)^{-1}\left[(\alpha I + \hat K^*\hat K) - (\alpha I + K^*K)\right](\alpha I + K^* K)^{-1}\beta \\ & = \alpha(\alpha I + K^* K)^{-1}\beta - \alpha(\alpha I + \hat K^*\hat K)^{-1} \beta \\ & = \left[I - \alpha(\alpha I + \hat K^*\hat K)^{-1}\right] \beta + \left[\alpha(\alpha I + K^* K)^{-1} - I\right]\beta\\ & = (\alpha I + \hat K^*\hat K)^{-1}\hat K^*\hat K\beta - (\alpha I + K^*K)^{-1}K^*K\beta. \end{aligned} \end{equation*} Therefore, \begin{equation*} \ensuremath{\mathds{E}}\left\|\hat \beta - \beta\right\|^2 \leq 4\ensuremath{\mathds{E}}\|R_1\|^2 + 4\ensuremath{\mathds{E}}\|R_2\|^2 + 4\ensuremath{\mathds{E}}\|R_3\|^2 + 4\ensuremath{\mathds{E}}\|R_4\|^2. \end{equation*} The fourth term is a regularization bias and its order follows directly from the Assumption (ref) and the isometry of the functional calculus \begin{equation*} \begin{aligned} \ensuremath{\mathds{E}}\|R_4\|^2 & = \left\|\left[(\alpha I + K^*K)^{-1}K^*K - I\right]\beta\right\|^2 \\ & \leq \left\|\left[I - (\alpha I + K^*K)^{-1}K^*K\right](K^*K)^{\gamma}\right\|^2_\infty R \\ & = \sup_{\lambda\in\sigma(K^*K)}\left|\left(1 - \frac{\lambda}{\alpha + \lambda}\right)\lambda^{\gamma}\right|^2R \\ & = \sup_{\lambda\in\sigma(K^*K)}\left|\frac{\lambda^{\gamma}}{\alpha + \lambda}\right|^2\alpha^2R. \end{aligned} \end{equation*} We can have two cases depending on the value of $\gamma>0$. For $\gamma\in(0,1)$, the function $\lambda\mapsto\frac{\lambda^{\gamma}}{\alpha+\lambda}$ admits maximum at $\lambda = \frac{\gamma}{1-\gamma}\alpha$. For $\gamma\geq1$, the function $\lambda\mapsto \frac{\lambda^\gamma}{\alpha + \lambda}$ is strictly increasing on $[0,\infty)$, attaining maximum at the end of the spectrum $\lambda=\|K^*K\|_\infty$. Therefore, since $\gamma^{\gamma}(1-\gamma)^{1-\gamma}\leq 1,\gamma\in(0,1)$, we have \begin{equation*} \sup_{\lambda\in\sigma(K^*K)}\frac{\lambda^{\gamma}}{\alpha + \lambda} \leq \begin{cases} \|K^*K\|^{\gamma-1}, & \gamma\geq1 \\ \alpha^{\gamma-1}, & \gamma\in(0,1). \end{cases} \end{equation*} This gives $\ensuremath{\mathds{E}}\|R_4\|^2 \leq \alpha^{2\gamma}R$ since $\gamma\in(0,1]$. Similar computations\footnote{Note that $\hat K$ is a finite-rank operator, hence, compact.} give \begin{equation*} \begin{aligned} \ensuremath{\mathds{E}}\|R_1\|^2 & \leq \ensuremath{\mathds{E}}\left\|(\alpha I + \hat K^*\hat K)^{-1}\hat K^*\right\|^2_\infty\left\|\hat r - \hat K\beta\right\|^2 \\ & = \ensuremath{\mathds{E}}\left[\sup_{\lambda\in\sigma(\hat K^*\hat K)}\left|\frac{\lambda^{1/2}}{\alpha+\lambda}\right|^2\left\|\hat r - \hat K\beta\right\|^2\right] \\ & \leq \frac{2}{\alpha}\left(\ensuremath{\mathds{E}}\left\|\hat r - r\right\|^2 + \|\beta\|^2\ensuremath{\mathds{E}}\left\|\hat k - k\right\|^2\right), \end{aligned} \end{equation*} where the last inequality follows by Lemma (ref). Next, \begin{equation*} \begin{aligned} \ensuremath{\mathds{E}}\|R_2\|^2 & \leq \ensuremath{\mathds{E}}\left\|(\alpha I + \hat K^*\hat K)^{-1}\hat K^*\right\|_\infty^2\left\|\hat K - K\right\|^2_\infty\left\|\alpha(\alpha I + K^*K)^{-1}(K^*K)^{\gamma}\right\|_\infty^2 R\\ & \leq \ensuremath{\mathds{E}}\left[\sup_{\lambda\in\sigma(\hat K^*\hat K)}\left|\frac{\lambda^{1/2}}{\alpha + \lambda}\right|^2\left\|\hat K - K\right\|^2_\infty \sup_{\lambda\in\sigma(K^*K)}\left|\frac{\lambda^{\gamma}}{\alpha + \lambda}\right|^2\alpha^2R\right]\\ & \leq \frac{\alpha^{2\gamma}}{\alpha}\ensuremath{\mathds{E}}\left\|\hat k - k\right\|^2R, \end{aligned} \end{equation*} where the last inequality follows by Lemma (ref). Likewise, for the third term, we have \begin{equation*} \begin{aligned} \ensuremath{\mathds{E}}\|R_3\|^2 & = \ensuremath{\mathds{E}}\left\|(\alpha I + \hat K^*\hat K)^{-1}(\hat K^* - K^*)\alpha K(\alpha I + K^*K)^{-1}\beta\right\|^2 \\ & \leq \ensuremath{\mathds{E}}\left\|(\alpha I + \hat K^*\hat K)^{-1}\right\|^2_\infty\left\|\hat K^* - K^*\right\|^2_\infty \left\|K(\alpha I + K^*K)^{-1}(K^*K)^{\gamma}\right\|_\infty^2\alpha^2R \\ & = \ensuremath{\mathds{E}}\left[\sup_{\lambda\in\sigma(\hat K^*\hat K)}\left|\frac{1}{\alpha + \lambda}\right|^2\left\|\hat K^* - K^*\right\|^2_\infty\sup_{\lambda\in\sigma(K^*K)}\left|\frac{\lambda^{\gamma+1/2}}{\alpha + \lambda}\right|^2\alpha^2R\right] \\ & \leq \sup_{\lambda\in\sigma(K^*K)}\left|\frac{\lambda^{\gamma+1/2}}{\alpha + \lambda}\right|^2\ensuremath{\mathds{E}}\left\|\hat K^* - K^*\right\|^2_\infty R \\ & \leq \frac{\alpha^{2\gamma\wedge 1}}{\alpha}\|K^*K\|^{(2\gamma - 1)\vee0}\ensuremath{\mathds{E}}\left\|\hat k - k\right\|^2R. \end{aligned} \end{equation*} Lastly, let $\gamma_h^{(1)}$ be the autocovariance function of the process $(Y_t\Psi(.,W_t))_{t\in\ensuremath{\mathbf{Z}}}$ and let $\gamma_h^{(2)}$ be the autocovariance function of the process $(Z_t(.),\Psi(.,W_t))_{t\in\ensuremath{\mathbf{Z}}}$. Put \begin{equation*} \eta_1 = \sum_{h\in\ensuremath{\mathbf{Z}}}\|\gamma_{h}^{(1)}\|_1 \qquad and\qquad \eta_2 = \sum_{h\in\ensuremath{\mathbf{Z}}}\|\gamma_{h}^{(2)}\|_1. \end{equation*} Then under Assumption (ref) by Lemma (ref) \begin{equation*} \begin{aligned} \ensuremath{\mathds{E}}\left\|\hat r - r\right\|^2 & = \ensuremath{\mathds{E}}\left\|\frac{1}{T}\sum_{t=1}^T\left\{Y_t\Psi(.,W_t) - \ensuremath{\mathds{E}}[Y_t\Psi(.,W_t)]\right\}\right\|^2 \\ & \leq \frac{\eta_1}{T} \end{aligned} \end{equation*} and \begin{equation*} \begin{aligned} \ensuremath{\mathds{E}}\left\|\hat k - k\right\|^2 & = \ensuremath{\mathds{E}}\left\|\frac{1}{T}\sum_{t=1}^T\left\{Z_t(.)\Psi(.,W_t) - \ensuremath{\mathds{E}}[Z(.)\Psi(.,W)]\right\}\right\|^2\\ & \leq \frac{\eta_2}{T}. \end{aligned} \end{equation*} Combining all the estimates, we obtain \begin{equation} \ensuremath{\mathds{E}}\left\|\hat \beta - \beta\right\|^2 \leq \frac{8(\eta_1 + \eta_2\|\beta\|^2)}{\alpha T} + \frac{4R\eta_2}{\alpha T}\left(\alpha^{2\gamma} + \alpha^{2\gamma\wedge 1}\|K^*K\|^{(2\gamma-1)\vee 0}\right) + 4\alpha^{2\gamma}R. \end{equation}

proof[Proof of Theorem (ref)] Decompose \begin{equation*} \hat\beta_m - \beta = \hat\beta_m - \hat\beta + \hat\beta - \beta. \end{equation*} By Theorem (ref), we know that $\ensuremath{\mathds{E}}\|\hat\beta - \beta\|^2 = O\left(\frac{1}{\alpha T} + \alpha^\gamma\right)$. Consequently, it remains to control $\ensuremath{\mathds{E}}\|\hat\beta_m - \hat\beta\|^2$. To that end, note that if $\hat\psi_m$ solves \begin{equation*} (\alpha I + \hat K_m\hat K^*)\hat\psi_m = \hat r, \end{equation*} then $\hat\beta_m = \hat K^*\hat\psi_m$. Therefore, \begin{equation*} \hat\beta_m = \hat K^*(\alpha I + \hat K_m\hat K^*)^{-1}\hat r. \end{equation*} Next, decompose \begin{equation*} \begin{aligned} \hat\beta_m - \hat \beta & = \hat K^*(\alpha I + \hat K_m\hat K^*)^{-1}\hat r - \hat K^*(\alpha I + \hat K\hat K^*)^{-1}\hat r \\ & = \hat K^*\left[(\alpha I + \hat K_m\hat K^*)^{-1} - (\alpha I + \hat K\hat K^*)^{-1}\right]\hat r \\ & = \hat K^*(\alpha I + \hat K\hat K^*)^{-1}\left[\hat K\hat K^* - \hat K_m \hat K^*\right](\alpha I + \hat K_m\hat K^*)^{-1}\hat r \\ & = \hat K^*(\alpha I + \hat K\hat K^*)^{-1}(\hat K - \hat K_m)\hat K^*(\alpha I + \hat K_m\hat K^*)^{-1}\hat r. \end{aligned} \end{equation*} Then \begin{equation*} \begin{aligned} \left\|\hat\beta_m - \hat\beta\right\|^2 & \leq \left\|\hat K^*(\alpha I + \hat K\hat K^*)^{-1}\right\|^2_\infty\left\|(\hat K - \hat K_m)\hat K^*\right\|^2_\infty\left\|(\alpha I + \hat K_m\hat K^*)^{-1}\right\|^2_\infty\|\hat r\|^2 \\ & \leq \frac{\|\hat r\|}{4\alpha^3}\left\|(\hat K - \hat K_m)\hat K^*\right\|_\infty^2. \end{aligned} \end{equation*} Next, the expression inside of the operator norm is the integral operator on $L_2$ \begin{equation*} (\hat K - \hat K_m)\hat K^*\psi = \int\psi(u)\left(\int \hat k(s,v)\hat k(s,u)\ensuremath{\mathrm{d}} s - \sum_{j=1}^m\hat k(s_j,v)\hat k(s_j,u)\delta_j\right)\ensuremath{\mathrm{d}} u. \end{equation*} Therefore, by the same computations as in Eq. (ref) and the triangle inequality \begin{equation*} \begin{aligned} \left\|(\hat K - \hat K_m)\hat K^*\right\|_\infty & \leq \left\|\int\hat k(s,.)\hat k(s,.)\ensuremath{\mathrm{d}} s - \sum_{j=1}^m\hat k(s_j,.)\hat k(s_j,.)\delta_j\right\| \\ & = \left\|\frac{1}{T^2}\sum_{t=1}^T\sum_{k=1}^T\Psi(.,W_t)\Psi(.,W_k)\left\{\int Z_t(s)Z_k(s)\ensuremath{\mathrm{d}} s - \sum_{j=1}^mZ_t(s_j)Z_k(s_j)\delta_j\right\} \right\| \\ & \leq \frac{1}{T^2}\sum_{t=1}^T\sum_{k=1}^T\|\Psi(.,W_t)\|\|\Psi(.,W_k)\|\left|\int Z_t(s)Z_k(s)\ensuremath{\mathrm{d}} s - \sum_{j=1}^mZ_t(s_j)Z_k(s_j)\delta_j\right| \\ & \leq \max_{1\leq t\leq T}\|\Psi(.,W_t)\|^2\max_{1\leq t,k\leq T}\left|\sum_{j=1}^mZ_t(s_j)Z_k(s_j)\delta_j - \int Z_t(s)Z_k(s)\ensuremath{\mathrm{d}} s\right|. \end{aligned} \end{equation*} Under Assumption (ref) (i) \begin{equation*} \begin{aligned} \left|Z_t(s_j)Z_k(s_j) - Z_t(s)Z_k(s)\right| & \leq |Z_t(s_j) - Z_t(s)||Z_k(s_j)| + |Z_t(s)||Z_k(s_j) - Z_k(s)| \\ & \leq 2L^2|s_j - s|^\kappa, \end{aligned} \end{equation*} and whence \begin{equation*} \begin{aligned} \left|\sum_{j=1}^mZ_t(s_j)Z_k(s_j)\delta_j - \int Z_t(s)Z_k(s)\ensuremath{\mathrm{d}} s\right| & = \left|\sum_{j=1}^m\int_{s_{j-1}}^{s_j}\left\{Z_t(s_j)Z_k(s_j) - Z_t(s)Z_k(s)\right\}\ensuremath{\mathrm{d}} s\right| \\ & \leq \sum_{j=1}^m\int_{s_{j-1}}^{s_j}\left|Z_t(s_j)Z_k(s_j) - Z_t(s)Z_{k}(s)\right|\ensuremath{\mathrm{d}} s \\ & = 2L^2\sum_{j=1}^m\int_{s_{j-1}}^{s_j}|s_j - s|^\kappa\ensuremath{\mathrm{d}} s \\ & \leq 2L^2\max_{1\leq j\leq m}\delta_j^\kappa. \end{aligned} \end{equation*} Therefore, \begin{equation*} \begin{aligned} \ensuremath{\mathds{E}}\left\|\hat\beta_m - \hat\beta\right\|^2 & \leq \frac{\ensuremath{\mathds{E}}\|\hat r\|}{\alpha^3}L^4\bar\Psi^2\max_{1\leq j\leq m}\delta_j^{2\kappa} \\ & = O\left(\frac{\Delta_m^{2\kappa}}{\alpha^3}\right) \\ & = O\left(\frac{1}{\alpha T}\right), \end{aligned} \end{equation*} where the second line follows under Assumption (ref) by Lemma (ref) and the last under Assumption (ref) (iii).