Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
88,287 characters · 18 sections · 47 citation commands
Benign Overfitting in Economic Forecasting via Noise Regularization
\onehalfspacing
Economic outcomes arise from complex, high-dimensional processes. Output growth, inflation, and asset returns are shaped by many interacting forces that are not directly observed. Economists have nonetheless sought parsimonious representations through variable selection or dimensionality reduction. But when predictors share latent factors, predictive information is diffuse: each variable carries only partial information, and no small subset captures what the full panel provides collectively. This appears to be the empirical reality. giannone2021economic show that economic data rarely favor sparse predictive representations, and kolesar2023fragility document the fragility of sparsity assumptions. This paper takes the denseness of economic predictive models seriously and examines its implications for the practice of forecasting. It addresses two questions: First, under what economic models can the denseness be justified? Second, what methods are appropriate, which approaches fail, and why?
In economic forecasting, observable variables are driven by a small number of latent factors representing macroeconomic, financial, or policy conditions. Because factors are unobserved, empirical practice relies on large panels of observable predictors as proxies, and estimates linear regressions as reduced-form working models. This factor structure has a fundamental implication for the reduced-form working model: the population regression coefficients are dense. Predictive information is spread across many predictors, and no small subset captures the full signal. This contrasts with the sparsity assumption underlying Lasso and related methods. Yet when the number of predictors $p$ is comparable to the sample size $n$, the high dimensionality imposes challenges to estimating the working model for forecasts. The standard response, namely variable selection by Lasso or dimension reduction by PCA stock2002macroeconomic, BN02, rests on the belief that eliminating noise improves out-of-sample accuracy. This belief can be misleading. Reliable factor estimation requires sufficiently strong eigenvalue separation. When factors are weak, or when many predictors dilute factor strength across the full panel, determining the number of factors and recovering their span poses significant challenges onatski2012asymptotics, carrasco2016sample.
Our paper develops a new approach that avoids estimating factors, whether strong or weak, while achieving the same first order optimality as if the true factors were used. We show that when predictors substantially outnumber observations, the design matrix self regularizes. In this regime, the Ridgeless estimator, that is, the minimum norm least squares solution computed via the Moore–Penrose pseudoinverse, benefits from implicit regularization. Many predictors collectively dilute each variable’s influence on the fitted values, driving out-of-sample variance toward zero, a manifestation known as benign overfitting belkin2019reconciling,mei2019generalization,hastie2022surprises. Variable selection, however, disrupts this mechanism. Even perfect selection keeps the predictor-to-observation ratio near one, where self-regularization fails. The failure is not about omitted signal, but a consequence of insufficient predictor dimension. The empirical relevance is immediate: for example, the FRED-MD dataset offers roughly 130 macroeconomic predictors for rolling estimation windows of approximately 120 monthly observations mccracken2016fred, so the dimension to sample size ratio is close to one, a regime in which the out of sample variance becomes very large.
An appropriate response is to expand the feature space rather than contract it. We propose a simple forecasting approach, which we call “noise-regularization”: applying the Ridgeless regression while retaining uninformative predictors. When economic theory does not provide a sufficiently rich set of informative predictors, we artificially augment the predictor set with randomly generated noise variables independent of the data. The key insight is that benign overfitting operates through regularization of the eigenvalue spectrum rather than predictor informativeness. Even though noise variables contain no signal, they inflate the eigenvalues of the design matrix and reduce out-of-sample variance. Because the shrinkage function is concave, Jensen’s inequality implies that this random shrinkage yields a smaller shrinkage bias in finite samples.
We formalize these insights in four results in order. Let $p_0$ denote the number of informative predictors that load on the latent factors. We consider a setting in which a large number of economic variables, including the forecast target, are driven by a few latent factors. Our first result establishes a dense representation of the forecasting working model (Theorem (ref)): the population regression coefficients are dense, with their squared norm decaying as the number of important regressors increases. As dimension grows, the working model forecast converges to the oracle prediction --- the same asymptotic mean squared error (MSE) obtained using the true factors. The second result quantifies the cost of perfect variable selection (Proposition (ref)). Even if an oracle correctly identifies the $p_0$ informative predictors, applying the Ridgeless regression to them alone fails to achieve first-order optimality when $p_0/n$ converges to a positive constant. Post-Lasso, OLS on the model with perfect variable selection, or interpolation using only informative predictors all remain suboptimal.
While we acknowledge that variable selection is imperative for structural interpretation or causal inference, in this paper forecast accuracy is our focus. The third result establishes benign overfitting (Theorems (ref) and (ref)). The noise-regularization achieves the oracle MSE asymptotically, without estimating the factors or their number, accommodating weak factors and temporal dependence. When noise predictors are added deliberately and forecasts are averaged over repeated noise draws via Rao--Blackwell, the variance rate improves strictly. Because factor estimation is bypassed, the required factor strength conditions are mild, and the results apply to both cross-sectional and time-series settings.
Finally, the gains from noise regularization require expanding the predictor set with genuinely exogenous variables; otherwise, overparameterization does not universally produce benign overfitting. In particular, for the autoregressive model AR$(p)$, adding more lagged predictors does not deliver similar gains. In the fourth result, we establish an AR$(p)$ symmetry property (Proposition (ref)): for a time series of length $T$, the Ridgeless forecast from AR$(p)$ numerically coincides with that from AR$(T-p)$ for every lag order $p$. Increasing the lag order to the overfitting regime is therefore equivalent to fitting a shorter autoregression. The intuition is that each additional lag reduces the effective sample by one observation, and thus lag expansion fails to induce eigenvalue inflation.
Together, these four results identify a unified mechanism: benign overfitting in dense, factor-driven forecasting environments is achievable by expanding the feature space, not by selecting within it. Our analysis provides an economic structure for benign overfitting and extends the double descent literature belkin2019reconciling, bartlett2020benign, hastie2022surprises, lee2023mean to factor-driven environments. We show that the dense coefficient vector in the reduced-form working model arises naturally from the oracle factor structure, rather than being imposed as a technical condition. We further demonstrate that the second descent in forecast MSE reflects eigenvalue shrinkage, which can be induced by adding noise variables, a distinction particularly relevant in economic studies when $p_0/n$ is not large. Finally, whereas existing analyses typically assume bounded eigenvalues, factor models feature a few rapidly diverging spikes, requiring new technical arguments.\footnote{In a contemporaneous study, meng2025estimation analyze double descent in portfolio allocation with spiked eigenvalues. Among economic applications of overparameterization, kelly2022virtue and didisheim2023complexity document the benefits of rich nonlinear models.}
Empirically, we apply our approach to three settings: monthly U.S.\ inflation forecasting using FRED-MD mccracken2016fred, cross-country GDP growth forecasting using barro1994sources, and the U.S.\ equity risk premium forecasting Goyal2008comprehensive. In each application, we implement the proposed noise-regularization forecast by augmenting the predictor set with artificially generated noise variables. Across all settings and forecast horizons, predictive accuracy is stable or improves as noise predictors are added. For the U.S.\ inflation, noise regularization achieves the lowest predictive MSE among all methods considered. For cross-country GDP growth, the method stabilizes at an MSE near 1.0, compared with 20.24 using the original 60 predictors alone, and remains robust over a wide range of noise counts. For the equity premium, where competing methods yield negative out-of-sample $R^2$, noise-regularization achieves a positive $R^2$ of approximately 1.2%. It turns out that noise-regularization outperforms Ridge, Lasso, and PCA across these applications, and exhibits substantially lower sensitivity to the tuning parameter.
We adopt the following notation. Let $\left\lVertA\right\rVert$ denote the $\ell_2$ norm if $A$ is a vector, or the operator norm if $A $ is a matrix. For two positive sequences $a_{p,n}$ and $b_{p,n}$, we denote $a_{p,n} \ll b_{p,n}$ (or $b_{p,n} \gg a_{p,n}$) if $a_{p,n} =o(b_{p,n})$, and denote $a_{p,n} \asymp b_{p,n}$ if $a_{p,n} = O(b_{p,n})$ and $b_{p,n} = O(a_{p,n})$.
Our objective is to forecast a scalar outcome $y_t$, driven by $K$ latent factors $f_t$ in the true data generating process
where $\rho$ is a $K \times 1$ vector of factor loadings on the outcome. The model admits an intercept by setting the first component of $f_t$ to unity. In addition to the target, the economist observes a $p$-dimensional vector of predictors $x_t= (x_{1,t},\ldots,x_{p,t})'$ sharing the latent factors through
where $\Lambda = (\lambda_1, \ldots, \lambda_p)'$ is a $p\times K$ matrix of factor loadings and $u_t = (u_{1t}, \ldots, u_{pt})'$ is an idiosyncratic error. The mean independence condition $\mathbb E (u_{t}|f_t, \epsilon_{y,t})=0$ ensures that all predictive power of $x_t$ for $y_t$ operates through the common factors; Remark (ref) below discusses why this condition is central to the dense forecasting setting.
We allow both time series and cross-sectional forecasts. In the time series context, the primitive processes $(f_t, \epsilon_{y,t}, u_t)$ are stationary over $t=1,\ldots,n$, and $f_t$ may be dynamic, containing finitely many lags of a baseline factor vector as in SW02; for cross-sectional applications, $t$ indexes individuals rather than time periods.
Not all predictors need be informative. We partition $x_t$ into components that load on and off the factors:
where $\Lambda_I$ is a $p_0\times K$ matrix with all rows nonzero. The ordering is without loss of generality. When $p - p_0$ is large relative to $p_0$, the factors are diluted across the full predictor set and become weak in the sense of the factor-strength literature, even if they are strong within the informative block alone. Our methodology treats the identities of $x_{I,t}$ and $x_{N,t}$ as potentially unknown and does not require variable selection to screen off the noise.
Two empirical scenarios motivate ((ref)).
Scenario I: Unknown noise identities.\; Economists routinely assemble large predictor sets such as macroeconomic indicators, financial variables, survey data that are potentially correlated with the target, but economic theory may not pinpoint which variables carry the factor signal. Many collected predictors may be pure noise, diluting factor strength across the full set. The partition in (ref) is unknown to the forecaster.
Scenario II: Intentionally added noise.\; Suppose, instead, that the economist has a priori knowledge that all $p_0$ collected predictors are informative, but $p_0$ is not much larger than $n$ (formally, $p_0/n \to \gamma_0 \in (0,\infty)$). Traditional wisdom would retain only these predictors, but Proposition (ref) below shows that this regime is insufficient for first-order forecast optimality, regardless of how accurately the informative variables are identified. A central finding of this paper is that deliberately augmenting the predictor set with pure noise variables, i.e. making $p$ much larger than $n$ improves forecast accuracy through eigenvalue regularization.
Now that we have set up the data generating process, we proceed to the reduced-form procedure of supervised learning.
Given the data $(y_t, x_t)_{t=1}^n$, the linear working model
with $\beta = \mathbb E(x_t x_t')^{-1} \mathbb E(x_t y_t)$, is the standard reduced-form approach to forecasting with many predictors. Economic theory implies that many outcomes depend on low-dimensional state variables (consumption growth, productivity, or risk factors) that are not directly observed merton1973intertemporal, lucas1978asset. While no single predictor perfectly proxies the latent factors, their linear combination provides a close approximation. This suggests that the coefficient vector $\beta$ should not be sparse but dense: many small nonzero entries, rather than a few large ones.
Theorem (ref) formalizes this intuition. To state it, we introduce notation and the asymptotic regime. Let $\sigma_j(\cdot)$ denote the $j$th largest singular value of a matrix, and let $\sigma_{\min}(\cdot)$ and $\sigma_{\max}(\cdot)$ denote the smallest and largest nonzero singular values, respectively. The factor strength is indexed by a sequence $\psi_{p,n} \to \infty$. Our asymptotic framework requires $p, p_0, \psi_{p,n} \to \infty$ as $n \to \infty$, with $p_0 \asymp n$ and $n = O(p)$.
Condition (i) requires that all $K$ eigenvalues of $\Lambda_I'\Lambda_I$ grow at a common rate $\psi_{p,n}$. The homogeneous strength among all factors is imposed for the ease of presentation, and can be extended to allow heterogeneous strength among factors. Crucially, we allow $\psi_{p,n}/p_0 \to 0$: the factors may be weak even within the informative block. The divergence of $\psi_{p,n}$ is necessary to separate the factor signal from the idiosyncratic component. Conditions (ii) and (iii) are standard.
We now characterize the coefficient vector $\beta$ and the residual variance of the working model. Let $\sigma^2_{\epsilon} = \mathbb E(\epsilon^2_{y,t})$ denote the oracle forecast variance under the true factors, and $\sigma^2_{e} = \mathbb E(e^2_{t})$ the variance of the residual in (ref).
Result (i) establishes that the working model is dense: as factor strength grows, the $\ell_2$-norm of $\beta$ decays to zero. This contrasts with sparse models, in which at least one coefficient remains bounded away from zero regardless of dimension. The conventional sparse view assumes that only a small subset of predictors carries predictive power. However, this perspective has been challenged by empirical evidence, including giannone2021economic and many related studies. Perhaps a more convincing interpretation is that predictors are not collected arbitrarily, but are selected under the guidance of economic theory and empirical experience. As a result, many predictors are informative because they share exposure to latent economic factors. When such predictors are used directly in a reduced-form regression, each variable carries a small amount of signal through these common factors. As the cross-sectional dimension grows, this signal is distributed across many regressors, leading to a dense representation where $\|\beta\|\to 0$. Result (i) formalizes this intuition and is closely aligned with the empirical findings in giannone2021economic.
Result (ii) quantifies the gap between the working model (ref) and the oracle one ((ref)): the residual variances $\sigma_e^2$ and $\sigma_\epsilon^2$ coincide as $\psi_{p,n} \to \infty$. In principle, the economist can achieve the same asymptotic forecast accuracy using the observed predictors $x_t$ directly, without estimating the latent factors or specifying their number.
Given the dense nature of the working model established in Theorem (ref), does dimension reduction or noise removal remain desirable? Consider a hypothetical oracle that only the first $p_0$ predictors load on the latent factors while the remaining $p -p_0$ are pure noise, and thus the economist retains only the informative predictors and estimates the following “ideal" model:
Keeping $x_{I,t}$ and dropping the noise appears to be the natural course of action. The objective is to forecast out-of-sample data $y_{\operatorname{new}}$ satisfying: \[ y_{\operatorname{new}} = \rho' f_{\operatorname{new}} + \epsilon_{y,\operatorname{new}}, \qquad x_{I,\operatorname{new}} = \Lambda_I f_{\operatorname{new}} + u_{\operatorname{new}}, \] and the in-sample and out-of-sample environment is stable so that $\sigma_\epsilon^2 = \mathbb E(y_t - \rho'f_t)^2 = \mathbb E(y_{\operatorname{new}} - \rho'f_{\operatorname{new}})^2$.
We focus on a simple tuning-free dense forecasting method, the Ridgeless regression
to estimate the dense coefficient in ((ref)), where $\left( \cdot \right)^+$ denotes the Moore–Penrose pseudoinverse, $X_I = (x'_{I,1},\ldots,x'_{I,n})'$, and $y = (y_1,\ldots,y_n)'$. This estimator coincides with OLS when $p_0 < n$ and interpolates the data when $p_0 \ge n$ (meaning $y_t= x_{I,t}'\widetilde\beta_I$ for all in-sample data). It turns out that the natural forecast $$\widetilde{y}_{\operatorname{new}}^I = x_{I,\operatorname{new}}' \widetilde{\beta}_I$$ is not a good practice, as shown in the following proposition.
The assumptions in Proposition (ref) require the regression error terms $(\epsilon_{y,\operatorname{new}}, u_{\operatorname{new}})$ to be independent of the in-sample data; however, we allow correlations between $f_{\operatorname{new}}$ and the in-sample $f_t$ to accommodate dynamic factors in time series settings. The proposition establishes that even perfect variable selection fails to achieve first-order forecast optimality. The failure is not due to omitted signal --- all retained predictors are informative. Rather, when $p_0$ is proportional to $n$, the dimension is not sufficiently large to regularize the eigenvalues of the design matrix $X_I'X_I$, which is crucial for reducing the out-of-sample variance. Hence Ridgeless does not benefit from benign overfitting.
This result should be interpreted broadly as an extension of the classical post-variable-selection phenomenon to the overparameterized regime. Suppose a dimension reduction method, such as belloni2013least's Post-Lasso, identifies the informative predictors without error, and OLS is then applied to the selected set. When $p_0 < n$, the oracle Ridgeless estimator $\widetilde{\beta}_I$ coincides with OLS, making it equivalent to Post-Lasso under perfect selection. Proposition (ref) illustrates that even in this idealized setting, the resulting forecast fails to achieve first-order optimality when $p_0/n \to \gamma_0 \in (0,\infty)$.
Symmetry of AR($p$).\; If eigenvalue inflation cannot be achieved by retaining the informative predictors only, one might instead seek overparameterization by expanding model complexity in a different direction. In time series forecasting, an off-the-shelf candidate is to increase the lag order in an autoregressive model. Consider a time series of length $T$, where the objective is to forecast $y_{\operatorname{new}} = y_{T+1}$. Let $y_{a:b} = (y_t)_{t=a}^b$ denote the column vector of $y_t$ for periods $a$ to $b$. Consider the AR($p$) model with no intercept, estimated by the Ridgeless regression, which is equivalent to running AR($p$) with the demeaned data. The Ridgeless AR($p$) coefficient and one-step-ahead forecast are
For AR($p$), the effective sample size is $n=T-p$. So (ref) coincides with OLS when $p < T/2$ and interpolates when $p \ge T/2$.
If benign overfitting were driven simply by increasing the number of regressors, one would expect gains from choosing $p \geq T/2$. The following proposition, however, shows that this intuition breaks down due to a “symmetry” property of AR($p$).
Proposition (ref) reveals a novel symmetry property: an overparameterized AR($p$) model with $p > T/2$ is numerically identical to an underparameterized AR($T-p$) model. As $p$ increases, the effective sample size $n = T - p$ decreases proportionally, so the eigenvalues of the design matrix do not grow. Lag expansion therefore does not generate the eigenvalue inflation necessary for benign overfitting. Figure (ref) illustrates this symmetry.
Taken together, Propositions (ref) and (ref) convey a unified message: neither oracle variable selection nor lag expansion delivers eigenvalue growth in the design matrix. Under oracle selection, $p_0$ proportional to $n$ leaves eigenvalues insufficient in magnitude; under AR($p$), the effective sample size shrinks as the lag order grows. The central lesson is that the spectrum, not informativeness alone, governs forecast accuracy in the overparameterized regime. This motivates the approach developed in Section (ref), where we expand the predictor dimension to induce the spectral regularization.
Why not Lasso or PCA?\; Theorem (ref) establishes that $\beta$ is dense, which immediately limits the scope of the Lasso chernozhukov2017lava, giannone2021economic. In addition, Proposition (ref) covers the Post-Lasso under perfect variable selection as a special case, which also yields suboptimal forecast. PCA offers an alternative by estimating the factors directly, but the quality of the estimated factors depends critically on factor strength. When many noise variables are present, the factors appear weak across the full set of predictors, and correctly estimating the number of factors is difficult in practice; see recent work by giglio2023prediction and chao2022selecting from the perspective of feature selection. Furthermore, weak factors need not arise solely from noise contamination; factors can also be weak within informative predictors. If the eigenvalues of $\Lambda_I'\Lambda_I/p_0$ decay to zero as $p_0 \to \infty$ (so that $\psi_{p,n}$ grows slowly relative to $p_0$), the factor signal is intrinsically weak and PCA continues to suffer from poor factor recovery regardless of how successfully noise variables are screened.
Under the conditions of Theorem (ref), the working model is dense, meaning that a non-negligible fraction of predictors contributes to predictive performance. In practice, the predictor set comprises the informative variables $X_I$ ($n \times p_0$) and the noise variables $X_N$ ($n \times (p - p_0)$). Regardless of whether the economist knows the identities of informative predictors, Proposition (ref) shows that informative predictors solely are insufficient for first order forecast optimality when the number of informative predictors $p_0$ satisfies $p_0 / n \to \gamma_0 < \infty$. This is empirically relevant because stylized economic theory may provide only a few informative predictors.
This motivates our proposed forecast approach: rather than eliminating noise variables, one should retain all available predictors and, when necessary, intentionally add pure noise variables to expand the predictor set. Importantly, our approach does not distinguish between the two scenarios described in Section (ref). Formally, let $X = [X_I, X_N]$ be the $n \times p$ matrix pooling all predictors. In the first scenario, $X_N$ includes the noise variables already present in the dataset; in the second scenario, $X_N$ consists of artificially added noise columns. In both cases we apply the Ridgeless regression on predictors with noise:
We refer to ((ref)) as a “noise regularized estimator,” regardless of whether the noise predictors arise from the dataset or are intentionally introduced by the econometrician. A central contribution of this paper is to show that high-dimensional noise variables serve as an implicit regularization device: they inflate the non-factor eigenvalues of $X'X$ (those not corresponding to the factors), thereby reducing the out-of-sample variance. The next subsection explains this regularization mechanism in detail.
Given that the added noise contains no information, how does it facilitate prediction? It transpires that adding noise can be viewed as a regularization method, which inflates the eigenvalues of the design matrix. To see this, define the following $p_0\times 1$ vector-valued shrinkage function:
$$ \mathcal A(M)= X_I'(X_IX_I'+ M)^{-1}y, $$ where $M$ is an $n\times n$ symmetric positive semidefinite matrix, provided that the inverse exists. When $p>n$, the definition of pseudoinverse gives $(X'X)^+ X = X' (XX')^{-1}$. Therefore the subvector of the noise-regularized estimator ((ref)) corresponding to the informative predictors can be written as $$ \widehat\beta_I= \iota_I(X'X)^+X'y = X_I'(XX')^{-1}y =\mathcal A(X_NX_N') $$ where, without loss of generality, the first $p_0$ predictors are the informative ones picked out by the $p_0\times p$ selection matrix $\iota_I=(I_{p_0},0)$, and the last equality decomposes the $n\times n$ matrix $XX'= X_IX_I'+ X_NX_N'$, which is invertible when $p>n$. As a comparison, perfect variable selection specifies $M = 0$, and the Ridge regression with tuning parameter $\lambda$ sets $M = \lambda I_n$. The choice of $M$ governs how aggressively the eigenvalues of $X_IX_I'$ are inflated; a larger $M$ makes $(X_IX_I' + M)^{-1}$ smaller in the semidefinite order, directly controlling the out-of-sample variance.
To see the effect of shrinkage using a noise matrix, take the singular value decomposition (SVD):
where $D_I$ is the diagonal matrix of nonzero singular values of $X_I.$ Define $$ \mathcal S(M,\Sigma)=\operatorname{tr}\left[V_ID_I(D_I^2+U_I' M U_I)^{-2}D_IV_I'\Sigma\right] $$
Consider, for simplicity, the case when $x_{\operatorname{new}, I}$ is independent of the training data (to be relaxed in Assumption (ref)). It follows that the out-of-sample variance is $$ \operatorname{Var}(x_{\operatorname{new},I}'(\widehat \beta_I- \beta_I)|X)= \mathcal S(X_NX_N',\Sigma_{X_I}). $$ When $M=X_NX_N'$, it inflates the eigenvalues from $D_I^2$ along a non-factor direction to $D_I^2+U_I' M U_I$. Meanwhile, it is well-known from the random matrix theory that the nonzero eigenvalues of the noise matrix $X_NX_N'$ grow proportionally to $p$ if $ p/n \to \infty$. As a consequence, the order of the non-spiked (the largest $(K+1)$-th to the $n$-th) eigenvalues change from $p_0$ to the order of $p.$ Also for factor models, we have $\Sigma_{X_I}=\Lambda_I \Sigma_f \Lambda_I'+ \Sigma_{u_I}$, which yields a further decomposition of the variance into the factor-related component and the idiosyncratic component, and for now let us focus on the latter. This yields $$ \mathcal S(X_NX_N', \Sigma_{u_I})\leq C_u\operatorname{tr}\left[(D_I^2+U_I'WU_I)^{-2}D_I^2\right] = O_P\left(\frac{n+\psi_{p,n}}{p^2}\right), $$ which has a fast converging out-of-sample forecast variance.
In comparison, if we do not shrink the eigenvalues but only use $X_I$ after perfect variable selection, the Ridgeless estimator ((ref)) would take the form: $$ \widetilde\beta_I= (X_I'X_I)^+X_I'y =\mathcal A(0\cdot I_{n }), $$ whose forecast variance is $$ \operatorname{Var}(x_{\operatorname{new},I}'(\widetilde \beta_I- \beta_I)|X)= \mathcal S(0\cdot I_{n },\Sigma_{X}). $$ Here $M = 0\cdot I_{n }$ does not shrink the eigenvalues of $D_I^2$ and thus is of order $p_0$. Then under Assumption (ref) there exists a $0 < c_u \leq \sigma_{\min}( \Sigma_{u_I} )$ such that $$ \mathcal S(0\cdot I_n,\Sigma_{u_I})\geq c_u \operatorname{tr}(D_I^{-2}) \asymp \frac{n}{p_0}. $$ The variance does not converge when $n/p_0\nrightarrow 0$.
Inclusion of many random noise variables in essence inflates the eigenvalues to a much larger magnitude to reduce the out-of-sample variance. In this sense, it is similar to the Ridge regression with a large choice of tuning parameter, which also shrinks the eigenvalues. It is therefore important to see how the noise-regularized estimator compares with the Ridge estimator.
The Ridge estimator, when applied to $X_I$ with tuning parameter $\lambda$, takes the form $$ \widetilde\beta_{\operatorname{ridge}}(\lambda)= (X_I'X_I + \lambda \cdot I_n)^{-1} X_I' y = \mathcal A(\lambda \cdot I_{n}). $$ In addition, since $ \mathbb E X_NX_N' = (p-p_0)I_n $, a natural benchmark of the Ridge tuning parameter is $\lambda= p-p_0$.
Hence the noise-regularized estimator uses random shrinkage $ X_NX_N'$, whereas Ridge uses the deterministic shrinkage $\lambda\cdot I_n=\mathbb EX_NX_N'$. Both estimators introduce shrinkage bias, which will be compared heuristically. Conditioning on the informative predictors, the estimation bias of any estimator taking the form $ \widehat\beta= \mathcal A(M) $ is given by $$ \mathbb E(\widehat\beta|X_I)-\beta= \mathbb E[\mathcal A(M)|X_I] - \beta =\mathbb E[f(M)|X_I]\beta, $$ where $ f(M)= X_I'(X_IX_I'+M)^{-1}X_I- I_{p_0}. $ Note that $f(M)$ is a convex matrix-valued function, and is a negative semidefinite matrix whenever $M$ is positive semidefinite. Applying the matrix-valued Jensen's inequality (e.g., tropp2015introduction), $$ \mathbb E[f( X_NX_N')|X_I] \succcurlyeq f(\mathbb E( X_NX_N'|X_I)) =f(\lambda\cdot I_n), $$ where we write $B \succcurlyeq A$ (equivalently $A \preccurlyeq B$) if $A-B$ is negative semidefinite. Hence, the shrinkage effects to $\beta$ have the following relationship: $$ \underbrace{\mathbb E[f(\lambda\cdot I_n)|X_I]}_{\text{Ridge bias}} \preccurlyeq \underbrace{\mathbb E[f(X_NX_N')|X_I]}_{\text{Noise-regularized bias}} \preccurlyeq 0\cdot I_n. $$ Intuitively speaking, this means the shrinkage bias introduced by the random matrix $X_NX_N'$ is less than that of the deterministic matrix $\mathbb EX_NX_N' =\lambda\cdot I_n$ with $\lambda = p - p_0$, yielding a smaller finite-sample shrinkage bias than that of the corresponding Ridge.
Interestingly, the bias improvement over Ridge becomes more substantial as $p_0$ increases. This can be seen by examining the Jensen's gap, i.e., the difference between the upper and lower side of the Jensen's inequality. Consider the scalar version of $f(M)$, say $f(m, p_0)= \frac{p_0}{p_0+m}-1$. The Jensen gap is defined as $$ \mathbb Ef(m,p_0) - f(\mathbb E m ,p_0). $$ As shown in Figure (ref), when $m$ is much larger than $p_0$, the Jensen gap widens as $p_0$ grows. In the matrix-valued case, this corresponds to the case that when $\dim(X_N)> \dim(X_I),$ a larger dimension of $X_I$ yields a larger reduction of the shrinkage bias compared to the Ridge.
To summarize, the shrinkage representations through $\mathcal{A}(M)$ reveal how the structure of regularization differs. Perfect variable selection fixes $M = 0$ and provides no eigenvalue inflation, leaving the variance bounded away from zero. Ridge regression sets $M = \lambda\cdot I_n$ and imposes uniform, deterministic shrinkage. Noise regularization specifies $M = X_NX_N'$ and induces random shrinkage that, by the convexity of $f$, produces less bias than its Ridge counterpart while achieving the same variance reduction.
Below, we formalize this intuition of regularization, by establishing the first-order forecast optimality: The Ridgeless regularized by sufficiently many noise achieves the same asymptotic MSE as if the true factors were directly used for prediction. We shall present the results separately for the two scenarios.
In the first scenario, the economist does not know the identities of informative and noise variables. She uses all predictors in $X=[X_I, X_N]$, and predicts $y_{\operatorname{new}}$ by $$ \widehat y_{\operatorname{new}} = x_{\operatorname{new}}' \widehat \beta. $$ We impose technical assumptions below to establish the benefit of forecast with noises. Recall the data generating process in (ref) and (ref) shared by the training data and the test data indexed by for $t\in \{1,\ldots,n\}\cup\{\operatorname{new}\}$.
Assumption (ref) (i) imposes that $(U', u_{\operatorname{new}})$ are linear transformations of the two-way i.i.d. matrix with a finite 4th moment, which allows us to invoke results from the random matrix theory to reveal the behavior of empirical eigenvalues of the idiosyncratic sample covariance matrix $U'U/n$ of large dimensions. The independence between $(W', w_{\operatorname{new}})'$ and $\{ (f_t, \epsilon_{y,t})_{t \in \{1,\ldots,n\}\cup\{\operatorname{new}\}} \}$ strengthens the mean independence in (ref) to simplify the computation of the variance of the estimator. Condition (ii) regulates the unpredictable error in the target variable. Though it is possible to extend to {martingale difference sequences}, we maintain the independence to keep it in line with the assumption on $u_t$. Condition (iii) is a technical assumption that bounds the conditional variance of the new factor realizations, which accommodates the time series case when $f_{\operatorname{new}}$ is correlated with $F$. Finally, Condition (iv) ensures the uniqueness of the eigenvectors associated with the distinct top $K$ eigenvalues.
Conditioning on $X$, the predictive MSE is $$ \mathbb E[(y_{new}-\widehat y_{\operatorname{new}})^2|X] = \sigma_\epsilon^2+ \mathbb E\left[(\rho'f_{\operatorname{new}}- \widehat {y}_{\operatorname{new}} )^2|X\right] $$ The first term $\sigma_\epsilon^2 $ is the minimum forecast MSE under the true latent factors. The second term, depending on the estimator, can be decomposed as
where $ \rho'f_{\operatorname{new}}-x_{\operatorname{new}}'\beta $ is the approximation error of the working linear model (ref). The estimation error $x_{\operatorname{new}}'(\widehat{\beta}-\beta)$ can be further decomposed into
with $A_X= (X'X)^+X'X-I_p$ and $B_X= (X'X)^{+}X'$. The term $x_{\operatorname{new}}' \delta_\beta$ can be viewed as the bias (conditioning on $(U,F$)). The other term $x_{\operatorname{new}}'B_{X}\epsilon_{y}$ constitutes the variance due to the innovation $\epsilon_y$, and it is independent of $\beta$. Since $$\mathbb E[[ x_{\operatorname{new}}'(\widehat{\beta}-\beta)]^2|X]\leq 2\mathbb E[( x_{\operatorname{new}}'\delta_\beta)^2|X]+ 2\mathbb E[( x_{\operatorname{new}}'B_X\epsilon_y)^2|X],$$ the following theorem presents the convergence of each component on the right-hand side.
Theorem (ref) implies that keeping the noise variables and applying the Ridgeless forecast, we can achieve the oracle predictive MSE as if the latent factors were used directly in forecasting. This result allows factors to be relatively weak among the informative predictors, i.e., $\psi_{p,n}=o(p_0)$ in Assumption (ref) (i).
Contrary to conventional wisdom suggesting that the variance is amplified under overfitting, here the variance diminishes as $p\gg n$. The reduction of variance requires no condition on the predictability, i.e., it does not matter whether the predictors are mostly noise or informative. This is the phenomenon as documented as “benign-overfitting" in the literature. But different from the literature, the overfitting is mainly caused by high-dimensional uninformative predictors $X_N$. In the presence of many noise variables the out-of-sample variance decays as $p\to\infty$.
When economic theory and empirical evidence guide data collection, regressors are likely to be informative, but the number of available predictors $p_0$ may not be very large, say, just proportional to $n$. This occurs, for instance, in diffusion index forecasts using FRED-MD as mentioned in Introduction.
We recommend adding noise to intentionally increase the overall dimensionality. The inclusion of noise predictors inflates eigenvalues of the design matrix and thus reduces the out-of-sample variance. In the meantime, inspired by the Rao–Blackwellization for risk reduction hirano2017forecasting, the impact of randomness can be smoothed out by taking the average of repeated Ridgeless predictions.
Recall that $(X_I, y)$ denotes the in-sample data in this scenario, and the corresponding out-of-sample predictor is $x_{\operatorname{new}, I}$, where both $X_I$ and $x_{\operatorname{new}, I}$ are informative predictors.
Step 1 creates artificial pure noise $X_N$ to be merged with $X_I$, and then uses only the subvector $\widehat\beta_I$ for prediction; the forecast depends on the realization of $X_N$. Step 2 averages out the randomness caused by $X_N$ through the $B$ repetitions. This estimator approximates
with a sufficiently large $B$. This estimator can be connected to Bayesian model average: it arises as an equally weighted average forecasts (conditioning on the data) from $B$ submodels $\widehat y^b_{\operatorname{new}, I}$, where each submodel is implied by the drawn random noise, whose posterior probability is $1/B$.
As a model averaging estimator, it reduces the mean squared error of submodels. With a slight abuse of notation, we shall use $\widehat y_{\operatorname{new}}^* $ to denote ((ref)); it is a function of the observed data only. It follows that $$ x_{\operatorname{new}}'\beta-\widehat y_{\operatorname{new}}^*= \mathbb E(x_{\operatorname{new}}'\beta- \widehat y_{\operatorname{new}, I} | X_I, y,x_{\operatorname{new}, I}),$$ and the law of iterated expectations gives
where the inequality follows by the celebrated Rao-Blackwell theorem. It shows that $\widehat y_{\operatorname{new}}^*$ enjoys a smaller predictive MSE than that of a single realization $\widehat y_{\operatorname{new}, I}$.
In this scenario we define $x_{\operatorname{new}} = (x'_{\operatorname{new}, I}, 0')'$ and $\beta = (\beta'_I,0')'$. Then $$\mathbb E[ ( \widehat y_{\operatorname{new}}^*-x_{\operatorname{new}}'\beta)^2|X_I]\leq 2\mathbb E[( x_{\operatorname{new}}' \delta_\beta^*)^2|X_I]+ 2\mathbb E[( x_{\operatorname{new}}'B^*_X\epsilon_y)^2|X_I]$$ where
with $A_X^* = \mathbb E[ (X'X)^+ X'X | X_I]- I_p$ and $B_X^* = \mathbb E[ (X'X)^+ X |X_I] $. We have the following result for the noise-regularized forecast.
This theorem shows that adding the noise enables achieving the oracle forecast MSE as if the true factors were used. In addition, the rate of convergence of the forecast variance is faster than that of Scenario I when $p=o(n^2)$, as a result of the Rao–Blackwellization step.
To provide a quick demonstration, we show in Figure (ref) the theoretical curves of bias-variance (left panel) and predictive MSE $\mathbb E[(y_{new}-\widehat y^*_{\operatorname{new}})^2|X_I]$ (right panel) in a 3-factor model. Here the first $p_0=\min\{p,n\}$ predictors are informative, while the remaining $p-p_0$ predictors are i.i.d. $N(0,1)$ white noise variables. As is clearly illustrated, the variance monotonically increases as $p$ increases even though the first $p_0$ added predictors are all informative, and peaks at $p=n$ where the in-sample data are perfectly interpolated. Meanwhile, after $p>n$, the added predictors are pure noise, and the variance starts to decay. This is consistent with our theory: as $p\to\infty$, the variance continues to decrease until the $1/n$ term becomes dominant. In addition, the squared bias remains zero until $p=n$. Though after passing this threshold it starts to increase, it is in a much smaller magnitude than that of the variance. This is also consistent with the theory. The depicted bias does not diminish because here we fix $\psi_{p,n}\asymp n$ at $n=100$ while we vary $p$.
Overall, the predictive MSE (right panel) illustrates the double descent phenomenon, where the first descent occurs before $p<20$, due to the decay of the gap $\sigma_e^2- \sigma_{\epsilon}^2$. The second descent occurs after $p>n$, due to the decay of variance, and eventually the MSE approaches the oracle MSE as if the latent factors were used for prediction. While double descent has been documented in the recent statistical literature, our result is novel in the literature, as elaborated in Section (ref).
Both Figure (ref) and our theory indicate an inflation of bias when $p$ becomes excessively large. We therefore propose a data-driven way to choose the number of added noise. If it is believed that most of the informative predictors load strongly on the common factors, then $ p_0\asymp\psi_{p,n}$ and our theory shows: $$ \mathbb E[(x_{\operatorname{new}}'\beta-\widehat y_{\operatorname{new}}^*)^2|X]= O_P\left(\frac{p}{p_0n}+\frac{np_0}{p^2}\right). $$ We can choose the optimal number of predictors $p$ by minimizing the rate of convergence: $$ p = C\times (np_0)^{2/3} \asymp \arg\min_p\left(\frac{p}{p_0n}+\frac{np_0}{p^2}\right), $$ where $C$ is a constant to be determined. A data-driven choice of $C$ is available via commonly used tuning strategies, which we now discuss.
For cross-sectional predictions where the ordering of data is unimportant, the “leave-one-out" is a computationally attractive method. Suppose $C$ is a candidate choice, then denote the total number of predictors, $p(C)=C\times (np_0)^{2/3}$ (after adding noise). For each $t\leq n$, let $\widehat\beta_{-t}(C) $ denote the Ridgeless estimator using $p(C)$ predictors and all data except the $t$-th observation. Then the optimal $C$ can be chosen by minimizing: $$ \text{leave-one-out: } \quad C^*=\arg\min_{C}\sum_{t=1}^n\left( y_t- x_t'\widehat\beta_{-t}(C)\right)^2. $$ Interestingly, we do not have to compute $n$ leave-one-out $\widehat\beta_{-t}(C)$ for this procedure, thanks to an elegant analytic formula given by shen2023algebraic. They showed that the leave-one-out procedure is equivalent to minimizing:
where $X(C)$ denotes the $n\times p(C)$ matrix of $p(C)$ predictors, and $\operatorname{Diag}(G)$ takes the diagonal elements of $G$. Here $G(C)$ is invertible because $\operatorname{rank}(X(C))=n<p(C)$, whose computation is manageable so long as $n$ is not very large.
For time-series predictions where there is a natural ordering of the data, we recommend the rolling window tuning for time series cross-validation: predetermine $T_0$ and $V_0$ as the sizes of training and validation sets. Let $\widehat \beta_{t}(C)$ denote the estimated coefficient using $T_0$ periods of data: $t-T_0+1,..., t$, with tuning parameter $C$. We then choose $C^*$ by: $$ C^*=\arg\min_C \sum_{l=0}^{L-1}(y_{T_0+l}- x_{T_0+l}'\widehat \beta_{T_0+l}(C))^2. $$
To illustrate the effectiveness of data-driven choices of $p_{cv}=p(C^*)$, Figure (ref) plots the predictive MSE of two simulated designs, where the DGP is generated as in one of the simulation designs in Section (ref) with weak factors. The first $p_0=200$ predictors are informative predictors, whereas the rest $p-p_0$ predictors are intentionally added noise variables. In both cases the data-driven choice $p_{cv}$ suitably identifies a proper number of added noise.
Contrary to conventional statistical wisdom, which asserts that overfitting undermines forecast performance by inflating out-of-sample variance, the double descent phenomenon shows that it is possible to retain predictability by accumulating the number of regressors beyond the threshold of interpolation.
It has gained increasing attention in the machine learning community. As the model complexity exceeds the sample size and continues to grow, a second descent of the prediction occurs in the excessively overparametrized regime. It was first illustrated in the empirical work by belkin2019reconciling, hastie2022surprises, and arora2019implicit, and its theory has been explored in linear models with Ridgeless regressions, e.g., mei2019generalization, belkin2020two, dobriban2018high, lee2023mean.
Our work differs from the recent statistical literature on overparameterized models in three important respects. First, while the literature does not explicitly assume $\|\beta\|\to 0$, this condition is in fact necessary for the forecast consistency. In the i.i.d. setting without presence of strong factors, hastie2022surprises characterize the limiting predictive MSE, whose convergence requires this condition. It turns out that $\|\beta\|\to 0$ remains necessary in the presence of factors; otherwise, the overparametrized bias would not vanish asymptotically. However, the literature has provided little justification for why such a condition should hold in a practical model.
Our paper provides a formal and economically motivated justification for this condition. Our true DGP is driven by a low-dimensional factor structure $y_t = \rho' f_t + \epsilon_{y,t},$ while the econometrician estimates an overparameterized linear regression $ y_t = x_t' \beta + u_t $ as a working model. In this environment, the population regression coefficient $\beta$ is a reduced-form object implied by the underlying factor structure. As the cross-sectional dimension grows, the factor structure implies $\|\beta\|\to 0$. Hence, the shrinkage condition required for consistency is not imposed, but arises naturally from a standard economic DGP.
Secondly, as explained in the previous subsection, we show that the second descent arises from the shrinkage of the eigenvalue spectrum induced by the presence of many noise variables. Although these noise variables contain no predictive information about the target variable, they cause the high-dimensional sample covariance matrix to have a highly dispersed eigenvalue spectrum. This lowers the out-of-sample variance.
Lastly, from an important technical perspective, our framework explicitly analyzes the impact of a few very large eigenvalues on overparametrized forecasts. In much of the recent work, the design matrix is assumed to have uniformly bounded eigenvalues, which effectively imposes a weak dependence structure across predictors. In contrast, many economic forecasting environments are inherently driven by a small number of latent common factors. These factors generate strong cross-sectional dependence among informative predictors, inducing an approximate low-rank structure in the population covariance matrix. While some recent work on overparameterized models allows for diverging eigenvalues induced by factor structures (e.g., meng2025estimation), the implications of such growing eigenvalues in forecasting models remain unclear.
We provide a careful analysis of this case and show how the interaction between factor-driven spectral spikes and overparameterization shapes the predictive risk. To understand the intuition, start with the matrix form of the factor model
By applying SVD on both $X'X$ and $\mathbb E x_tx_t'$, we can separate the effect of the factor model from its remaining counterparts. First, consider the SVD of $\mathbb Ex_tx_t' $: $$\mathbb Ex_tx_t' = \bar V_{K} \bar D_K^2\bar V_{K}'+\bar V_{-K} \bar D_{-K}^2\bar V_{-K}' $$ corresponding to the first $K$ eigenvectors/values $(\bar V_{K}, \bar D_K)$ and last $p-K$ eigenvectors/values $(\bar V_{-K}, \bar D_{-K})$. In addition, consider the SVD on $X'X$: $$ X'X = V_KD_K^2 V_K' + V_{-K}D_{-K}^2 V_{-K}' + V_{-n} 0 V_{-n}', $$ where $D_K^2$ and $D^2_{-K}$ respectively denote the diagonal matrices of the largest $K$ and the remaining $n-K$ nonzero eigenvalues of $X'X$; columns of $V_K$ and $V_{-K}$ are the corresponding eigenvectors; columns of $V_{-n}$ are the eigenvectors corresponding to the $p-n$ zero-eigenvalues.
Then we decompose the forecast bias and variance of $\widehat y_{\operatorname{new}}$ into:
where for $\Sigma_X = \mathbb E (x_t x_t')$,
Both bias and variance depend on components that arise from the factor model $A_1, B_1$, and from the idiosyncratic noise $A_2, B_2$ that behave similarly to the classic i.i.d. design. In particular, the convergence of $A_1$ and $B_1$ rely on three properties of the factor model: (i) the population eigenvectors $\bar V_K$ can be well approximated by the sample eigenvectors $V_K$; (ii) the implied reduced form parameter is dense in that $\|\beta\|\to0 $; and (iii) the top $K$ eigenvalues grow very fast. As for $A_1$, the sample eigenvectors corresponding to the first $K$ eigenvalues are orthogonal to $V_{-n}$. Hence (i) implies $\| \bar V_{K}'V_{-n}\|\approx \| V_{K}'V_{-n}\| \stackrel{P}{\to} 0$, and the convergence of $A_1$ is ensured together with (ii). In addition, the behavior of $B_1$ is driven by the top $K$ sample eigenvalues, and thus $B_1$ converges as $\|D_K^{-2}\|= O_P((\psi_{p,n}n)^{-1})$.
In summary, when most of the high dimensional predictors are generated by a few common factors, the factors lead to a few spiked eigenvalues facilitating the convergence of variance; they also naturally yield a dense regression coefficient to shrink the bias.
We run Monte Carlo experiments to check the performance of noise-regularization under the factor model specification in (ref) and (ref). The outcome variable is generated from a $K=4$ factor model $y_t = \rho' f_t + \epsilon_{y,t}$, with $p_0$ informative predictors and $p - p_0$ noise: $$ x_{i,t} =
\quad where \quad \lambda_i = \lambda_{i,0} \times p_0^{\tau}, $$ and $(f_t, \epsilon_{y,t}, \lambda_{i,0}, u_{i,t})$ are all standard normal, and $\rho \sim N(0, I_K)$. Here $\tau \in [-1/2, 0]$ determines the factor strength so that $\Lambda_I' \Lambda_I \asymp p_0^{1+2\tau}$; the closer $\tau$ to zero, the stronger the factors. To capture heterogeneous factor strengths, we allow $\tau$ to differ across the $K = 4$ factors and consider two configurations:
The mixed case is of particular interest because the two factors with $\tau = -1/2$ are very weak. The sample size is $n = 100$ throughout, and test MSE is evaluated on $n_{\operatorname{new}} = 500$ new observations. All results are averaged over 100 Monte Carlo replications. We set $p$ to take values on a grid evenly spaced from 1 to $p_{\max} = 1000$. We consider the two scenarios as in Sections (ref) and (ref), respectively.
The economist uses all $p$ collected predictors without knowing which predictors are informative. We set $p_0 \in \{500, 1000\}$; all predictors are informative when $p \leq p_0$, and the additional $p - p_0$ are noise when $p > p_0$. Three methods are compared, all using the full set of $p$ predictors: the proposed noise-regularized estimator, PCA with the number of factors chosen by BN02, and CV-Lasso where the tuning parameter is chosen by 10-fold cross-validation.
Figure (ref) plots the predictive MSE as $p$ increases. All predictors are informative when $p \leq p_0$, whereas $p - p_0$ noise variables enter when $p > p_0$. In all four panels, the noise-regularization dominates both PCA and CV-Lasso, particularly in the overparameterized regime ($p > n$). In panel (b), where all predictors are informative ($p_0 = 1000$) with a homogeneous $\tau = -1/3$, including more weak predictors continuously benefits noise-regularization, driving its MSE down to 2.40 at the right edge. In panels (a) and (c), where $p_0 = 500$ and the number of informative predictors stops increasing, the MSE curve flattens after $p_0$ but remains well below the competitors.
PCA gains little from a large $p$ because the factors are relatively weak, which misses a substantial portion of the predictive signal. CV-Lasso struggles for a different reason: the dense factor structure spreads the signal across all informative predictors, each carrying a small but nonzero coefficient. Lasso's variable selection, designed for sparse models, selects too few predictors to recover this diffuse signal.
When the economist knows which predictors are informative, she intentionally adds noise to implement noise-regularization. The $p_0$ informative predictors are augmented with $p - p_0$ columns of $N(0,1)$ noise. The noise-regularized estimator is computed for $B = 50$ times with independent noise draws, and the predictions are averaged.
We set $p_0 \in \{200, 300\}$ and compare four methods: Noise-regularized with the number of added noise variables selected via leave-one-out cross-validation;\footnote{Algorithm (ref) in Section (ref) of the appendix presents a detailed algorithm for selecting the number of added noise using leave-one-out method.} CV-Ridge which applies the Ridge regression on the $p_0$ informative predictors and tuned by 10-fold cross-validation; CV-Lasso and PCA. Except for noise-regularization, all the three competing methods use the informative predictors only: they are oracle methods with perfect variable selection.
Figure (ref) plots the predictive MSE as $p$ increases. The horizontal dashed lines show the MSE of the oracle methods, and the solid curve is the noise-regularized forecast. The vertical dashed line indicates $p_{cv}$ selected by leave-one-out cross-validation. The cross-validation selects $p_{cv}$ close to the oracle optimum. In panel (a), for instance, we select $p_{cv} \approx 400$ while the minimum of the MSE curve occurs at $p \approx 380$.
In all four panels, noise-regularization at $p_{cv}$ outperforms CV-Ridge, despite the latter's access to the oracle. The improvement is most pronounced in the mixed-$\tau$ panels: with $p_0 = 200$, the noise-regularized MSE is 2.85 versus 2.93 for CV-Ridge; with $p_0 = 300$, the gap widens to 3.32 versus 3.41. Even in the homogeneous case ($\tau = -1/3$), noise-regularization slightly outperforms Ridge (2.15 vs.\ 2.17 for $p_0 = 200$; 2.40 vs.\ 2.41 for $p_0 = 300$).
The advantage over Ridge is amplified when factor strengths are heterogeneous. Ridge regression applies a single penalty $\lambda$ to all coefficients, whereas the implicit regularization through noise addition adapts to the signal structure---coefficients associated with stronger factors are shrunk less. This heterogeneous shrinkage is a consequence of the randomized regularization mechanism and cannot be replicated by a single Ridge penalty. CV-Lasso and PCA both perform substantially worse. Lasso suffers from the dense factor structure, while PCA fails to detect the weakest factors, especially in the mixed-$\tau$ panels where two factors have $\tau = -1/2$.
We apply the noise-regularized forecast to three settings in which high-dimensional prediction is of first-order importance: U.S.\ inflation, cross-country GDP growth, and the U.S.\ equity premium. In each application the economist observes $p_0$ economic predictors and augments them with $p - p_0$ columns of i.i.d. $N(0,1)$ noise. The Rao-Blackwell denoised prediction averages $B = 50$ independent noise draws. The total dimension $p$ is selected by data-driven cross-validation, specifically leave-one-out in the cross-sectional application (growth) and rolling-window time-series CV in the two time-series applications (inflation and equity premium). We compare with CV-Lasso and CV-Ridge, both tuned by five-fold cross-validation and using only the $p_0$ economic predictors.
In the subsequent subsections, we study the three applications in detail. Table (ref) summarizes the main results across the studies.
Accurate inflation forecasts are essential for monetary policy, business planning, and investment decisions. We use the FRED-MD dataset to forecast the change in the U.S.\ inflation rate $$ \Delta \text{inflation}_{t+1}= \log(\text{CPI}_{t+1} /\text{CPI}_{t} ) -\log(\text{CPI}_{t} /\text{CPI}_{t-1} ), $$ as defined by mccracken2016fred. The dataset includes macroeconomic predictors spanning June 1959 to January 2024. We exclude variables with missing observations over this period, resulting in $p_0 = 102$ predictors. We use 120-month (10-year) rolling windows to produce one-month-ahead forecasts. This dataset is widely recognized for its inherent challenge of relatively weak factors: data-driven criteria such as BN02's information criteria typically suggests 8--10 factors, explaining only 50--62% of total variation.
Adding noise predictors inflates the eigenvalues of the design matrix and provides the implicit regularization that avoids estimating weak factors. Noise-regularization augments the $p_0$ macro variables with $p - p_0$ columns of $N(0,1)$ noise, and the dimension $p$ is selected by time-series cross-validation on a burn-in period prior to the out-of-sample evaluation. The optimal number of predictors is $p_{cv} = 526$, corresponding to roughly 420 added noise columns. CV-Lasso, CV-Ridge, and PCA use only the original macro predictors throughout.
Figure (ref) plots the predictive MSE as $p$ increases on a logarithmic grid. The noise-regularized forecast exhibits a sharp spike near the interpolation threshold ($p \approx n = 120$) but its MSE drops rapidly as more noise is introduced, outperforming benchmarks by $p \approx 400$. At $p_{cv} = 526$ the noise-regularized MSE is 0.982, an 11% improvement over CV-Lasso (1.103) and CV-Ridge (1.102). Beyond $p_{cv}$, the MSE curve remains nearly flat through $p > 13{,}000$, demonstrating the robustness to the number of added noise variables predicted by the theory.
Economic growth forecasting has been one of the earliest high-dimensional applications in economics sala1997just. We use the dataset of barro1994sources, which contains $p_0 = 60$ socio-economic and geographical characteristics for 90 countries. In each replication we randomly split the countries into a training set of $n = 45$ and a test set of 45, fit the model on the training set, and evaluate predictions on the test set. Results are averaged over 50 random splits. The noise-regularized method augments the 60 predictors with $p - p_0$ noise columns. The selected total number of predictors, including noise, is $p_{cv} = 396$ using leave-one-out cross-validation.
Figure (ref) plots the predictive MSE. The MSE using only the original 60 predictors via Ridgeless is very large ($\approx 14$) due to overfitting in the $p \approx n$ regime. After the interpolation threshold, the MSE drops precipitously as noise dimensions are added and stabilizes below 1.0 by $p \approx 200$. At $p_{cv} = 396$ the noise-regularized MSE is 0.940, a 25% improvement over CV-Lasso (1.262) and a 21% improvement over CV-Ridge (1.193). PCA achieves an MSE of 1.073, outperforming Lasso and Ridge but still 14% above noise regularization. The MSE remains stable through $p > 34{,}000$, i.e., nearly 600 times the original predictor count, underscoring the method's robustness.
Predictability of the U.S.\ equity premium is a central and contentious question in asset pricing. Goyal2008comprehensive conducted a comprehensive out-of-sample examination of prevailing predictive models, concluding that most fail to beat the historical-mean benchmark. goyal2023comprehensive updated the analysis with new predictors and methods, arriving at qualitatively similar conclusions: the majority of models yield negative or barely positive out-of-sample $R^2$.
We revisit this question using the 15 predictors in Goyal2008comprehensive: dividend-price ratio, earnings-price ratio, book-to-market, Treasury bill rate, default yield spread (AAA, BAA), long-term yield, net equity expansion, risk-free rate, inflation, long-term return, corporate bond return, stock variance, and lagged market returns. We forecast the equity premium at annual, quarterly, and monthly horizons.\footnote{We drop the cross-sectional premium (csp) because it is discontinued after 2002; the remaining 15 predictors extend through 2024.} The out-of-sample evaluation covers 1960--2024 using 20-year rolling windows, harmonized across all three frequencies. The out-of-sample $R^2$ is defined as $$ R^2_{\text{oos}} = 1 - \frac{\sum_{t}(y_{t+1}-\widehat y_{t+1})^2}{\sum_{t}(y_{t+1}-\bar y_t)^2}, $$ where $\bar y_t$ is the (expanding window) historical mean of returns up to time $t$.
At each out-of-sample date, we augment the 15 predictors with $p - 15$ columns of $N(0,1)$ noise, compute the noise-regularization method averaged over $B=50$ independent draws as Rao-Blackwellization. The dimension $p$ is selected by time-series cross-validation on a burn-in period (pre-1960 data).
Figure (ref) displays the results. For small $p$, the noise-regularized forecast has strongly negative $R^2$ because the augmented design is near or below the interpolation threshold ($p \approx n$). As $p$ grows into the overparameterized regime, the $R^2$ rises sharply and stabilizes in the positive territory: precisely the second-descent behavior predicted by the theory.
At the annual frequency (panel a), the noise-regularized forecast achieves $R^2_{\text{oos}} = +1.81\%$ at $p_{cv} = 1{,}000$, compared with $-14.49\%$ for CV-Lasso and $-11.92\%$ for CV-Ridge. The annual horizon offers the strongest gains, consistent with the well-documented finding that return predictability is more pronounced at lower frequencies cochrane2011presidential. At the monthly frequency (panel b), the noise-regularized $R^2$ is $+0.37\%$ at $p_{cv} = 12{,}000$, while CV-Lasso ($-0.46\%$) and CV-Ridge ($-1.59\%$) again fail to beat the historical mean. Although the monthly $R^2$ is modest in absolute terms, a positive out-of-sample $R^2$ at the monthly horizon is economically meaningful: as campbell2008predicting note, monthly $R^2$ values in the range of 0.5% can generate substantial utility gains for a mean-variance investor. The pattern is consistent at the quarterly frequency as well (Appendix (ref)). In all cases, the $R^2$ for PCA is strongly negative (see Table (ref)).
Three features merit emphasis. First, the cross-validated $p_{cv}$ scales roughly in proportion to the sample size: $p_{cv}/n$ is approximately 50 at all three frequencies ($1{,}000/20$, $4{,}000/80$, $12{,}000/240$), suggesting a stable optimal noise-to-signal ratio. Second, the $R^2$ curve is remarkably flat beyond $p_{cv}$; even doubling the number of noise variables has little effect, confirming the robustness documented in the simulations. Third, the improvement over CV-Lasso and CV-Ridge does not come from better variable selection but from the opposite direction: by adding pure noise, the method achieves implicit regularization that is better adapted to the dense, weak-factor structure of equity premium predictors than explicit penalization with a single tuning parameter.
When economic outcomes are driven by latent factors shared across a large predictor panel, the reduced-form forecasting model is dense. This paper takes denseness as a starting point and establishes four results. The population working model converges to the oracle that uses the true factors as predictor dimension grows (Theorem (ref)). Perfect variable selection fails to achieve first-order forecast optimality when the number of retained predictors is comparable to $n$, because the design matrix lacks the dimensionality needed for eigenvalue regularization (Proposition (ref)). Noise regularization, i.e., retaining or deliberately adding uninformative predictors restores optimality by inflating the eigenvalue spectrum, without estimating the factors or their number (Theorems (ref) and (ref)). The AR($p$) symmetry property clarifies that overparameterization through additional lags cannot substitute for expanding the predictor set (Proposition (ref)).
Our empirical applications illustrate this principle in three areas of economic forecasting. Noise regularization improves and stabilizes forecasts of U.S.\ inflation, cross-country GDP growth, and the U.S.\ equity risk premium, outperforming Lasso, Ridge, and PCA benchmarks. The central message is that the relevant margin for improving dense linear forecasts is the dimension of the predictor space, not the informativeness of its individual components.