Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
89,128 characters · 27 sections · 55 citation commands
High-Dimensional Learning in Finance
The integration of machine learning methods into financial prediction has emerged as one of the most active areas of research in empirical asset pricing kelly2024virtue,Gu2020ML,Bianchi2021Bond,Chen2024Deep,fallahgoul2024asset,Feng2020Taming. The appeal is clear: while financial markets generate increasingly high-dimensional data, traditional econometric methods remain constrained by limited sample sizes and the curse of dimensionality. Machine learning promises to uncover predictive relationships that elude traditional linear models by leveraging nonlinear approximations and high-dimensional overparameterized representations, thereby expanding the frontier of return predictability and portfolio construction.
Yet despite rapid adoption and impressive empirical successes, our theoretical understanding of when and why machine learning methods succeed in financial applications remains incomplete. This gap is particularly pronounced for high-dimensional methods applied to the notoriously challenging problem of return prediction, where signals are weak, data are limited, and spurious relationships abound. A fundamental question emerges: under what conditions can sophisticated machine learning methods genuinely extract predictive information from financial data, and when might apparent success arise from simpler mechanisms?
The pioneering work of kelly2024virtue has significantly advanced our theoretical understanding by establishing rigorous conditions under which complex machine learning models can outperform traditional approaches in financial prediction. Their theoretical framework, grounded in random matrix theory, demonstrates that the conventional wisdom about overfitting may not apply in high-dimensional settings, revealing a genuine 'virtue of complexity' under appropriate conditions. This breakthrough provides crucial theoretical foundations for understanding when and why sophisticated methods succeed in finance.
However, fundamental questions remain about the information-theoretic feasibility of high-dimensional learning under the weak signal conditions typical in finance. While kelly2024virtue demonstrate that complexity can be beneficial under appropriate theoretical conditions, their analysis does not address whether realistic financial environments provide sufficient signal strength and sample sizes to support reliable learning in high-dimensional regimes. This gap is particularly pronounced given the notoriously challenging nature of return prediction, where signals are weak, data are limited, and the signal-to-noise ratios may be insufficient to support the extraction of predictive information from thousands of features using dozens of training observations.
Building on these theoretical advances, this paper examines how practical implementation details interact with established mechanisms. This becomes important as recent empirical analysis nagel2025seemingly suggests that high-dimensional methods may achieve success through multiple pathways that differ from theoretical predictions. Several questions emerge: What are the information-theoretic requirements for learning with weak signals? How do implementation choices affect underlying mathematical properties? When do complexity benefits reflect different learning mechanisms? Understanding these interactions helps characterize the complete landscape of learning pathways in high-dimensional finance applications.
This paper provides theoretical foundations for answering these questions through three main contributions that help characterize the different mechanisms through which high-dimensional methods achieve predictive success in financial prediction.
First, I extend the theoretical analysis to practical implementations, showing how the standardization procedures commonly used for numerical stability modify the kernel approximation properties that underlie existing theory. While Random Fourier Features (RFF) theory rigorously proves convergence to shift-invariant Gaussian kernels under idealized conditions rahimi2007random,sutherland2015error, I prove that the within-sample standardization employed in every practical implementation modifies these theoretical properties. The standardized features converge instead to training-set dependent kernels that violate the mathematical foundations required for kernel methods. This breakdown explains why methods cannot achieve the kernel learning properties established by existing theory and must rely on fundamentally different mechanisms.
rahimi2007random prove that for features $z_i(x) = \sqrt{2}\cos(\omega_i^\top x + b_i)$ with $\omega_i \sim \mathcal{N}(0, \gamma^2 I)$ and $b_i \sim \text{Uniform}[0,2\pi]$, the empirical kernel $\frac{1}{P}\sum_{i=1}^P z_i(x)z_i(x')$ converges in probability to the Gaussian kernel $k(x,x') = \exp(-\gamma^2\|x-x'\|^2/2)$ as $P \to \infty$. This convergence requires that features maintain their original distributional properties and scaling. However, I prove that the within-sample standardization $\tilde{z}_i(x) = z_i(x)/\hat{\sigma}_i$ employed in every practical implementation—where $\hat{\sigma}_i^2 = \frac{1}{T}\sum_{t=1}^T z_i(x_t)^2$—fundamentally alters the convergence properties. The standardized features converge instead to training-set dependent kernels $k^*_{\text{std}}(x,x'|\mathcal{T}) \neq k_G(x,x')$ that violate the shift-invariance and stationarity properties required for kernel methods. A detailed analysis of how standardization breaks the specific theoretical conditions appears in Section (ref), following the formal proof of this breakdown.
Second, I derive sharp sample complexity bounds that characterize the information-theoretic limits of high-dimensional learning in financial settings. Using PAC-learning theory,\footnote{ In PAC-learning valiant1984pac, a predictor is “probably approximately correct’’ if, with $T\!\gtrsim\!(\text{capacity})/\varepsilon^{2}$ samples, its risk is within $\varepsilon$ of optimal with probability $1-\delta$; I apply these bounds (see kearns1994intro) to gauge when weak return signals are learnable.} I establish both exponential and polynomial lower bounds showing when reliable extraction of weak predictive signals becomes impossible regardless of the sophistication of the employed method. These bounds reveal that learning with thousands of parameters using typical financial sample sizes requires signal-to-noise ratios exceeding realistic bounds by orders of magnitude, demonstrating that the "virtue of complexity" regime ($P \gg T$) is information-theoretically infeasible under standard financial conditions. This explains why apparent success in high-dimensional regimes must arise through alternative mechanisms rather than genuine complex learning nagel2025seemingly.
This capability to flexibly trade computational cost for representational power makes RFF a powerful tool. However, this paper reveals two fundamental barriers to realizing these benefits in practice. First, I prove that the standardization procedures universally employed violate the theoretical conditions required for kernel approximation (Section (ref)). Second, I establish information-theoretic limits showing that reliable learning in high-dimensional regimes requires sample sizes and signal strengths far exceeding those available in typical financial applications (Section (ref)).
While these theoretical results provide clear mathematical boundaries on learning feasibility, their practical relevance depends on how they manifest across the parameter ranges typically employed in financial applications. The gap between asymptotic theory and finite-sample reality can be substantial, particularly when dealing with the moderate dimensions and sample sizes common in empirical asset pricing. Moreover, the breakdown of kernel approximation under standardization represents a fundamental departure from assumed theoretical properties that requires empirical quantification to assess its practical severity.
To bridge this theory-practice gap, I conduct comprehensive numerical validation of the kernel approximation breakdown across realistic parameter spaces that span the configurations used in recent high-dimensional financial prediction studies kelly2024virtue,nagel2025seemingly. The numerical analysis examines how within-sample standardization destroys the theoretical Gaussian kernel convergence that underlies existing RFF frameworks, quantifying the magnitude of approximation errors under practical implementation choices. These experiments reveal that standardization-induced kernel deviations reach mean absolute errors exceeding 40% relative to the theoretical Gaussian kernel in typical configurations ($P = 12{,}000$, $T = 12$), with maximum deviations approaching 80% in high-volatility training windows. The kernel approximation failure manifests consistently across different feature dimensions and sample sizes, with relative errors scaling approximately as $\sqrt{\log P/T}$ in line with theoretical predictions. The numerical validation thus provides concrete evidence that practical implementation details create substantial violations of the theoretical assumptions underlying high-dimensional RFF approaches, with error magnitudes sufficient to fundamentally alter method behavior.
To assess the practical relevance of my theoretical bounds, I conduct an empirically-grounded calibration of the polynomial minimax lower bound from Theorem (ref)(a). Using defensible parameters for signal strength (calibrated to an \(R^2\) of 1--5%) and noise variance drawn from historical market data, we diagnose the nature of the learning problem faced by financial prediction models. My analysis reveals a profound implication: the critical sample size (\(T_{\mathrm{crit}}\)) required to overcome the limitations imposed by weak signals is on the order of decades, even for low-dimensional models. Since typical applications, such as those in KMZ, employ much shorter training windows (e.g., 12 months), they operate deep within a signal-limited regime. In this regime, the theoretical floor on performance is dictated by the weak economic signal, not model complexity, calling into question the core premise of using high-dimensional methods for this task.
This paper builds on three distinct but interconnected theoretical traditions to provide foundations for understanding high-dimensional learning in financial prediction.
The Probably Approximately Correct (PAC) framework valiant1984pac,kearns1994intro provides fundamental tools for characterizing when reliable learning is information-theoretically feasible. Classical results establish that achieving generalization error $\varepsilon$ with confidence $1-\delta$ requires sample sizes scaling with the complexity of the function class, typically $T = O(\text{complexity} \cdot \log(1/\varepsilon)/\varepsilon^2)$ shalev2014understanding. Recent advances in high-dimensional learning theory belkin2019double,bartlett2020benign,hastie2022surprises have refined these bounds for overparameterized models, showing that the effective rather than nominal complexity determines learning difficulty. However, these results have not been systematically applied to the specific challenges of financial prediction, where weak signals and limited sample sizes create particularly demanding learning environments.
The RFF methodology rahimi2007random provides computationally efficient approximation of kernel methods through random trigonometric features, with theoretical guarantees assuming convergence to shift-invariant kernels under appropriate conditions rudi2017generalization. Subsequent work has characterized the approximation quality and convergence rates for various kernel classes mei2022generalization, establishing RFF as a foundation for scalable kernel learning. However, existing theory assumes idealized implementations that may not reflect practical usage. In particular, no prior work has analyzed how the standardization procedures commonly employed to improve numerical stability affect the fundamental convergence properties that justify the theoretical framework.
The phenomenon of "benign overfitting" in overparameterized models has generated substantial theoretical interest belkin2019double,bartlett2020benign, with particular focus on understanding when adding parameters can improve rather than harm generalization performance. The VC dimension provides a classical measure of model complexity that connects directly to generalization bounds vapnik1998statistical, while recent work on effective degrees of freedom hastie2022surprises shows how structural constraints can limit the true complexity of nominally high-dimensional methods. These insights have been applied to understanding ridge regression in high-dimensional settings, but the connections to kernel methods and the specific constraints imposed by ridgeless regression in financial applications remain underexplored.
The application of machine learning to financial prediction has generated extensive empirical literature Gu2020ML,kelly2024virtue,Chen2024Deep, with particular attention to high-dimensional methods that can potentially harness large numbers of predictors Feng2020Taming,Bianchi2021Bond. The theoretical framework of kelly2024virtue provides crucial insights into when high-dimensional methods can succeed, particularly their demonstration that ridgeless regression can achieve positive performance despite seemingly problematic complexity ratios. However, this work has faced significant empirical challenges. buncic2025simplified demonstrates that the key empirical finding of a "virtue of complexity"—where portfolio performance increases monotonically with model complexity—results from specific implementation choices including zero-intercept restrictions and particular aggregation schemes rather than genuine complexity benefits. When these restrictions are removed, simpler linear models using only 15 predictors substantially outperform the complex machine learning approaches. Similarly, nagel2025seemingly provides evidence that high-dimensional methods may achieve success through multiple pathways that differ from theoretical predictions, suggesting that apparent complexity benefits often reflect simpler pattern-matching mechanisms. This paper extends the analysis by examining how practical implementation considerations interact with these theoretical mechanisms, providing a framework for understanding when apparent high-dimensional learning reflects genuine complexity benefits versus statistical artifacts.
This paper contributes to each of these literatures by providing the first unified theoretical analysis that connects sample complexity limitations, kernel approximation breakdown, and effective complexity bounds to explain the behavior of high-dimensional methods in financial prediction.
The remainder of the paper proceeds as follows. Section (ref) provides background on kernel methods and Random Fourier Features, explaining the theoretical foundations and practical implementations that motivate our analysis. Section (ref) establishes the theoretical framework and formalizes the theory-practice disconnect in RFF implementations. Section (ref) proves that within-sample standardization fundamentally breaks kernel approximation, explaining why claimed theoretical properties cannot hold in practice. Section (ref) establishes information-theoretic barriers to high-dimensional learning, showing that genuine complexity benefits are impossible under realistic financial conditions. Section (ref) provides numerical validation of the theoretical predictions. Section (ref) concludes. All technical details are relegated to a supplementary document containing Appendices (ref), (ref), and (ref).
Kernel regression provides a powerful non-parametric framework for capturing complex relationships by implicitly mapping inputs into high-dimensional feature spaces scholkopf2002learning. For a given kernel function $k(\cdot, \cdot)$, the solution is constrained by the Representer Theorem to lie within the span of the training data. Predictions thus take the form $\hat{f}(x) = \sum_{t=1}^T \alpha_t k(x, x_t)$, where the coefficients $\boldsymbol{\alpha}$ are learned. The Gaussian kernel, $k_G(x, x') = \exp(-\gamma^2 \|x - x'\|^2/2)$, is particularly attractive due to its universality and smooth, localized similarity structure.
To prevent overfitting and ensure numerical stability, kernel methods are often paired with ridge regularization, which involves solving the optimization problem $\min_{\boldsymbol{\alpha}} \|\mathbf{y} - \mathbf{K}\boldsymbol{\alpha}\|^2 + \lambda \|\boldsymbol{\alpha}\|^2$. This yields the stable closed-form solution $\boldsymbol{\alpha}^* = (\mathbf{K} + \lambda \mathbf{I})^{-1}\mathbf{y}$, where $\mathbf{K}$ is the $T \times T$ kernel matrix with entries $\mathbf{K}_{ij} = k(x_i, x_j)$. However, kernel methods face a fundamental computational bottleneck: constructing and inverting the kernel matrix requires $O(T^2)$ storage and $O(T^3)$ operations, respectively, making them impractical for large datasets.
RFF circumvent this limitation by constructing an explicit, finite-dimensional feature map $z(x) \in \mathbb{R}^P$ that approximates the kernel function rahimi2007random. The theoretical foundation relies on Bochner's theorem, which states that a continuous and shift-invariant kernel $k(x, x') = k(x - x')$ is the Fourier transform of a non-negative measure $p(\omega)$. This allows the kernel to be expressed as an expectation:
For the Gaussian kernel, this distribution $p(\omega)$ is a Gaussian $N(0, \gamma^2 I)$. RFF approximates this expectation using a Monte Carlo estimate with $P$ samples, forming features $z_i(x) = \sqrt{2} \cos(\omega_i^T x + b_i)$ with $\omega_i \sim N(0, \gamma^2 I)$ and $b_i \sim \text{Uniform}[0, 2\pi]$, such that $k_G(x, x') \approx z(x)^T z(x')$.
The crucial insight is that kernel ridge regression becomes a linear ridge regression problem in the RFF feature space: $\hat{f}(x) = z(x)^T \mathbf{w}^*$. This approach offers a dual advantage depending on the chosen dimensionality $P$:
This capability to flexibly trade computational cost for representational power makes RFF a powerful tool. However, as I demonstrate below, the standardization procedures universally employed in practical RFF implementations fundamentally violate the theoretical conditions required for this kernel approximation, creating a critical gap between theory and practice.
Consider predicting next-month returns using two macro predictors. We have training data:
While the high-dimensional expansion regime ($P \gg T$) promises substantial benefits, fundamental questions remain about when such complexity can genuinely improve prediction performance. In financial applications, where signals are weak and training samples are limited, information-theoretic constraints may render the promised gains illusory. This paper provides the first rigorous analysis of these limitations, showing that the combination of weak signal-to-noise ratios typical in finance and moderate sample sizes creates information-theoretic barriers that prevent reliable learning in high-dimensional regimes. Specifically, our minimax analysis reveals that for realistic financial parameters, reliable extraction of predictive signals requires sample sizes far exceeding those typically available, suggesting that apparent success in $P \gg T$ regimes must arise through mechanisms fundamentally different from the theoretical framework.
This section establishes the theoretical framework for analyzing high-dimensional prediction methods in finance. I first formalize the return prediction problem and core assumptions, then examine the critical disconnect between RFF theory and practical implementation that underlies my main results.
Consider the fundamental challenge of predicting asset returns using high-dimensional predictor information. We observe predictor vectors $x_t \in \mathbb{R}^K$ and subsequent returns $r_{t+1} \in \mathbb{R}$ for $t = 1, \ldots, T$, with the goal of learning a predictor $\hat{f}: \mathbb{R}^K \to \mathbb{R}$ that minimizes expected squared loss $\mathbb{E}[(r_{t+1} - \hat{f}(x_t))^2]$.
The challenge lies in the fundamental characteristics of financial prediction: signals are weak relative to noise, predictors exhibit complex persistence patterns, and available sample sizes are limited by the nonstationarity of financial markets. These features create a particularly demanding environment for high-dimensional learning methods, which I formalize through the following core assumptions.
This assumption captures the essential challenge of financial prediction: signals are weak (low $R^2$), predictors are persistent (like dividend yields and interest rate spreads), and noise dominates returns. The weak signal-to-noise ratio reflects the empirical reality that financial models typically explain only 1-5% of return variation welch2008comprehensive.
This assumption formalizes the RFF methodology as implemented in practice. It includes the crucial standardization step, which is commonly applied to ensure features have comparable scales for regularization and to improve numerical stability. Despite its ubiquity in practical applications, this step has not been analyzed in existing theoretical frameworks.
My framework, including Assumption (ref), uses the canonical RFF construction for its clarity and prominence in the literature. It is crucial, however, to analyze the specific variant used in kelly2024virtue. Their method generates a pair of features, $[\sin(\gamma\omega_i^\top x), \cos(\gamma\omega_i^\top x)]$, for each random projection $\omega_i$. I now provide a brief technical motivation for this choice and explain why my main results on the breakdown of kernel approximation remain fully relevant.
The theoretical goal of RFF is to approximate a shift-invariant kernel $k(x-x')$, which can be expressed via Bochner's theorem as the expectation $\mathbb{E}_\omega[\cos(\omega^\top(x-x'))]$. The standard RFF formulation, $z_i(x) = \sqrt{2}\cos(\omega_i^\top x + b_i)$, achieves this in expectation over the random phase $b_i$, as $\mathbb{E}_{b_i}[z_i(x)z_i(x')] = \cos(\omega_i^\top(x-x'))$. The variant used by kelly2024virtue achieves this same objective more directly. By the trigonometric identity $\cos(A-B) = \cos(A)\cos(B) + \sin(A)\sin(B)$, the inner product of their feature pair $[\sin(\omega_i^\top x), \cos(\omega_i^\top x)]$ with a corresponding pair at $x'$ is deterministically equal to $\cos(\omega_i^\top(x-x'))$. This removes the Monte Carlo randomness associated with the phase shift $b_i$, potentially leading to a lower-variance approximation of the kernel for a finite number of features $P$, as documented by sutherland2015error.
Despite the elegance of this formulation, its theoretical properties are invalidated by the same practical implementation step we analyze: data-dependent standardization. kelly2024virtue confirm that they standardize each feature by its standard deviation within the training sample. The deterministic relationship described above is therefore broken. The inner product of the standardized features becomes:
where the scaling factors $\hat{\sigma}_{s,i}^2$ and $\hat{\sigma}_{c,i}^2$ are calculated from the training data. The introduction of these data-dependent denominators fundamentally alters the function being approximated. The resulting kernel is no longer the shift-invariant target but a complex, data-dependent one. Thus, the kernel breakdown theorem, which hinges on this exact mechanism, remains directly applicable and essential for understanding the behavior of the method as implemented.
Having established the financial prediction environment, we now examine the fundamental disconnect between RFF theory and practical implementation. This disconnect is not merely technical—it undermines the entire theoretical justification for these methods.
The RFF methodology rahimi2007random provides rigorous theoretical foundations for kernel approximation. For target shift-invariant kernels $k(x,x') = k(x-x')$, the theory establishes that:
in probability, under the condition that features maintain their original distributional properties. This convergence enables kernel methods to be approximated through linear regression in the RFF space, with all the theoretical guarantees that kernel learning provides.
Every practical RFF implementation deviates from the theoretical setup in a seemingly minor but mathematically crucial way. To improve numerical stability and ensure comparable scales across features, practitioners standardize features using training sample statistics:
Since each $\hat{\sigma}_i$ depends on the specific observed training sample, different training windows produce different standardization factors, even for the same underlying data generating process. This training-set dependence fundamentally violates the theoretical requirement that $z_i(x)z_i(x') \to k_G(x,x')$ regardless of the training data used. The standardized empirical kernel:
no longer converges to the Gaussian kernel, but instead converges to a training-set dependent limit that violates the mathematical foundations required for kernel methods.
The standardization procedure $\hat{\sigma}_i^2 = \frac{1}{T}\sum_{t=1}^T z_i(x_t)^2$ creates features whose kernel approximation properties depend on the absolute positions of training points $\{x_t\}_{t=1}^T$, not just the relative distances $\|x-x'\|$ required for shift-invariant kernels. This fundamentally violates the mathematical foundation of RFF theory.
To establish my main theoretical results, I require three additional technical assumptions that ensure proper convergence and concentration properties.
This is a standard technical requirement in high-dimensional learning theory. The assumption of bounded support is mild and realistic for financial predictors, which naturally operate within a finite range (e.g., interest rates, dividend yields). This well-behaved nature of the inputs ensures the resulting random features also have finite moments. The condition that the feature covariance matrix $\Sigma_z$ has eigenvalues bounded away from zero and infinity ($c_z I_P \preceq \Sigma_z \preceq C_z I_P$) is crucial for mathematical stability. It guarantees the features are not perfectly redundant and have bounded variance, a necessary prerequisite for the concentration inequalities that form the basis of my non-asymptotic proofs.
This geometric condition on the training data is essential for the analysis of the standardized kernel. It enters our proofs through the small-ball probability estimates required to control the behavior of the random denominators $\hat{\sigma}_i^2$. The full-rank requirement ensures that the linear map from the random parameters to the feature arguments, $(\omega, b) \mapsto (2\omega^\top x_t + 2b)_{t=1}^T$, is locally bi-Lipschitz. This enables the use of powerful geometric tools to establish exponential small-ball bounds, which are critical for proving the convergence of the standardized kernel and the finiteness of key expectations. In typical financial applications where predictors are continuous macroeconomic variables, any exact affine dependence among the training points has Lebesgue measure zero, making this a mild assumption.
This is a standard concentration assumption in high-dimensional learning theory and is automatically satisfied in all practical RFF implementations. Since each feature $z_i(x) = \sqrt{2}\cos(\omega_i^\top x + b_i)$ is bounded in $[-\sqrt{2}, \sqrt{2}]$, any linear combination $u^\top z(x)$ is also bounded. By Hoeffding's lemma hoeffding1963probability, any bounded random variable is sub-Gaussian. This condition is essential for applying uniform convergence results (e.g., matrix Bernstein inequalities) to control the difference between the empirical feature covariance matrix and its population counterpart. These non-asymptotic bounds are fundamental to my sample complexity analysis and allow me to derive results that hold with high probability for finite samples.
Having established the fundamental theory-practice disconnect in RFF implementations, the following sections provide rigorous mathematical analysis of its implications. Section (ref) proves that standardization fundamentally breaks kernel approximation properties (Theorem (ref)), while Section (ref) establishes information-theoretic limits showing that genuine high-dimensional learning is impossible under realistic financial conditions (Theorems (ref) and (ref)). Together, these results explain why apparent "virtue of complexity" successes must arise through mechanisms fundamentally different from the theoretical framework.
Having established the theory-practice disconnect in Section (ref), I now prove rigorously that standardization fundamentally alters the kernel approximation properties that justify RFF methods.
The proof proceeds in two steps. First, I establish that the standardized kernel function $h(\omega,b)$ has finite expectation despite the random denominator, enabling application of the strong law of large numbers for part (a). This requires controlling the probability that the empirical variance $\hat{\sigma}^2$ becomes arbitrarily small, which I achieve through geometric analysis exploiting the full-rank condition. Second, I prove training-set dependence by explicit construction: scaling any training point $x_j \mapsto \alpha x_j$ with $\alpha > 1$ yields different limiting kernels, establishing that $k_{\text{std}}^* \neq k_G$. The complete technical proof appears in Appendix (ref).
To understand the implications of Theorem (ref), I examine precisely how standardization violates the conditions under which RFF theory operates. rahimi2007random prove convergence to the Gaussian kernel under two essential conditions: distributional alignment of frequencies $\omega_i$ and phases $b_i$ with the target kernel's Fourier transform, and preservation of the prescribed scaling $z_i(x) = \sqrt{2}\cos(\omega_i^\top x + b_i)$.
Standardization $\tilde{z}_i(x) = z_i(x)/\hat{\sigma}_i$ systematically violates both conditions. The original features have theoretical properties derived from specified distributions of $\omega_i$ and $b_i$, but the standardization factor $1/\hat{\sigma}_i$ varies with the training set, altering the effective distribution in a data-dependent manner. The expectation $\mathbb{E}[\tilde{z}_i(x)\tilde{z}_i(x')]$ now depends on $\hat{\sigma}_i$, disrupting the direct mapping to $k_G(x,x')$. Additionally, the fixed scaling $\sqrt{2}$ that ensures correct kernel approximation is replaced by a random, sample-dependent factor, breaking the fundamental relationship between feature products and kernel values.
These modifications have important mathematical implications. The standardized features yield an empirical kernel that converges to $k_{\text{std}}^*(x,x'|\mathcal{T})$, which is training-set dependent rather than depending only on $\|x-x'\|$ like the Gaussian kernel. The resulting kernel is not shift-invariant since $\hat{\sigma}_i$ reflects absolute positions of training points, and shifting the data changes $\hat{\sigma}_i$. This creates temporal non-stationarity as kernel properties change when training windows roll forward.
Theorem (ref) resolves the fundamental puzzles in high-dimensional financial prediction by revealing that claimed theoretical properties simply do not hold in practice. kelly2024virtue develop their theoretical analysis assuming RFF converge to Gaussian kernels. Their random matrix theory characterization, effective complexity bounds, and optimal shrinkage formula all depend critically on this convergence. However, their empirical implementation employs standardization, which fundamentally alters the convergence properties, creating a notable difference between theory and practice.
With modified kernel structure, methods may perform learning that differs from the theoretical framework, potentially involving pattern-matching mechanisms based on training-sample dependent similarity measures. The standardized kernel creates similarity measures based on training-sample dependent weights rather than genuine predictor relationships. This explains nagel2025seemingly empirical finding that high-complexity methods produce volatility-timed momentum strategies regardless of underlying data properties. The broken kernel structure makes the theoretically predicted learning more challenging, leading methods to weight returns based on alternative similarity measures within the training window.
The apparent virtue of complexity may arise through different mechanisms than originally theorized. Their method cannot achieve its theoretical properties due to standardization, so any success must arise through alternative mechanisms. This resolves the central puzzle of how methods claiming to harness thousands of parameters succeed with tiny training samples: they may operate through mechanisms that differ from the high-dimensional framework, potentially involving simpler pattern-matching approaches that happen to work in specific market conditions.
The kernel approximation breakdown in Section (ref) reveals that methods cannot achieve their claimed theoretical properties. This section establishes that even if this breakdown were corrected, fundamental information-theoretic barriers would still prevent genuine high-dimensional learning in financial applications. These results explain why methods must rely on the mechanical pattern matching that emerges from broken kernel structures.
I begin by clarifying our notion of complexity. Throughout this analysis, we consider the high-dimensional or overparameterized regime where the number of features substantially exceeds the sample size, i.e., $P \gg T$.
The following results establish fundamental barriers to learning in this regime through two complementary approaches: exponential bounds that apply broadly but may be loose, and polynomial bounds that are tighter but require additional technical conditions.
Both lower bounds characterize the minimax risk:
I conduct the minimax analysis in the finite–dimensional random-feature space by representing each candidate predictor as \(f_{\omega}(x)=\omega^{\top}z(x)\). This inner-product form serves three key purposes. First, tt mirrors the reproducing-kernel expansion of Gaussian-kernel regression: \(z(x)\) collects the Random Fourier Features and \(\omega\) specifies their linear weights, keeping the setup directly comparable to the kernel methods analysed by kelly2024virtue. Second, expressing the function class through the constrained parameter vector \(\omega\in\mathbb{B}_{2}^{P}(B)\) converts an infinite-dimensional functional problem into a finite linear one, enabling PAC- and information-theoretic risk bounds via standard packing, Kullback-Leibler (KL), and Fano arguments. Third, the factorisation \(\omega^{\top}z(x)\) neatly separates the learner-controlled parameters (\(\omega\)) from the data-driven randomness (\(z(x)\)), a separation that is crucial for deriving worst-case (minimax) prediction-error lower bounds while allowing probabilistic assumptions on the covariate distribution.
Assumption (ref) specifies that excess returns satisfy \(r_{t+1}=f^{\ast}(x_t)+\varepsilon_{t+1}\) with (i) a square-integrable signal obeying \(\mathbb E[f^{\ast}(x)^2]\le B^{2}\) and (ii) mean–zero noise of variance \(\sigma^{2}\). By further positing that the true signal lies in the random-feature class, namely \(f^{\ast}(x)=\omega^{\star\top}z(x)\) for some \(\omega^{\star}\in\mathbb B_{2}^{P}(B)\), we impose no extra restriction beyond Assumption (ref)(a): the norm bound on \(\omega^{\star}\) guarantees \(\mathbb E[(\omega^{\star\top}z(x))^{2}]\le B^{2}\), so the squared-moment condition is preserved. Hence the target function used in the minimax analysis is fully compatible with the return-generating environment outlined in Assumption (ref), while providing a concrete parametric structure that makes the subsequent risk bounds tractable.
Expression (ref) captures fundamental learning difficulty through its nested structure. The infimum over $\hat{f}_T$ represents optimization over all possible estimators, including OLS, ridge regression, neural networks, and any other conceivable method. The supremum over $\|w\|_2 \leq B$ corresponds to an adversarial choice of the hardest parameter to estimate within the bounded parameter space. The expectation $\mathbb{E}_{x,D_T,\epsilon}$ averages over all randomness in the learning problem.
The expectation encompasses three sources of randomness that characterize the learning environment. Training data $D_T = \{(x_t, r_t)\}_{t=1}^T$ represents different possible datasets that could be observed. The query point $x$ corresponds to test inputs where performance is evaluated. Noise $\epsilon$ captures irreducible randomness in observations. This framework provides information-theoretic limits where no estimator, regardless of computational complexity, can achieve better performance than these bounds in the specified regime.
The proof employs a minimax argument with Fano's inequality. I construct a $2\delta$-packing $\{w_1, \ldots, w_M\} \subset B_2^P(B)$ with $M = (B/(2\delta))^P$ well-separated parameters. The KL divergence between corresponding data distributions satisfies $\mathrm{KL}(P_j \| P_\ell) \leq \frac{2TC_z B^2}{\sigma^2}$. Fano's inequality implies any decoder has error probability $\Pr[\hat{J} \neq J] \geq 1/2$. Since low estimation risk would enable perfect identification, I obtain $\mathbb{E}[(\hat{f}_T(x) - f_J(x))^2] \geq c_z\delta^2$. Optimizing $\delta$ yields the exponential bound.
Theorem (ref) applies directly to machine learning methods employing RFF as implemented in practice. The framework covers the complete pipeline where random feature weights $\{\omega_i, b_i\}_{i=1}^P$ are drawn from specified distributions, standardization procedures are applied for numerical stability, and learning proceeds over the linear-in-features function class using any estimation method. The bounds establish information-theoretic impossibility in complementary forms: expectation bounds averaged over all possible feature realizations, and high-probability bounds for most individual feature draws.
The exponential lower bound of Theorem (ref) reveals the possibility of an information–theoretic barrier, but its dependence on $P$ is intentionally pessimistic: it stems from a coarse packing argument that ignores the finer geometry of the random-feature covariance. By exploiting that geometry—specifically the sub-Gaussian eigenvalue decay in $\Sigma_{z}$ (Assumption (ref))—we can tighten the analysis and replace the exponential dependence with a polynomial one, as formalised in the next theorem.
The proof uses canonical basis packing with refined concentration analysis. I construct $M = P + 1$ functions using $w_0 = 0$ and $w_j = \delta e_j$ where $\delta = \min\{B/4, \sigma/(4\sqrt{TC_z \log P})\}$. The population covariance $\Sigma_z \succeq c_z I_P$ ensures separation $\|f_j - f_\ell\|_{L^2(\mu)}^2 \geq 2c_z\delta^2$. Fano's inequality with error probability $\geq 1/2$ yields $\mathbb{E}[(\hat{f}_T(x) - f_J(x))^2] \geq \frac{c_z}{4}\delta^2$, producing the polynomial bound.
The in-expectation bounds (parts (a) of Theorems (ref) and (ref)) apply directly to the high-dimensional regime $P \gg T$ and provide fundamental limits on learning performance averaged over all possible feature realizations and datasets. The high-probability bounds (parts (b)) require $T \geq C_0 P$ for technical reasons related to matrix concentration, making them inapplicable when $P \gg T$. However, the in-expectation bounds suffice to establish information-theoretic impossibility in practical high-dimensional scenarios.
The two bounds offer complementary characterizations of learning difficulty. The exponential bound applies broadly but may be loose when the exponent is large, with key parameter $TC_zB^2/(P\sigma^2)$ and limited practical relevance. The polynomial bound provides sharp characterization through the complexity ratio $\log P / T$ and offers high practical relevance. For practical applications, the polynomial bound provides the more meaningful characterization since the complexity ratio $\log P / T$ offers a concrete threshold that directly connects problem parameters to learning feasibility.
While inapplicable to $P \gg T$, high-probability bounds serve important purposes in moderate-dimensional settings where $T \geq C_0 P$. They enable principled algorithm design with known failure probabilities, provide non-asymptotic characterizations that bridge theory and practice, and ensure empirical feature covariance concentrates around its population counterpart, preventing pathological ill-conditioning.
This section provides comprehensive empirical validation of Theorem (ref) through systematic parameter exploration across the entire space of practical RFF implementations. My experimental design spans realistic financial prediction scenarios, testing whether standardization preserves the Gaussian kernel approximation properties that underlie existing theoretical frameworks. The results provide definitive evidence that standardization fundamentally breaks RFF convergence properties, confirming that methods cannot achieve their claimed theoretical guarantees in practice.
I generate realistic financial predictor data following the autoregressive structure typical of macroeconomic variables used in return prediction. For each parameter combination $(T, K)$, I construct predictor matrices $X \in \mathbb{R}^{T \times K}$ where:
The persistence parameters $\Phi = \text{diag}(\phi_1, \ldots, \phi_K)$ are drawn from the range $[0.82, 0.98]$ to match the high persistence of dividend yields, interest rates, and other financial predictors welch2008comprehensive. The correlation structure $\Sigma_u = \rho \mathbf{1}\mathbf{1}^T + (1-\rho)I_K$ with $\rho = 0.1$ captures modest cross-correlation among predictors.
Random Fourier Features are constructed as $z_i(x) = \sqrt{2}\cos(\omega_i^T x + b_i)$ where $\omega_i \sim \mathcal{N}(0, \gamma^2 I_K)$ and $b_i \sim \text{Uniform}[0, 2\pi]$. Standardization is applied as $\tilde{z}_i(x) = z_i(x)/\hat{\sigma}_i$ where $\hat{\sigma}_i^2 = T^{-1}\sum_{t=1}^T z_i(x_t)^2$ following universal practice in RFF implementations.
My parameter exploration covers the comprehensive space:
The primary objective is to test whether standardization preserves the convergence $k^{(P)}_{\text{std}}(x,x') \xrightarrow{P \to \infty} k_G(x,x')$ established in rahimi2007random. Under the null hypothesis that standardization has no effect, both standard and standardized RFF should exhibit identical convergence properties and error distributions. Theorem (ref) predicts systematic breakdown with training-set dependent limits $k^*_{\text{std}}(x,x'|\mathcal{T}) \neq k_G(x,x')$.
I conduct 1,000 independent trials per parameter combination, generating fresh training data, RFF weights, and query points for each trial. This provides robust statistical power to detect systematic effects across the parameter space while controlling for random variations in specific realizations.
My empirical analysis employs four complementary approaches to characterize the extent and nature of kernel approximation breakdown. I begin by examining convergence properties through mean absolute error $|k^{(P)}(x,x') - k_G(x,x')|$ between empirical and true Gaussian kernels, tracking how approximation quality evolves as $P \to \infty$. This directly tests whether standardized features preserve the fundamental convergence properties established in rahimi2007random.
To quantify the systematic nature of performance deterioration, I construct degradation factors as the ratio $\mathbb{E}[|\text{error}_{\text{standardized}}|]/\mathbb{E}[|\text{error}_{\text{standard}}|]$ across matched parameter combinations. Values exceeding unity indicate that standardization worsens kernel approximation, while larger ratios represent more severe breakdown. This metric provides a scale-invariant measure of standardization effects that facilitates comparison across different parameter regimes.
Statistical significance is assessed through Kolmogorov-Smirnov two-sample tests comparing error distributions between standard and standardized RFF implementations. Under the null hypothesis that standardization preserves distributional properties, these tests should yield non-significant results. Systematic rejection of this null across parameter combinations provides evidence that standardization fundamentally alters the mathematical behavior of RFF methods beyond what could arise from random variation.
Finally, I conduct comprehensive parameter sensitivity analysis to identify the conditions under which breakdown effects are most pronounced. Heatmap visualizations reveal how degradation severity depends on $(P,T,\gamma,K)$ combinations, enabling us to characterize the parameter regimes where theoretical guarantees are most severely compromised. This analysis is particularly relevant for understanding the implications for existing empirical studies that employ specific parameter configurations.
Figure (ref) provides decisive evidence of convergence breakdown. Standard RFF (blue circles) exhibit the theoretically predicted $P^{-1/2}$ convergence rate, with mean absolute error declining from $\approx 0.06$ at $P=100$ to $\approx 0.003$ at $P=20,000$. This confirms that unstandardized features preserve Gaussian kernel approximation properties.
In stark contrast, standardized RFF (red squares) completely fail to converge, plateauing around $0.02$-$0.03$ mean error regardless of $P$. For large $P$, standardized features are $\mathbf{6 \times}$ worse than standard RFF, demonstrating that additional features provide no approximation benefit when standardization is applied. This plateau behavior directly validates Theorem (ref)'s prediction that standardized features converge to training-set dependent limits rather than the target Gaussian kernel.
Figure (ref) reveals that breakdown occurs universally across all parameter combinations, with no regime where standardization preserves kernel properties. The degradation patterns exhibit clear economic intuition and align closely with the theoretical mechanisms underlying Theorem (ref).
The most pronounced effects emerge along the feature dimension, where degradation increases dramatically with $P$, ranging from 1.2 times at $P=100$ to 6.0 times at $P=20,000$. This escalating pattern reflects the cumulative nature of standardization artifacts: as more features undergo within-sample standardization, the collective distortion of kernel approximation properties intensifies. Each additional standardized feature contributes random scaling factors that compound to produce increasingly severe departures from the target Gaussian kernel.
Sample size effects provide particularly compelling evidence for the breakdown mechanism. Smaller training windows exhibit severe degradation, reaching 41.6 times deterioration for $T=6$ months. This extreme sensitivity to sample size occurs because standardization relies on empirical variance estimates $\hat{\sigma}_i^2$ that become increasingly unreliable with limited data. When training windows shrink to the 6-12 month range typical in financial applications, these variance estimates introduce substantial noise that fundamentally alters the scaling relationships required for kernel convergence. The magnitude of this effect—exceeding 40 times degradation in realistic scenarios—demonstrates that standardization can completely overwhelm any approximation benefits from additional features.
Kernel bandwidth parameters reveal additional structure in the breakdown pattern. Low bandwidth values ($\gamma=0.5$) produce 12.8 times degradation, while higher bandwidths stabilize around 3.1 times deterioration. This occurs because tighter kernels, which decay more rapidly with distance, are inherently more sensitive to the scaling perturbations introduced by standardization. Small changes in feature magnitudes translate into disproportionately large changes in kernel values when the bandwidth is narrow, amplifying the distortions created by training-set dependent scaling factors.
In contrast, input dimension effects remain remarkably stable, with degradation ranging only between 3.1 and 4.6 times across $K \in [5,30]$. This stability confirms that breakdown stems primarily from the standardization procedure itself rather than the complexity of the underlying input space. Whether using 5 or 30 predictor variables, the fundamental mathematical properties of standardized RFF remain equally compromised, suggesting that the kernel approximation failure is intrinsic to the standardization mechanism rather than an artifact of high-dimensional inputs.
Figure (ref) provides detailed parameter sensitivity analysis through degradation factor heatmaps. The $(P,T)$ interaction reveals that combinations typical in financial applications—such as $P \geq 5,000$ features with $T \leq 12$ months—produce degradation factors exceeding $3\times$. This directly impacts methods like kelly2024virtue using $P=12,000$ and $T=12$.
The $(P,\gamma)$ interaction shows that standardization effects compound: high complexity ($P \geq 10,000$) combined with tight kernels ($\gamma \leq 1.0$) yields degradation exceeding $10\times$. These parameter ranges are commonly employed in high-dimensional return prediction, suggesting widespread applicability of my breakdown results.
The error distributions between standard and standardized RFF are fundamentally different across the entire parameter space, providing strong statistical evidence against the null hypothesis that standardization preserves kernel approximation properties. Figure (ref) presents Kolmogorov-Smirnov test statistics that consistently exceed 0.5 across most parameter combinations, with many approaching the theoretical maximum of 1.0. Such large test statistics indicate that the cumulative distribution functions of standard and standardized RFF errors diverge substantially, ruling out the possibility that observed differences arise from sampling variation.
The statistical evidence is most compelling in parameter regimes commonly employed in financial applications. For high feature counts ($P \geq 5,000$), KS statistics approach 0.9, while short training windows ($T \leq 12$) yield statistics near 1.0. These values correspond to p-values that are effectively zero, providing overwhelming evidence to reject the null hypothesis of distributional equivalence. The magnitude of these test statistics exceeds typical significance thresholds by orders of magnitude, establishing statistical significance that is both robust and economically meaningful.
The systematic pattern of large KS statistics across parameter combinations demonstrates that the breakdown identified in Theorem (ref) is not confined to specific implementation choices or edge cases. Instead, the distributional differences persist universally across realistic parameter ranges, indicating that standardization fundamentally alters the stochastic properties of RFF approximation errors. This statistical evidence complements the degradation factor analysis by confirming that the observed differences represent genuine distributional shifts rather than changes in central tendency alone.
These results establish that standardization creates systematic, statistically significant alterations to RFF behavior that cannot be attributed to random variation, specific parameter selections, or implementation artifacts. The universality and magnitude of the statistical evidence provide definitive support for the conclusion that practical RFF implementations cannot achieve the theoretical kernel approximation properties that justify their use in high-dimensional prediction problems.
Figure (ref) provides empirical validation of Theorem (ref)'s central prediction that within-sample standardization fundamentally alters Random Fourier Features convergence properties. The analysis compares three distinct convergence behaviors across varying feature dimensions $P \in [100, 500, 1000, 2500, 5000, 12000]$:
The blue line demonstrates that standard (non-standardized) RFF achieve the theoretical convergence rate $P^{-1/2}$ to the Gaussian kernel $k_G(x,x') = \exp(-\gamma^2\|x-x'\|^2/2)$, validating the foundational result of rahimi2007random. The convergence follows the expected Monte Carlo rate, with mean absolute error decreasing from approximately $0.06$ at $P=100$ to $0.005$ at $P=12{,}000$.
The red line reveals the fundamental breakdown predicted by Theorem (ref): standardized RFF fail to converge to the Gaussian kernel, instead exhibiting slower convergence with substantially higher errors. At $P=12{,}000$, the error remains above $0.02$---four times larger than the standard case---demonstrating that standardization prevents achievement of the theoretical guarantees.
Most importantly, the green line confirms Theorem (ref)'s constructive prediction by showing that standardized RFF do converge to the modified limit $k^*_{\text{std}}(x,x'|T)$. This convergence exhibits the canonical $P^{-1/2}$ rate, reaching error levels below $0.015$ at $P=12{,}000$, thereby validating my theoretical characterization of the standardized limit.
My empirical validation employs the sample standard deviation standardization actually used in practice:
rather than the simpler RMS normalization $\hat{\sigma}^2_i = \frac{1}{T}\sum_{t=1}^T z_i^2(x_t)$ that might be assumed theoretically. This distinction strengthens rather than weakens my validation for two crucial reasons.
First, Theorem (ref)'s fundamental insight---that any reasonable standardization procedure breaks Gaussian kernel convergence and creates training-set dependence---remains intact regardless of the specific standardization formula. The theorem establishes that standardized features converge to some training-set dependent limit $k^*_{\text{std}} \neq k_G$, with the exact form depending on implementation details.
Second, testing against the actual standardization procedure used in practical implementation ensures that my theoretical predictions match real-world behavior. The fact that standardized RFF converge to the correctly computed $k^*_{\text{std}}$ rather than to $k_G$ provides the strongest possible validation: my theory successfully predicts the behavior of methods as actually implemented, not merely as idealized.
The convergence patterns thus confirm all key predictions of Theorem (ref): standardization breaks the foundational convergence guarantee of RFF theory, creates training-set dependent kernels that violate shift-invariance, and produces systematic errors that persist even with large feature counts. These findings validate my theoretical framework while highlighting the critical importance of analyzing methods as actually implemented rather than as theoretically idealized.
My results provide definitive empirical validation of Theorem (ref) across the entire parameter space relevant for financial applications. The universal nature of degradation—ranging from modest $1.2\times$ effects to extreme $40\times$ breakdown—demonstrates that standardization fundamentally alters RFF convergence properties regardless of implementation details.
Notably, parameter combinations employed by leading studies exhibit substantial degradation: kelly2024virtue's configuration ($P=12,000$, $T=12$, $\gamma=2.0$) falls in the $3$-$6\times$ degradation range, while more extreme combinations approach $10\times$ or higher degradation. This suggests that empirical successes documented in the literature cannot arise from the theoretical kernel learning mechanisms that justify these methods.
The systematic nature of these effects, combined with their large magnitudes, supports the conclusion that alternative explanations—such as the mechanical pattern matching identified by nagel2025seemingly—are required to understand why high-dimensional RFF methods achieve predictive success despite fundamental theoretical breakdown.
My theoretical analysis has established information-theoretic bounds on learning performance. To assess their practical relevance, we focus on the polynomial minimax lower bound from Theorem (ref). Specifically, our analysis relies on the in-expectation bound (part a), as it applies directly to the high-dimensional regime ($P \gg T$) that is characteristic of modern financial applications. To operationalize this bound, we now calibrate its key parameters—\(\tilde c, B^{2}, \sigma^{2}, c_{z},\) and \(C_{z}\)—using empirically defensible values. We ground this calibration in the challenging setting of kelly2024virtue, which uses \(K=15\) predictors to generate \(P=12{,}000\) features for prediction over a \(T=12\) month training window.
\paragraph{Noise Variance (\(\sigma^{2}\)).} The model's noise variance \(\sigma^2\) is formally the conditional variance of returns, \(E[\epsilon_{t+1}^2|x_t]\). For calibration, I begin with the total unconditional variance of returns, \(\mathrm{Var}(r)\). This choice is motivated by the characteristically weak nature of financial signals, which ensures that total variance is dominated by the noise component. The total excess return on the U.S. equity market exhibits an annualised volatility in the 14%--17% range CampbellLoMacKinlay1997, welch2008comprehensive, implying a monthly standard deviation of approximately \(0.047\) and thus a total variance of \(\mathrm{Var}(r) \approx 2.2\times10^{-3}\). Since \(\mathrm{Var}(r) = B^2 + \sigma^2\) and, as I show next, \(B^2\) is an order of magnitude smaller, I use the total variance as a close and robust proxy for the noise variance, setting \(\sigma^2 \approx 2.2\times10^{-3}\).
\paragraph{Signal Power (\(B^{2}\)).} Assumption (ref)(a) bounds the variance of the predictive signal \(f^{\ast}(x)\) by \(B^{2}\). The fraction of total return variance attributable to this signal is the population R-squared, \(R^2 = B^2 / \mathrm{Var}(r)\). Empirical \(R^{2}\) values for monthly return forecasts using macroeconomic predictors are typically in the 1%--5% range welch2008comprehensive, nagel2025seemingly. Using our calibrated total variance \(\mathrm{Var}(r) \approx 2.2\times10^{-3}\), this implies a range for the signal variance:
I adopt \(B^{2}=5\times10^{-5}\) (implying \(R^{2}\approx2.3\%\)) as a representative benchmark for a realistic signal strength.
\paragraph{Feature-Covariance Bounds (\(c_{z},\,C_{z}\)).} For the vanilla RFF map \(z_{i}(x)=\sqrt2\cos(\omega_{i}^{\top}x+b_{i})\), each coordinate has unit variance in expectation, such that \(\Sigma_{z}\approx I_{P}\). Standard concentration of measure results for sub-Gaussian random matrices Vershynin2018 imply that for \(K=15\), the spectral bounds on the empirical covariance matrix will be tight with high probability. We set a baseline scenario of: \[ 0.8\;\lesssim\;c_{z}\;\le\;1, \qquad 1\;\le\;C_{z}\;\lesssim\;1.2, \] which corresponds to an eigenvalue condition number below \(1.5\). To stress-test our conclusions, we also consider a “collinear’’ setting of \(c_{z}=0.5\) and \(C_{z}=2\).
\paragraph{The Universal Constant (\(\tilde{c}\)).} By definition, \(\tilde{c}=c_{z}/(128\,C_{z})\). The parameter ranges established above yield \(\tilde{c} \in [0.005, 0.008]\) in the baseline scenario and \(\tilde{c}\approx0.002\) in the collinear stress test. We therefore view the band \(\tilde{c}\in[0.002,\,0.008]\) as both realistic and defensible for practical financial applications.
My quantitative calibration allows us to diagnose the fundamental nature of the learning problem confronting return prediction models. The polynomial minimax bound on risk is determined by the minimum of two terms: one related to the signal power (\(B^2\)) and one related to model complexity and data scarcity (\((C_z^{-1}\sigma^2/T)\log P\)). The critical sample size, \(T_{\mathrm{crit}}\), defines the crossover point between these two regimes: \[ T_{\mathrm{crit}} = \frac{C_{z}^{-1}\sigma^{2}}{B^{2}}\log P. \] The value of the operational sample size, \(T\), relative to this theoretical threshold determines the primary barrier to learning:
Applying our calibrated parameters to this framework reveals a striking result. For the high-dimensional design in kelly2024virtue, we find: \[
\] This threshold is remarkably insensitive to the nominal feature dimension due to its logarithmic dependence on \(P\). For instance, a standard machine learning setup with \(P=1,000\) features Gu2020ML still yields a \(T_{\mathrm{crit}}\) of approximately 276 months (\(\approx 23\) years). Even for a traditional low-dimensional econometric model with just \(P=15\) features, the critical sample size remains substantial at \(T_{\mathrm{crit}}\approx108\) months (\(\approx 9\) years).
To underscore the robustness of these findings, Figure (ref) illustrates the sensitivity of the critical sample size, \(T_{\mathrm{crit}}\), to the signal-to-noise conditions. This plot shows that \(T_{\mathrm{crit}}\) is acutely sensitive to the signal strength, consistent with the \(B^2\) term in the denominator of the formula. For a high-dimensional model (\(P=12,000\)), the required sample size ranges from 188 months for a strong signal (\(R^2 \approx 5\%\)) to an infeasible 1,875 months for a very weak signal (\(R^2 \approx 0.45\%\)). Figure (ref) illustrates the sensitivity of \(T_{\mathrm{crit}}\) to the conditional variance noise values. As expected, this plot shows that \(T_{\mathrm{crit}}\) is also directly proportional to the level of noise, \(\sigma^2\). The required sample window for the \(P=12,000\) model increases from 282 months in a low-noise environment to 564 months in a high-noise one. Together, these plots visually confirm that the \(T \ll T_{\mathrm{crit}}\) condition is not a borderline case but a robust feature across all empirically plausible parameterisations, making the signal-limited regime a pervasive challenge for financial return prediction.
The implication of this finding is profound. Since typical applications in the literature employ short estimation windows (e.g., \(T=12\) months), they operate deep within the signal-limited regime (\(T \ll T_{\mathrm{crit}}\)). This holds true regardless of whether the model is low-dimensional or high-dimensional. Consequently, the minimax lower bound on risk simplifies to \(\tilde{c}B^2\), a performance floor that is independent of the number of features \(P\) or the sample size \(T\).
This re-frames our understanding of the role of complex models in finance. The central promise of high-dimensional methods—their ability to process vast feature sets to overcome the curse of dimensionality—is rendered moot. The fundamental barrier to predictive accuracy in this domain is not a dimensionality problem that can be solved with more features; it is an economic problem rooted in the inherent weakness of the predictive signal. This suggests that the documented success of these models likely arises not from genuine high-dimensional learning. Ultimately, our analysis indicates that the frontier for improving financial prediction lies not in building ever-larger models, but in either identifying stronger economic signals (increasing \(B^2\)) or developing methods specifically robust to the challenges of the signal-limited, short-sample regime.
This paper resolves fundamental puzzles in high-dimensional financial prediction by providing rigorous theoretical foundations that explain when and why complex machine learning methods succeed or fail. My analysis contributes three key results that together clarify the apparent contradictions between theoretical claims and empirical mechanisms in recent literature.
First, I prove that within-sample standardization—employed in every practical Random Fourier Features implementation—fundamentally breaks the kernel approximation that underlies existing theoretical frameworks. This breakdown explains why methods operate under different conditions than theoretical assumptions and must rely on simpler mechanisms than advertised.
Second, I establish sharp sample complexity bounds showing that reliable extraction of weak financial signals requires sample sizes and signal strengths far exceeding those available in typical applications. These information-theoretic limits demonstrate that apparent high-dimensional learning often reflects mechanical pattern matching rather than genuine complexity benefits.
Third, I derive precise learning thresholds that characterize the boundary between learnable and unlearnable regimes, providing practitioners with concrete tools for evaluating when available data suffices for reliable prediction versus when apparent success arises through statistical artifacts.
These results explain why methods claiming sophisticated high-dimensional learning often succeed through simple volatility-timed momentum strategies operating in low-dimensional spaces bounded by sample size. Rather than discouraging complex methods, my findings provide a framework for distinguishing genuine learning from mechanical artifacts and understanding what such methods actually accomplish.
The theoretical insights extend beyond the specific methods analyzed, offering guidance for evaluating any high-dimensional approach in challenging prediction environments. As machine learning continues to transform finance, rigorous theoretical understanding remains essential for distinguishing genuine advances from statistical mirages and enabling more effective application of these powerful but often misunderstood techniques.