Karun Adusumilli, Maximilian Kasy, Ashia Wilson
arXiv 20 Mar 2026 · Mathematics — Statistics Theory
arXiv:2603.20388 · PDF · OpenAlex · Extracted main text
We derive the asymptotic risk function of regularized empirical risk minimization (ERM) estimators tuned by $n$-fold cross-validation (CV). The out-of-sample prediction loss of such estimators converges in distribution to the squared-error loss (risk function) of shrinkage estimators in the normal means model, tuned by Stein's unbiased risk estimate (SURE). This risk function provides a more fine-grained picture of predictive performance than uniform bounds on worst-case regret, which are common in learning theory: it quantifies how risk varies with the true parameter. As key intermediate steps, we show that (i) $n$-fold CV converges uniformly to SURE, and (ii) while SURE typically has multiple local minima, its global minimum is generically well separated. Well-separation ensures that uniform convergence of CV to SURE translates into convergence of the tuning parameter chosen by CV to that chosen by SURE.
appendix boundary found by appendix_command · 31% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Wilson, A., Kasy, M., and Mackey, L (2020) Approximate cross-validation: Guarantees for model assessment and selection self | 0.874 | 6 | 3 | 67% |
| 2 | Stein, C. M (1981) Estimation of the mean of a multivariate normal distribution | 0.811 | 4 | 2 | 100% |
| 3 | van der Vaart, A. W (2000) Asymptotic statistics | 0.669 | 10 | 5 | 30% |
| 4 | Efron, B (2004) The estimation of prediction error: covariance penalties and cross-validation | 0.644 | 2 | 2 | 100% |
| 5 | James, W. and Stein, C (1961) Estimation with quadratic loss | 0.511 | 2 | 1 | 100% |
| 6 | Donoho, D. L. and Johnstone, I. M (1995) Adapting to unknown smoothness via wavelet shrinkage | 0.405 | 1 | 1 | 100% |
| 7 | Golub, G. H., Heath, M., and Wahba, G (1979) Generalized cross-validation as a method for choosing a good ridge parameter | 0.405 | 1 | 1 | 100% |
| 8 | Hoerl, A. E. and Kennard, R. W (1970) Ridge regression: Biased estimation for nonorthogonal problems | 0.405 | 1 | 1 | 100% |
| 9 | Li, K.-C (1987) Asymptotic optimality for $c_p$, $c_l$, cross-validation and generalized cross-validation: Discrete index set | 0.405 | 1 | 1 | 100% |
| 10 | Mallows, C. L (1973) Some comments on $c_p$ | 0.405 | 1 | 1 | 100% |
Showing the top 10 of 20 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.
| Citing paper | Intensity | Mentions | Sections | |
|---|---|---|---|---|
| 1 | Assumption-Lean Shrinkage and Model Averaging for Spatial Parameters | 0.405 | 1 | 1 |