arXiv 3 May 2026 · Mathematics — Statistics Theory
arXiv:2605.02070 · PDF · DOI · OpenAlex · Extracted main text
A central problem in the theory of empirical Bayes is to control the regret (excess risk) of a learned Bayes rule by the Hellinger distance between the estimated and true marginal densities. In the normal means model, the classical result of Jiang and Zhang (2009, Annals of Statistics) achieves this only after regularizing the Bayes rule and incurs an extraneous cubic logarithmic factor through a delicate recursive argument. This paper introduces a new technique, based on polynomial approximation and Bernstein-type inequalities for weighted $L_2$ norms, that bounds the unregularized regret directly. The method is conceptually simpler and yields sharper, sometimes optimal, regret bounds. For compactly supported priors, we prove the sharp bound that the regret is at most $O(ε^2 \log(1/ε)/\log\log(1/ε))$, where $ε$ is the Hellinger distance between the marginal densities. The same method also extends to priors with exponential tails. Conversely, we show that regularization is genuinely necessary for heavy-tailed priors under only bounded moment assumptions. As a statistical consequence, we obtain improved regret bounds for the nonparametric maximum likelihood estimator.
appendix boundary found by appendix_command · 68% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Jiang, Wenhua and Zhang, Cun-Hui (2009) General Maximum Likelihood Empirical Bayes Estimation of Normal Means | 0.965 | 10 | 4 | 90% |
| 2 | Polyanskiy, Yury and Wu, Yihong (2021) Sharp regret bounds for empirical Bayes and compound decision problems self | 0.811 | 4 | 2 | 100% |
| 3 | Yury Polyanskiy and Yihong Wu (2020) Self-regularizing Property of Nonparametric Maximum Likelihood Estimator in Mixture Models self | 0.585 | 3 | 1 | 100% |
| 4 | Shen, Yandi and Wu, Yihong (2026) Poisson Empirical Bayes estimation: When does $g$-modeling beat $f$-modeling in theory (and in practice)? self | 0.585 | 3 | 1 | 100% |
| 5 | Chen, Jiafeng (2026) Empirical Bayes when estimation precision predicts parameters self | 0.585 | 3 | 1 | 100% |
| 6 | Cun-Hui Zhang (2009) Generalized maximum likelihood estimation of normal mixture densities | 0.511 | 2 | 2 | 50% |
| 7 | Lindsay, Bruce G (1983) The geometry of mixture likelihoods: a general theory | 0.511 | 2 | 2 | 50% |
| 8 | Ghosh, Sulagna and Ignatiadis, Nikolaos and Koehler, Frederic and Le… (2025) Stein's unbiased risk estimate and Hyvärinen's score matching | 0.511 | 2 | 1 | 100% |
| 9 | Jiang, Wenhua (2020) On general maximum likelihood empirical Bayes estimation of heteroscedastic IID normal means | 0.511 | 2 | 1 | 100% |
| 10 | Saha, Sujayam and Guntuboyina, Adityanand (2020) On the nonparametric maximum likelihood estimator for Gaussian location mixture densities with application to Gaussian denoising | 0.511 | 2 | 1 | 100% |
Showing the top 10 of 37 scored citations.