EconBase
← All papers

The Hellinger Bounds on the Kullback-Leibler Divergence and the Bernstein Norm

Tetsuya Kaji

arXiv 25 Jan 2026 · Mathematics — Statistics Theory · publishedJapanese Economic Review (2026)

arXiv:2601.17860 · PDF · DOI · OpenAlex · Extracted main text

Abstract

The Kullback-Leibler divergence, the Kullback-Leibler variation, and the Bernstein "norm" are used to quantify discrepancies among probability distributions in likelihood models such as nonparametric maximum likelihood and nonparametric Bayes. They are closely related to the Hellinger distance, which is often easier to work with. Consequently, it is of interest to characterize conditions under which the Hellinger distance serves as an upper bound for these measures. This article characterizes a necessary and sufficient condition for each of the discrepancy measures to be bounded by the Hellinger distance. It accommodates unbounded likelihood ratios and generalizes all previously known results. We then apply it to relax the regularity condition for the sieve maximum likelihood estimator.

Citation extraction

14
references
71
in-text mentions
14
distinct cited
0
self-citations
9,851
main-text words

appendix boundary found by none_found · 100% of the source is main text. Read the extracted text to check this.

Most heavily cited references

The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.

ReferenceIntensityMentionsSectionsMain text
1van der Vaart and Wellner (2023) Weak Convergence and Empirical Processes: With Applications to Statistics1.000213100%
2Ghosal and van der Vaart (2017) Fundamentals of Nonparametric Bayesian Inference1.000134100%
3Birgé and Massart (1998) Minimum Contrast Estimators on Sieves: Exponential Bounds and Rates of Convergence0.92843100%
4Kaji, Manresa and Pouliot (2023) An Adversarial Approach to Structural Estimation0.92843100%
5Kaji and Ro cková (2023) Metropolis–Hastings via Classification0.92843100%
6van der Vaart and Wellner (1996) Weak Convergence and Empirical Processes: With Applications to Statistics0.92843100%
7Ghosal, Ghosh and van der Vaart (2000) Convergence Rates of Posterior Distributions0.87472100%
8Wong and Shen (1995) Probability Inequalities for Likelihood Ratios and Convergence Rates of Sieve MLES0.87452100%
9Birgé (1983) Approximation dans les espaces métriques et théorie de l'estimation0.84333100%
10Birgé and Massart (1993) Rates of convergence for minimum contrast estimators0.51121100%

Showing the top 10 of 14 scored citations.