EconBase
← All papers

Debiased Machine Learning without Sample-Splitting for Stable Estimators

Qizhao Chen, Vasilis Syrgkanis, Morgane Austern

arXiv 3 Jun 2022 · Econometrics · 4 citations (OpenAlex)

arXiv:2206.01825 · PDF · DOI · OpenAlex · Extracted main text

Abstract

Estimation and inference on causal parameters is typically reduced to a generalized method of moments problem, which involves auxiliary functions that correspond to solutions to a regression or classification problem. Recent line of work on debiased machine learning shows how one can use generic machine learning estimators for these auxiliary problems, while maintaining asymptotic normality and root-$n$ consistency of the target parameter of interest, while only requiring mean-squared-error guarantees from the auxiliary estimation algorithms. The literature typically requires that these auxiliary problems are fitted on a separate sample or in a cross-fitting manner. We show that when these auxiliary estimation algorithms satisfy natural leave-one-out stability properties, then sample splitting is not required. This allows for sample re-use, which can be beneficial in moderately sized sample regimes. For instance, we show that the stability properties that we propose are satisfied for ensemble bagged estimators, built via sub-sampling without replacement, a popular technique in machine learning practice.

Citation extraction

64
references
81
in-text mentions
64
distinct cited
3
self-citations
7,741
main-text words

appendix boundary found by appendix_command · 40% of the source is main text. Read the extracted text to check this.

Most heavily cited references

The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.

ReferenceIntensityMentionsSectionsMain text
1Victor Chernozhukov, Whitney Newey, Rahul Singh, and Vasilis Syrgkanis (2020) Adversarial estimation of riesz representers self0.87452100%
2Victor Chernozhukov, Whitney K Newey, and Rahul Singh (2021) A simple and general debiased machine learning theorem with finite sample guarantees0.84333100%
3Morgane Austern and Wenda Zhou (2020) Asymptotics of cross-validation self0.64422100%
4Pierre Bayle, Alexandre Bayle, Lucas Janson, and Lester Mackey (2020) Cross-validation confidence intervals for test error0.64422100%
5Olivier Bousquet and André Elisseeff (2002) Stability and generalization0.64422100%
6Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo,… (2018) Double/debiased machine learning for treatment and structural parameters: Double/debiased machine learning0.58531100%
7Khashayar Khosravi, Greg Lewis, and Vasilis Syrgkanis (2019) Non-parametric inference adaptive to intrinsic dimension self0.5112250%
8Peter J Bickel and Yaacov Ritov (1988) Estimating integrated squared density derivatives: Sharp best order of convergence estimates0.51121100%
9Victor Chernozhukov, Juan Carlos Escanciano, Hidehiko Ichimura, Whit… (2016) Locally robust semiparametric estimation0.51121100%
10Rafail Z Hasminskii and Ildar A Ibragimov (1979) On the nonparametric estimation of functionals0.51121100%

Showing the top 10 of 64 scored citations.

Cited by, within the corpus

arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.

Citing paperIntensityMentionsSections
1Simultaneous Inference for Local Structural Parameters with Random Forests$^*$1.00063
2Neighborhood Stability in Double/Debiased Machine Learning with Dependent Data0.890176
3Cross-Fitting-Free Debiased Machine Learning with Multiway Dependence0.73732
4Adversarial Estimation of Riesz Representers0.64422
5Inference in Partially Linear Models under Dependent Data with Deep Neural Networks0.64422
6Reproducible Aggregation of Sample-Split Statistics$^*$0.536132
7Fisher-Schultz Lecture: Generic Machine Learning Inference on Heterogenous Treatment Effects in Randomized Experiments, with an Application to Immunization in India0.51132
8=0pt =0pt plus .5=0pt plus .5=.3Identification and Semiparametric Estimation of Conditional Means from Aggregate Data0.51132
9RieszNet and ForestRiesz: Automatic Debiased Machine Learning with Neural Nets and Random Forests0.51121
10Dynamic Local Average Treatment Effects0.51121