EconBase
← All papers

Estimation and Inference by Stochastic Optimization

Jean-Jacques Forneron

arXiv 6 May 2022 · Econometrics · publishedJournal of Econometrics (2023) · 3 citations (OpenAlex)

arXiv:2205.03254 · PDF · DOI · OpenAlex · Extracted main text

Abstract

In non-linear estimations, it is common to assess sampling uncertainty by bootstrap inference. For complex models, this can be computationally intensive. This paper combines optimization with resampling: turning stochastic optimization into a fast resampling device. Two methods are introduced: a resampled Newton-Raphson (rNR) and a resampled quasi-Newton (rqN) algorithm. Both produce draws that can be used to compute consistent estimates, confidence intervals, and standard errors in a single run. The draws are generated by a gradient and Hessian (or an approximation) computed from batches of data that are resampled at each iteration. The proposed methods transition quickly from optimization to resampling when the objective is smooth and strictly convex. Simulated and empirical applications illustrate the properties of the methods on large scale and computationally intensive problems. Comparisons with frequentist and Bayesian methods highlight the features of the algorithms.

Citation extraction

70
references
108
in-text mentions
70
distinct cited
4
self-citations
13,613
main-text words

appendix boundary found by appendix_command · 45% of the source is main text. Read the extracted text to check this.

Most heavily cited references

The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.

ReferenceIntensityMentionsSectionsMain text
1Donaldson, D (2018) Railroads of the Raj: Estimating the impact of transportation infrastructure0.87452100%
2Forneron, J.-J. and S. Ng (2021) Estimation and Inference by Stochastic Optimization: Three Examples, in self0.84333100%
3Davidson, R. and J. G. MacKinnon (1999) Bootstrap Testing in Nonlinear Models0.73732100%
4Kline, P. and A. Santos (2012) A Score Based Approach to Wild Bootstrap Inference0.73732100%
5Newey, W. and D. McFadden (1994) Large Sample Estimation and Hypothesis Testing, in0.73732100%
6van der Vaart, A. W. and J. A. Wellner (1996) Weak Convergence and Empirical Processes0.6936333%
7Nocedal, J. and S. Wright (2006) Numerical Optimization0.6444250%
8Andrews, D. W. K (2002) Higher-Order Improvements of a Computationally Attractive k-Step Bootstrap for Extremum Estimators0.64422100%
9Forneron, J.-J (2022) Noisy, Non-Smooth, Non-Convex Estimation of Moment Condition Models self0.64422100%
10Honoré, B. E. and L. Hu (2017) Poor (Wo)man's Bootstrap0.64422100%

Showing the top 10 of 70 scored citations.

Cited by, within the corpus

arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.

Citing paperIntensityMentionsSections
1Stochastic Learning of Semiparametric Monotone Index Models with Large Sample Size1.00054
2Fast Inference for Quantile Regression with Tens of Millions of Observations0.40511
3Epsilon-Minimax Solutions of Statistical Decision Problems0.40511
4Approximate Least-Favorable Distributions and Nearly Optimal Tests via Stochastic Mirror Descent0.40511