Dhruv Rohatgi, Vasilis Syrgkanis
arXiv 6 Oct 2021 · Statistics — Machine Learning
arXiv:2110.03070 · PDF · DOI · OpenAlex · Extracted main text
For many inference problems in statistics and econometrics, the unknown parameter is identified by a set of moment conditions. A generic method of solving moment conditions is the Generalized Method of Moments (GMM). However, classical GMM estimation is potentially very sensitive to outliers. Robustified GMM estimators have been developed in the past, but suffer from several drawbacks: computational intractability, poor dimension-dependence, and no quantitative recovery guarantees in the presence of a constant fraction of outliers. In this work, we develop the first computationally efficient GMM estimator (under intuitive assumptions) that can tolerate a constant $\epsilon$ fraction of adversarially corrupted samples, and that has an $\ell_2$ recovery guarantee of $O(\sqrt{\epsilon})$. To achieve this, we draw upon and extend a recent line of work on algorithmic robust statistics for related but simpler problems such as mean estimation, linear regression and stochastic optimization. As two examples of the generality of our algorithm, we show how our estimation algorithm and assumptions apply to instrumental variables linear and logistic regression. Moreover, we experimentally validate that our estimator outperforms classical IV regression and two-stage Huber regression on synthetic and semi-synthetic datasets with corruption.
appendix boundary found by appendix_command · 39% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Ilias Diakonikolas, Gautam Kamath, Daniel Kane, Jerry Li, Jacob Stei… (2019) Sever: A robust meta-algorithm for stochastic optimization | 1.000 | 6 | 3 | 100% |
| 2 | Takeshi Amemiya (1982) Two stage least absolute deviations estimators | 0.511 | 2 | 1 | 100% |
| 3 | Ilias Diakonikolas, Gautam Kamath, Daniel Kane, Jerry Li, Ankur Moit… (2019) Robust estimators in high-dimensions without the computational intractability | 0.511 | 2 | 1 | 100% |
| 4 | Gabriela V Cohen Freue, Hernan Ortiz-Molina, and Ruben H Zamar (2013) A natural robustification of the ordinary instrumental variables estimator | 0.511 | 2 | 1 | 100% |
| 5 | Elvezio Ronchetti and Fabio Trojani (2001) Robust inference with gmm estimators | 0.511 | 2 | 1 | 100% |
| 6 | Ainesh Bakshi and Adarsh Prasad (2021) Robust linear regression: Optimal rates in polynomial time | 0.405 | 1 | 1 | 100% |
| 7 | Tamara Broderick, Ryan Giordano, and Rachael Meager (2021) An automatic finite-sample robustness metric: Can dropping a little data change conclusions?, 2021 | 0.405 | 1 | 1 | 100% |
| 8 | David Card (1993) Using geographic variation in college proximity to estimate the return to schooling, 1993 | 0.405 | 1 | 1 | 100% |
| 9 | Ilias Diakonikolas, Gautam Kamath, Daniel M Kane, Jerry Li, Ankur Mo… (2017) Being robust (in high dimensions) can be practical | 0.405 | 1 | 1 | 100% |
| 10 | Ilias Diakonikolas, Weihao Kong, and Alistair Stewart (2019) Efficient algorithms and lower bounds for robust linear regression | 0.405 | 1 | 1 | 100% |
Showing the top 10 of 19 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.
| Citing paper | Intensity | Mentions | Sections | |
|---|---|---|---|---|
| 1 | Occasionally Misspecified | 0.405 | 1 | 1 |