Nishanth Dikkala, Greg Lewis, Lester Mackey, Vasilis Syrgkanis
arXiv 12 Jun 2020 · Econometrics · 21 citations (OpenAlex)
arXiv:2006.07201 · PDF · DOI · OpenAlex · Extracted main text
We develop an approach for estimating models described via conditional moment restrictions, with a prototypical application being non-parametric instrumental variable regression. We introduce a min-max criterion function, under which the estimation problem can be thought of as solving a zero-sum game between a modeler who is optimizing over the hypothesis space of the target model and an adversary who identifies violating moments over a test function space. We analyze the statistical estimation rate of the resulting estimator for arbitrary hypothesis spaces, with respect to an appropriate analogue of the mean squared error metric, for ill-posed inverse problems. We show that when the minimax criterion is regularized with a second moment penalty on the test function and the test function space is sufficiently rich, then the estimation rate scales with the critical radius of the hypothesis and test function spaces, a quantity which typically gives tight fast rates. Our main result follows from a novel localized Rademacher analysis of statistical learning problems defined via minimax objectives. We provide applications of our main results for several hypothesis spaces used in practice such as: reproducing kernel Hilbert spaces, high dimensional sparse linear functions, spaces defined via shape constraints, ensemble estimators such as random forests, and neural networks. For each of these applications we provide computationally efficient optimization methods for solving the corresponding minimax problem (e.g. stochastic first-order heuristics for neural networks). In several applications, we show how our modified mean squared error rate, combined with conditions that bound the ill-posedness of the inverse problem, lead to mean squared error rates. We conclude with an extensive experimental analysis of the proposed methods.
appendix boundary found by appendix_command · 17% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Krikamol Muandet, Arash Mehrjou, Si Kai Lee, and Anant Raj (2019) Dual iv: A single stage instrumental variable regression | 0.763 | 6 | 2 | 67% |
| 2 | Krikamol Muandet, Wittawat Jitkrittum, and Jonas Kübler (2020) Kernel conditional moment test via maximum moment restriction | 0.737 | 5 | 3 | 40% |
| 3 | Andrew Bennett, Nathan Kallus, and Tobias Schnabel (2019) Deep generalized method of moments for instrumental variable analysis | 0.727 | 26 | 5 | 38% |
| 4 | Rahul Singh, Maneesh Sahani, and Arthur Gretton (2019) Kernel instrumental variable regression | 0.693 | 6 | 2 | 50% |
| 5 | Xiaohong Chen and Demian Pouzo (2012) Estimation of nonparametric conditional moment models with possibly nonsmooth generalized residuals | 0.606 | 18 | 6 | 22% |
| 6 | Jason Hartford, Greg Lewis, Kevin Leyton-Brown, and Matt Taddy (2017) Deep IV: A flexible approach for counterfactual prediction self | 0.606 | 6 | 2 | 33% |
| 7 | Denis Chetverikov and Daniel Wilhelm (2017) Nonparametric instrumental variable estimation under monotonicity | 0.511 | 3 | 2 | 33% |
| 8 | Peter L Bartlett, Olivier Bousquet, Shahar Mendelson, et al (2005) Local rademacher complexities | 0.511 | 3 | 2 | 33% |
| 9 | Guillaume Lecué and Shahar Mendelson (2018) Regularization and the small-ball method i: Sparse recovery | 0.511 | 2 | 2 | 50% |
| 10 | Richard Blundell, Xiaohong Chen, and Dennis Kristensen (2007) Semi-nonparametric iv estimation of shape-invariant engel curves | 0.511 | 2 | 2 | 50% |
Showing the top 10 of 83 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.