arXiv 28 May 2022 · Statistics — Machine Learning
arXiv:2205.14284 · PDF · DOI · OpenAlex · Extracted main text
Measuring the stability of conclusions derived from Ordinary Least Squares linear regression is critically important, but most metrics either only measure local stability (i.e. against infinitesimal changes in the data), or are only interpretable under statistical assumptions. Recent work proposes a simple, global, finite-sample stability metric: the minimum number of samples that need to be removed so that rerunning the analysis overturns the conclusion, specifically meaning that the sign of a particular coefficient of the estimated regressor changes. However, besides the trivial exponential-time algorithm, the only approach for computing this metric is a greedy heuristic that lacks provable guarantees under reasonable, verifiable assumptions; the heuristic provides a loose upper bound on the stability and also cannot certify lower bounds on it. We show that in the low-dimensional regime where the number of covariates is a constant but the number of samples is large, there are efficient algorithms for provably estimating (a fractional version of) this metric. Applying our algorithms to the Boston Housing dataset, we exhibit regression analyses where we can estimate the stability up to a factor of $3$ better than the greedy heuristic, and analyses where we can certify stability to dropping even a majority of the samples.
appendix boundary found by appendix_command · 35% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Tamara Broderick, Ryan Giordano, and Rachael Meager, An automatic fi… (2020) 16 | 0.946 | 13 | 5 | 85% |
| 2 | Nikolas Kuschnig, Gregor Zens, and Jes Cuaresma, Hidden in plain sig… | 0.874 | 6 | 3 | 67% |
| 3 | David A Belsley, Edwin Kuh, and Roy E Welsch, Regression diagnostics… (1980) | 0.811 | 4 | 2 | 100% |
| 4 | James Renegar, On the computational complexity and geometry of the f… (1992) no. 3, 255–299 | 0.737 | 5 | 2 | 60% |
| 5 | David Harrison Jr and Daniel L Rubinfeld, Hedonic housing prices and… (1978) no. 1, 81–102 | 0.644 | 2 | 2 | 100% |
| 6 | Suyash Gupta and Dominik Rothenhäusler, The $ r $-value: evaluating… (2021) | 0.585 | 3 | 1 | 100% |
| 7 | John Milnor, On the betti numbers of real varieties, Proceedings of… (1964) no. 2, 275–280 | 0.511 | 2 | 2 | 50% |
| 8 | Samprit Chatterjee and Ali S Hadi, Influential observations, high le… (1986) 379–393 | 0.511 | 2 | 1 | 100% |
| 9 | Adam Klivans, Pravesh K Kothari, and Raghu Meka, Efficient algorithm… (2018) pp. 1420–1430 | 0.511 | 2 | 1 | 100% |
| 10 | Edward E Leamer, Global sensitivity results for generalized least sq… (1984) no. 388, 867–870 | 0.511 | 2 | 1 | 100% |
Showing the top 10 of 42 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.
| Citing paper | Intensity | Mentions | Sections | |
|---|---|---|---|---|
| 1 | Sample Fit Reliability | 0.737 | 3 | 2 |
| 2 | Testing Most Influential Sets | 0.405 | 1 | 1 |
| 3 | Finding Most Influential Sets | 0.405 | 1 | 1 |