arXiv 12 Feb 2020 · Computers and Society · 51 citations (OpenAlex)
arXiv:2002.05193 · PDF · DOI · OpenAlex · Extracted main text
"All models are wrong, but some are useful", wrote George E. P. Box (1979). Machine learning has focused on the usefulness of probability models for prediction in social systems, but is only now coming to grips with the ways in which these models are wrong---and the consequences of those shortcomings. This paper attempts a comprehensive, structured overview of the specific conceptual, procedural, and statistical limitations of models in machine learning when applied to society. Machine learning modelers themselves can use the described hierarchy to identify possible failure points and think through how to address them, and consumers of machine learning models can know what to question when confronted with the decision about if, where, and how to apply machine learning. The limitations go from commitments inherent in quantification itself, through to showing how unmodeled dependencies can lead to cross-validation being overly optimistic as a way of assessing model performance.
appendix boundary found by none_found · 100% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Breiman, Leo (2001) Statistical modeling: The two cultures (with comments and a rejoinder by the author) | 1.000 | 7 | 3 | 100% |
| 2 | Rescher, Nicholas (1998) Predicting the future: An introduction to the theory of forecasting | 1.000 | 5 | 4 | 100% |
| 3 | McQuillan, Dan (2018) People's councils for ethical machine learning | 0.928 | 4 | 4 | 100% |
| 4 | Cardoso, Fatima, Laura J. van't Veer, Jan Bogaerts, Leen Slaets, Giu… (2016) 70-gene signature as an aid to treatment decisions in early-stage breast cancer | 0.928 | 4 | 3 | 100% |
| 5 | Smith, Linda Tuhiwai (2012) Decolonizing methodologies: Research and indigenous peoples | 0.928 | 4 | 3 | 100% |
| 6 | Jacobs, Abigail Z. and Hanna Wallach (1912) Measurement and fairness | 0.843 | 3 | 3 | 100% |
| 7 | Katz, Yarden (2017) Manufacturing an Artificial Intelligence revolution, November 2017 | 0.843 | 3 | 3 | 100% |
| 8 | Wagstaff, Kiri L (2012) Machine learning that matters | 0.843 | 3 | 3 | 100% |
| 9 | Efron, Bradley (2004) The estimation of prediction error: Covariance penalties and cross-validation | 0.811 | 4 | 2 | 100% |
| 10 | Mullainathan, Sendhil and Jann Spiess (2017) Machine learning: An applied econometric approach | 0.811 | 4 | 2 | 100% |
Showing the top 10 of 286 scored citations.