EconBase
← All papers

Statistical Tests for Replacing Human Decision Makers with Algorithms

Kai Feng, Han Hong, Ke Tang, Jingyuan Wang

arXiv 20 Jun 2023 · Econometrics · publishedManagement Science (2025) · 41 citations (OpenAlex)

arXiv:2306.11689 · PDF · DOI · OpenAlex · Extracted main text

Abstract

This paper proposes a statistical framework of using artificial intelligence to improve human decision making. The performance of each human decision maker is benchmarked against that of machine predictions. We replace the diagnoses made by a subset of the decision makers with the recommendation from the machine learning algorithm. We apply both a heuristic frequentist approach and a Bayesian posterior loss function approach to abnormal birth detection using a nationwide dataset of doctor diagnoses from prepregnancy checkups of reproductive age couples and pregnancy outcomes. We find that our algorithm on a test dataset results in a higher overall true positive rate and a lower false positive rate than the diagnoses made by doctors only.

Citation extraction

72
references
168
in-text mentions
109
distinct cited
2
self-citations
31,469
main-text words

appendix boundary found by appendix_command · 81% of the source is main text. Read the extracted text to check this.

Most heavily cited references

The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.

ReferenceIntensityMentionsSectionsMain text
1Chan, Gentzkow and Yu (2022) Selection with variation in diagnostic skill: Evidence from radiologists0.73732100%
2Currie and MacLeod (2017) Diagnosing expertise: Human capital, decision making, and performance among physicians0.73732100%
3Esteva, Kuprel, Novoa, Ko, Swetter, Blau and Thrun (2017) Dermatologist-level classification of skin cancer with deep neural networks0.69351100%
4Kermany, Goldbaum, Cai, Valentim, Liang, Baxter, McKeown, Yang, Wu,… (2018) Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning0.69351100%
5Kleinberg, Lakkaraju, Leskovec, Ludwig and Mullainathan (2018) Human decisions and machine predictions0.64441100%
6Berk (2017) An impact assessment of machine learning risk forecasts on parole board decisions and recidivism0.64441100%
7Fuster, Goldsmith-Pinkham, Ramadorai and Walther (2022) Predictably unequal? The effects of machine learning on credit markets0.64441100%
8Liang, Tsui, Ni, Valentim, Baxter, Liu, Cai, Kermany, Sun, Chen et al (2019) Evaluation and accurate diagnoses of pediatric diseases using artificial intelligence0.64441100%
9Peng, Liu, Lv, Liu, Zhou, Yang, Ren, Liu, Wang, Zhang et al (2021) Deep learning-based artificial intelligence model to assist thyroid nodule diagnosis and management: a multicentre diagnostic st…0.64441100%
10Bansal, Wu, Zhou, Fok, Nushi, Kamar, Ribeiro and Weld (2021) Does the whole exceed its parts? the effect of ai explanations on complementary team performance0.64422100%

Showing the top 10 of 109 scored citations.

Cited by, within the corpus

arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.

Citing paperIntensityMentionsSections
1Statistical Inference of Optimal Allocations 1: Regularities and their Implications0.73732