Statistical Tests for Replacing Human Decision Makers with Algorithms

Statistical tests for replacing human decision makers with algorithms

\thispagestyle{empty}

Introduction

\setcounter{page}{1} In the current era of machine learning, artificial intelligence (AI) algorithms have emerged as valuable tools that can learn significant features from data and potentially facilitate decision making in various disciplines. In medical research, esteva2017dermatologist and kermany1 utilize deep neural network to automate diagnosis based on medical images. berg2020rise and sadhwani2021deep leverage machine learning algorithms to predict the default probability for bank lending decision. mullainathan2017machine and athey2019machine apply machine learning models to broad optimal policy assignment problems.

A considerable number of papers have compared the performance of algorithms with that of the representative human decision makers. To mention a few, esteva2017dermatologist claims that their algorithm outperformed the aggregate dermatologist on some skin cancer classification tasks; kermany1 targets macular degeneration dagnosis and concluded that the machine algorithm outperforms some retinologists; rajpurkar1 highlights the performance gain of a deep convolutional neural network over an aggregate cardiologist for processing ECG sequences; QJEbail trains a gradient boosted decision tree model to predict crime risk during pre-trial bail and suggested that substituting human judges with the algorithm could result in large welfare gains.

The findings of the aforementioned literature rely mostly on the observation that the pair of false positive rate (FPR) and true positive rate (TPR) point lies strictly below the receiver operating characteristic (ROC) curve generated by the machine learning algorithm, implying that algorithms can achieve a higher TPR for a given FPR or a lower FPR for a given TPR. In binary classification / decision making, FPR is the number of wrongly classified negative events divided by the number of actual negative events, and TPR is the number of correctly classified positive events divided by the number of total positive events. FPR/TPR are synonymous to size and power in classical hypothesis testing.

We first caution against such interpretations without a deeper understanding of human decision making processes. The literature findings can be rationalized not only by the superior information quality of algorithms but also by the incentive heterogeneity, i.e. the differential preferences for taking the same action by multiple human decision makers, some of whom can be no less accurate than algorithms in processing information from observational data. Machine algorithms provide information, but decision making is a combination of incentives and information. Using machine algorithms to assist with decision making necessitates the modeling of incentives. Second, the FPR/TPR pairs of human decision makers are typically imprecisely measured, especially when the number of cases for each decision makers is not large. The data in our empirical analysis shows not only substantial heterogeneity between individual doctors, but also that the randomness in measuring the quality of decision makers plays a key role when the performance of decision makers is compared to that of machine algorithms.

To illustrate the first issue of concern, consider Figure (ref), in which a collection of FPR/TPR pairs for human decision makers, denoted as $j = 1, \ldots, J$, all lie approximately on a ROC curve generated by their employed decision rules $\widehat Y_{i,j} = \mathds{1}\(p\(X_{i,j}, U_{i,j}\right) > c_{j}\right)$ with different individual cutoff threshold $c_{j}$ for decision maker $j$. In the decision rule, $X_{i,j}$ are observed features used in the machine learning algorithm; $U_{i,j}$ are private information only observable to the human decision makers, and $p\(x,u\right) = \mathbb{P}\(Y = 1\vert x,u\right)$ is a propensity score function. Also drawn is the machine ROC curve based on classification rules $\widehat Y_i = \mathds{1}\(p\(X_i\right) > c\right)$, which collects the corresponding FPR/TPR pairs by varying the cutoff threshold $c$, and where $p\(x\right) = \mathbb{P}\(Y = 1\vert x\right)$ is the correctly specified propensity score function using observable features $x$ discovered by the machine learning algorithm. However, after averaging over all decision makers, the aggregate human FPR/TPR point lies visibly below the machine ROC curve.

Our paper makes several contributions. First, we offer a conceptual framework of information and incentives for interpreting comparisons between the machine algorithms and human decision makers. Second, we provide a model to identify human decision maker candidates for replacement by machine algorithms and to generate machine algorithm based decisions that incorporate human decision maker preferences. The replacement model accounts for the sampling uncertainty of individual decision makers. Third, an empirical application to abnormal birth detection demonstrates that our proposed replacement algorithm significantly improves the overall performance of risky pregnancy detection. Fourth, we conduct an extensive synthetic data analysis that serves as guidance for evaluating empirical procedures incorporating complementarity in the presence of information asymmetry between human decision maker and machine algorithms.

In this paper, in order to compare machines with human decision makers, we restrict incentive heterogeneity and focus on information processing capacity. If the individual FPR/TPR pairs are precisely known without sampling errors, they can be directly compared to the machine ROC curve; we interpret a human FPR/TPR pair below the machine ROC curve as evidence of the superiority of the machine algorithms. As shown in Figure (ref), a human FPR/TPR pair below the machine ROC curve can be dominated by any point on the line segment of the ROC curve between A and B. Replacing the human FPR/TPR pair by any point on the A-B segment of the machine ROC curve segment results in higher TPR without increasing FPR, or lower FPR without sacrificing TPR.

In reality, only an estimate of the FPR/TPR pair can be obtained from the empirical data. A statistical framework to compare the performance of algorithms with that of human decision makers requires accounting for estimation sampling errors. We focus on two main issues. First, we seek statistical evidence supporting the information superiority of an algorithm that favors the replacement of a human decision maker. Second, we aim to identify the most appropriate point on the machine ROC curve for this purpose. To address these two issues, we experiment with both a heuristic frequentist confidence set approach and a subsequent Bayesian inference approach. In both scenarios, confidence and credible regions are formed to inform the decision making process. The Bayesian approach results in more replacement and additional improvement in the aggregate FPR/TPR pair. We suggest the Bayesian analysis as an applicable decision framework.

At a disaggregated level, each individual human can have either less or more information processing capacity than an algorithm has. For instance, liang_evaluation_2019 proposes a machine learning algorithm to diagnose childhood diseases that outperforms junior physician groups but marginally underperforms senior physician groups.

Similar findings are reported in peng2021deep. esteva2017dermatologist and kermany1 also find substantial variability in the human experts' performance. Our framework chooses between each human decision maker and the algorithm for making future decisions. We identify a subset of human decision makers to be replaced by the algorithm, whereas the rest of human decision makers are retained.

We apply our statistical framework to a unique National Free Prepregnancy Checkups (NFPC) dataset. The NFPC is a free health checkup service for conceiving couples across 31 provinces in China. In addition to the pregnancy outcome and numerous patient-case features, the dataset also includes the doctors' IDs and diagnoses of adverse pregnancy outcomes. We first split the data into two parts. The first part is used to compare doctors with algorithms. Specifically, we employ a random forest (RF) method for diagnosing risky pregnancy, which achieves an area under the curve (AUC) above 0.68. Using 95% credible level, our Bayesian approach suggests that the random forest algorithm outperforms 46.1% of doctors. The second part of the dataset is used to evaluate the quality of the diagnosis procedure based on the combination of retained doctors and the algorithm. The combined decision making procedure achieves an increase of 46.6% in the TPR and a reduction of 10.1% in the FPR. Additional detailed analysis further shows that the potential for machine algorithms to improve decision making quality is substantial even when only a smaller portion of doctors are replaced. We also find that doctors practicing only in country hospitals have a lower replacement ratio compared with those having practiced in township clinics. The result of our Bayesian analysis suggests that 51.2% of doctors who have practiced only at township clinics are replaced compared to 39.8% of doctors who have practiced only at county hospitals.

\paragraph{Related Literature} Our paper relates to several strands of literature. An extensive literature studies the utilization of AI to enhance medical diagnostic capabilities. Machine learning, especially deep learning methods, which demonstrate great potential in tackling diverse and complex tasks, has been experimented to diagnose in multiple medical subfields, including ophthalmology kermany1; cardiology rajpurkar1, hannun2019cardiologist; dermatology esteva2017dermatologist; respirology ardila2019end, liang_evaluation_2019; clinical psychology and psychiatry ashar2017empathic, yoon2022d; tumor and cancer detection mckinney2020international, peng2021deep, uhm2021deep, tian2024prediction. Additional surveys can be found in erickson2017machine, johnson2018artificial, dwyer2018machine, chan2020machine, huang2020fusion and swanson2023patterns.

Beyond medical diagnoses, AI is widely explored in social-economic decision making. See for example berg2020rise, sadhwani2021deep and fekadu2022machine for loan approval and credit risk modeling; fuster2019role, vallee2019marketplace and fuster2022predictably for market structure implications of employing machine learning algorithms; berk2017impact, zeng2017interpretable, QJEbail for bail and parole analysis; chalfin2016productivity and sajjadiani2019using for productivity assessment.

Numerous papers in the literature have benchmark algorithms against human decision makers\footnote{ See for example long2017artificial, kermany1, de2018clinically, rajpurkar1, hannun2019cardiologist, esteva2017dermatologist, han2020augmented, daneshjou2022disparities, ardila2019end, liang_evaluation_2019, mckinney2020international, peng2021deep, uhm2021deep, tian2024prediction, berk2017impact, QJEbail, han2020augmented. }. While machine algorithms often outperform some aggregation of human decision makers, these findings are not uniform across individual human decision makers. Unlike machine algorithms, human decision makers can be highly heterogeneous in both incentives and information processing capacity. The health economics literature suggests that doctors may be swayed by multiple non-medical incentives including lawsuit avoidance studdert2005defensive, financial gain johnson2016physicians, demand pressure from patient lopez2018contribution and procedural skills currie3. Our analysis highlights the importance of modeling and restricting incentive heterogeneity in a human-AI comparison.

A growing literature also explores AI assisted human decision making. While brennan2019comparing, han2020augmented, peng2021deep, yang2021deep and tian2024prediction report salient performance improvement when doctors can refer to the prediction results of algorithms, jin2024evaluating and yu2024heterogeneity report contradicting evidence where the performance of a portion of doctors is worsened by AI assistance. By offering doctors monetary incentives prior to the experiment, wang2023diagnosing and wang2024overcoming find that doctor diagnoses can be significantly influenced by both the machine information and the preset incentive structures. While both bansal2021does and wang2023diagnosing find that giving doctors explanations increases the acceptance of machine recommendations, the effect of providing explanation on performance is inconclusive and varies. In the beverage vending machine business, a field experiment on product assortment decision conducted by kawaguchi2021will finds heterogeneous adoption by workers dependent on location and both worker and machine characteristics. alur2023auditing propose a test whether human experts help improve machine algorithm predictions. Our analysis also suggests that the human machine complementarity, argued for by donahue2022human, is not only theoretically challenging but also difficult to implement empirically. Overall, the mechanism by which AI assists human decision making is unclear. Our paper thus focuses on a simple strategy that identifies and replaces underperforming doctors. This modeling choice is also consistent with the analysis in mullainathan2022diagnosing and chan2022selection who suggest that doctor performance can not be explained by preference heterogeneity alone. In addition, the case-specific replacement algorithm described in Appendix (ref) does not outperform the baseline method for our NFPC dataset.

A substantial body of literature discusses the discriminatory bias of algorithms o2017weapons, eubanks2018automating, berk2021fairness. But the potential of AI and algorithmic decision making to reverse extant bias in the training data is also noted by rambachan2019bias and rambachan2020economic. Through efficent information provision and scarce resource utilization, AI can be instrumental for inequality reduction. Using artificial intelligence to empower medical diagnose in developing areas has been advocated by researches including adepoju2017mhealth, guo2018application, trivedi2019risks, mondal2023artificial and others. bulathwela2021could discuss the comprehensive application of AI to narrow education inequality gap. Our finding using the NFPC dataset is also consistent with anecdotal information regarding the quality disparity across tiers of the hospital classification system in China.

Optimal decision making using machine learning techniques is also drawing increasing attention from econometrics. kitagawa2018should, mbakop2021model, athey2021policy and feng2024statistical explore the asymptotic properties of empirical algorithm based optimal decision making. manski2018credible considers the identification problem when only the prediction conditional on a subset of covariates and the conditional distribution of the “omitted” covariates are known. In this paper, we assume, as in the baseline cases of kitagawa2018should, that the machine algorithm is known and deterministic, in view of the large data size and the comprehensive list of covariates in the NFPC dataset.

The rest of this paper is organized as follows. Section (ref) presents the statistical model of human-algorithm comparison. Section (ref) describes the data and the algorithm. Section (ref) reports results from the empirical analysis of the machine algorithm and human decision makers. Section (ref) presents a set of synthetic data analysis for additional insights, and Section (ref) concludes. Additional results are collected in the Appendix.

Replacing Human Decision Makers with Algorithms

Consider a sample dataset with labels $Y_i \in \{0,1\}$ and features $X_i,i=1,\ldots,n$. Define

where $\hat Y_i \in \{0, 1\}$ is a predictor for $Y_i$ based on a general decision rule, such that $\(X_i, Y_i, \hat Y_i\right)$ are i.i.d draws from an underlying population. The Law of Large Number implies that the FPR/TPR converge to their population analogs (denoted as $\beta$ and $\alpha$) as $n\rightarrow \infty$:

A key concept in this paper is incentive heterogeneity. In general terms, incentive refers to the motivation of certain actions, and incentive heterogeneity refers to different payoffs across multiple human decision makers for taking the same actions. Incentive heterogeneity can typically be classified intuitively as either intra-individual or inter-individual. On the one hand, inter-individual incentive heterogeneity refers to situations where the cost assessment only varies across decision makers but not across individual cases for each decision maker. On the other hand, intra-individual incentive heterogeneity refers to situations where the cost assessments across individual cases for a given decision maker vary based on both publicly observed features $x$ that are accessible by the machine algorithm and private information features $u$ that are only available to human decision makers.

To compare human decision makers and machines, we make the following two assumptions regarding incentive heterogeneity:

Assumption (ref) rules out intra-individual incentive heterogeneity and allows us to focus on inter-individual incentive heterogeneity. Doctors in our dataset exhibit highly variable degrees of conservativeness in their diagnoses. The FPR of the doctors with at least 300 diagnoses has a mean of 0.236 and a standard deviation of 0.230. Some doctors tend to diagnose almost all patients as highly risky, and some doctors rarely make a positive diagnosis. Figure (ref) illustrates. The diffusive distribution of doctors' FPR/TPR pairs also indicates both information processing capacity difference and inter-individual heterogeneity. The analysis in chan2022selection shows that when only either diagnostic skill difference or inter-individual incentive heterogeneity is present, the doctors FPR/TPR paris will concentrate in a small region, which is not the case in Figure (ref). Importantly, Assumption (ref) also allows for private information $u_{i,j}$ that is only observed by the doctors and not used in the machine learning algorithm. In the absence of the private information variable $u_{i,j}$, the doctors' diagnoses are perfectly predictable by the machine learning algorithm, which is very unlikely.

The first requirement is justified by the fact that the machine learning model is implemented using a training dataset that is by orders of magnitude larger than the number of cases for each individual human decision maker. For instance, around 150 thousand cases in the NFPC training dataset are used to estimate the machine ROC curve, whereas the median number of diagnoses per doctor in the classification set is only approximately 400. The estimation error in the machine ROC curve is negligible relative to the error in estimating doctors' FPR/TPR pairs. The sampling error in estimating the machine ROC curve is studied in a subsequent paper by feng2024statistical.

The second requirement that the $q_j\(x_{i,j},u_{i,j}\right)$ are used without sampling errors by the $j$th decision maker is interpreted as stating that the behavior of the $j$th decision maker is consistent with the use of a propensity score model $q_j\(x_{i,j},u_{i,j}\right)$. The method in this paper does not require the researcher to know the propensity score model $q_j\(x_{i,j},u_{i,j}\right)$ for the $j$th decision maker, and does not refer to any of the doctor propensity score models $q_j\(x_{i,j},u_{i,j}\right), j = 1,\ldots,J$.

The second requirement also rules out decision makers learning from the sample cases. A doctor $j$ does not update their propensity score model $q_j\(x_{i,j},u_{i,j}\right)$ during the data sample period.

This is a reasonable assumption in our empirical dataset because the eventual pregnancy outcomes are typically not available to doctors when they diagnose consecutive patient cases. We are not concerned with how doctors procure knowledge to develop their diagnosis model $q_j\(x_{i,j},u_{i,j}\right)$. Doctor $j$ might procure $q_j\(x_{i,j},u_{i,j}\right)$ from trainings obtained from a medical school or from working with a large pool of patients prior to the sampling period. Section (ref) provides more details about the data processing procedure, where sampling errors arise mainly from the relatively fewer number of observations for each individual decision maker.

Human FPR/TPR pairs and ROC curves in the population

The parameter of interest is the population pair of true positive and false positive rates for an individual decision maker: $\theta_{H} = \left(\alpha_{H}, \beta_{H}\right)$, where $\alpha$ and $\beta$ are defined as in (ref) and the subscript $H$ refers to “human”. If $\theta_{H}$ is above a machine ROC curve, as in Figure (ref), then given the FPR $\alpha_{H}$, the algorithm has a lower TPR $\beta_{M}$ than humans have. Similarly, given the TPR $\beta_{H}$, the machine has a larger FPR $\alpha_{M}$ than humans have. In this sense, humans outperform the algorithm. In particular, under Assumption (ref), if humans correctly use at least as much information as the algorithm does, in the sense that the propensity model is correctly specified, then $\theta_{H}$ lies above the machine ROC. The formal argument is given in Lemma (ref) in the appendix.

Figure (ref) shows a $\theta_{H}$ pair for a human decision maker that is below the machine ROC curve. A machine decision rule corresponding to a point on the machine ROC curve between A and B performs better than the human decision maker. The segment of the curve to the left of B (to the right of A) has a smaller $\alpha_{M}$ but a smaller $\beta_{M}$ (a larger $\beta_{M}$ but a larger $\alpha_{M}$) than the human decision maker. These segments of the machine ROC curve are not directly comparable to the human decision maker.

The classical Neyman-Pearson lemma states that when $p\(x\right)=\mathbb{P}\(Y=1\vert x\right)$ is correctly specified, for any pair of positive numbers $\left(\eta, \phi\right)$, there exists a cutoff value $c$, such that the point $\left(\alpha_c, \beta_c\right)$ on the ROC curve corresponding to the decision rule $\hat y = \mathds{1}\(p\(x\right)>c\right)$ maximizes a linear combination $\phi\beta - \eta\alpha$ of the TPR and FPR. Conversely, each point on this ROC curve optimizes some linear combination of TPR and FPR. Neyman-Pearson Lemma is also motivated by a Bayesian decision maker who minimizes posterior expected loss of misclassification upon observing the features $x$, based on a loss matrix of the following form:

where $c_{10}$ represents the cost of misclassifying $1$ as $0$ and $c_{01}$ represents the cost of misclassifying $0$ as $1$. Each point on the ROC curve, corresponding to a decision rule $\hat y = \mathds{1}\(p\(x\right) > c\right)$, minimizes Bayesian posterior expected loss for some choice of $c_{10}$ and $c_{01}$ through the relation $c = \frac{c_{01}}{c_{10} + c_{01}}$. Importantly, the cost matrix $c_{01}, c_{10}$ and linear combination coefficients $\eta, \phi$ in the Neyman-Pearson Lemma are independent of the features $x$ in order to provide optimality justification of the points on the ROC curve. Assumption (ref) is then interpreted as each doctor behaving as a Bayesian decision maker adopting the same constant cost matrix $c_{10}$ and $c_{01}$ among their patients. The ex ante expected minimized cost for the Bayesian decision maker corresponds to a linear combination of FPR and TPR:

If we denote point A and B in Figure (ref) as $\left(\alpha_{H}, \beta_{R}\right)$ and $\left(\alpha_{R}, \beta_{H}\right)$, then any point $\left(\alpha_{M}, \beta_{M}\right)$ between $\left(\alpha_{R}, \beta_{H}\right)$ and $\left(\alpha_{H}, \beta_{R}\right)$ on the machine ROC curve satisfies $\alpha_{M} < \alpha_{H}$ and $\beta_{M} > \beta_{H}$. For arbitrary $\phi, \eta > 0$ and $c_{10}, c_{01}$,

Therefore, compared with human decisions, any point $\left(\alpha_{M}, \beta_{M}\right)$ between $\left(\alpha_{R}, \beta_{H}\right)$ and $\left(\alpha_{H}, \beta_{R}\right)$ on the ROC curve achieves a better linear combination for all possible values of $\phi, \eta$, and lower expected cost for all loss matrixes of $c_{01}, c_{10}$. In this sense, $\left(\alpha_{M}, \beta_{M}\right)$ dominates $\left(\alpha_{H}, \beta_{H}\right)$.

Recall that $\theta_{H}$ is a population parameter that measures the expected performance of the human decision maker. When $\theta_H$ lies below the machine ROC curve, {\it on average}, the machine algorithm can outperform the human decision maker by increasing the population TPR or reducing the population FPR. However, unless the algorithm can make predictions with perfect accuracy, for each specific patient case, the human decision maker can either underperform or outperform the algorithm.

Our framework is likely to be applicable when people make many repeated decisions to allow for a sufficiently precise estimate of their error rates. It is less applicable in situations when people only ever make a few decisions of a type, a setting that may arise even when decisions are frequent if some action rarely is or should be taken. The lack of information renders the task of distinguishing humans from algorithms difficult, even if humans might be detectably worse in aggregate\footnote{We are grateful to the associate editor for pointing out this important clarification.}. Our application features a comprehensive set of covariates, a large dataset, reliable and measurable outcomes, and a stable prediction environment that are consistent with the scenarios in the important papers by kahneman2009conditions and currie3.

Human FPR/TPR pair and ROC curve comparison in the sample

Typically, the population value of $\theta_{H}$ is not directly observed, and instead needs to be estimated from a dataset. Denote the the sample estimates $\left(\text{FPR}, \text{TPR}\right)$ in (ref) as $\hat{\theta}_{H} = \left(\hat{\alpha}_{H}, \hat{\beta}_{H}\right)$. In the presence of sampling uncertainty, the inference problem pertains to how we can make a probabilistic statement regarding whether $\theta_{H}$ is above or below the machine ROC curve. From a frequentist analysis point of view, because the sample estimate $\hat{\theta}_{H}$ is a vector function of a multinomial distribution, the finite sample distribution function of $\hat{\theta}_{H}$ can be analytically intractable. Large sample frequentist analysis and simulation methods, however, are facilitated by the joint asymptotic normal distribution of $\hat{\theta}_{H}$.

The proof of Lemma (ref) is given in the appendix and follows from standard arguments for approximating multinomial distributions with normal limits. The asymptotic covariance $\Sigma$ can be consistently estimated by a sample analog $\hat\Sigma$ where the parameters $\alpha_H, \beta_H$ and $p$ are replaced by $\hat{\alpha}_H, \hat{\beta}_H$ and $\hat{p}$. The resulting asymptotic confidence set of the FPR/TPR pairs is elliptically shaped. As illustrated in Figure (ref), the blue ellipse indicates a 95% level confidence set for a decision maker in our dataset.

An immediate consequence of Lemma (ref) is that a direct comparison of the sample estimate $\left(\hat\alpha_H,\hat\beta_H\right)$ with the machine ROC curve is likely to be insufficient. Consider a situation where the true population pair of $\left(\alpha_H, \beta_H\right)$ lies strictly below the machine ROC curve: $\beta_H < g\left(\alpha_H\right)$ where $g\left(\cdot\right)$ denotes the machine ROC curve. Then with probability converging to $1$, $\left(\hat\alpha_H, \hat\beta_H\right)$ lies strictly below the machine ROC. Suppose we replace $\left(\hat\alpha_H, \hat\beta_H\right)$ by $\left(\hat\alpha_H, g\left(\hat\alpha_H\right)\right)$. By Lemma (ref), $\mathbb{P}\left(\hat\alpha_H \geq \alpha_H\right)\rightarrow 0.5$. In other words, the replacement point $\left(\hat\alpha_H, g\left(\hat\alpha_H\right)\right)$ on the ROC curve does not dominate the true population pair of $\left(\alpha_H, \beta_H\right)$ approximately half of the time. A similar situation arises if we replace $\left(\hat\alpha_H, \hat\beta_H\right)$ by $\(g^{-1}\left(\hat\beta_H\right), \hat{\beta}_{H}\right)$.

In order to provide a probabilistic statement regarding the property of the replacement procedure, a more formal statistical framework is necessary. We consider both a heuristic frequentist approach and a Bayesian approach.

A heuristic frequentist approach

We initially experiment with a heuristic procedure based on the frequentist principle, the predominant school of thought in statistical inference which typically begins with confidence set construction. The procedure is described as follows.

The convention of choosing a large confidence level suggests that the status quo of human decision making is to be replaced by an algorithm only when there is overwhelming evidence from the data indicating the superiority of the algorithm. In our application, we will only consider the machine algorithm to outperform the human decision when $\bar A \neq \emptyset$, i.e., when there exists a point on the machine ROC curve that dominates an entire confidence set of the human decision maker's $\theta_{H}$ parameter. In essence, we use estimates of the sampling uncertainty of the FPR/TPR pair of $\hat\theta_H$ to provide guidance on how far $\hat\theta_H$ needs to be below the machine ROC curve in order to justify replacing the human decision maker with the machine algorithm.

In the elliptical confidence set depicted in Figure (ref), denote by $\left(\alpha_{\beta_{high}}, \beta_{high}\right)$ the point that achieves the highest TPR on the confidence set, and denote by $\left(\alpha_{low}, \beta_{\alpha_{low}}\right)$ the point that achieves the smallest FPR on the confidence set. The point $\text{P}$ in Figure (ref) corresponds to $\left(\alpha_{low}, \beta_{high}\right)$. There are three possibilities for the relation between the position of the confidence set for the human FPR/TPR pair $\theta_H$ and the machine ROC curve.

In summary, if the point $P=\left(\alpha_{low}, \beta_{high}\right)$ is below the ROC curve, the human decision maker is replaced; otherwise, they are not. In cases 2 and 3, it is possible that the human decision maker is sufficiently capable compared to the machine algorithm. It is also possible that the human $\theta_{H}$ is not estimated precisely enough, for example, due to a lack of historical data that results in a large confidence set.

In frequentist statistical inference, $\theta_{H}$ is a fixed number. It is either above or below the machine ROC curve. The confidence set itself $\hat S$ is a random set, such that $\mathbb{P}\left(\theta_H \in \hat S\right) \longrightarrow 1 - \alpha.$ For a conventional size of $\alpha = 0.05$, if the confidence set is to be constructed 100 times from different samples, $\hat S$ will contain $\theta_{H}$ approximately 95 times. Consequently, by construction, $\bar A$ is a random set such that asymptotically,

Therefore, we have asymptotically

If $\theta_{H}$ lies below the ROC curve, $\mathbb{P}\left\{\bar{A} = \emptyset\right\} \to 0$ as the number of cases for the human decision maker increases to infinity. Furthermore, $\mathbb{P}\left\{ \bar{A} = \emptyset \text{ and P dominates } \theta_{H}\right\} < \mathbb{P}\left\{\bar{A} = \emptyset\right\} \rightarrow 0$, implying that asymptotically, $\mathbb{P}\left\{\bar{A} \text{ dominates } \theta_{H} \vert \bar{A} \neq \emptyset\right\} \geq 1 - \alpha.$ If $\theta_H$ lies above the ROC, we can control $\mathbb{P}\left\{\bar A = \emptyset\right\} \geq 1-\alpha$ by reasoning that

However, the finite sample size of the test can be severely distorted. By insisting on finding a segment of the machine ROC curve that simultaneously dominates the entire confidence region of $\theta_H$, the frequentist procedure is conservative by construction.

While conventional confidence sets are mostly symmetric, it is well known in statistics that by inverting a hypothesis test, alternative confidence sets can be constructed as the collection of parameter values for which the null hypothesis is not rejected. For example, if we let the null hypothesis be $H_0: \theta_H = \theta_{0H}$ and the alternative hypothesis be $H_1: \alpha_H > \alpha_{0H}\ \text{or}\ \beta_H < \beta_{0H}$, a test can be formed to reject the null hypothesis $H_0$ when either $\hat\alpha_H > \alpha_H + c$ or when $\hat\beta_H < \beta_H - d$, where $c$ and $d$ are chosen by the size of the test. The confidence set of $\theta_{0H}$ where the test does not reject takes the form of $\{\theta_{0H}: \alpha_{0H} \geq \hat\alpha_H -c,\ \text{and}\ \beta_{0H} \leq \hat\beta_H +d\}$, which is lower rectangular. While forming confidence sets based on inverting inequality tests is theoretically intriguing, it is not often used empirically.

An alternative in Appendix (ref) is to test whether the population value of $\theta_H$ is above or below the machine ROC curve.

We do not use this test because of its limitation that even if the null hypothesis of the status quo of human decision making is rejected, it does not locate a point of the machine ROC curve to replace human decisions, as discussed in section (ref).

A Bayesian approach

In the following we focus on an alternative Bayesian method, which often allows for a more transparent interpretation than frequentist inference. The Bayesian approach combines a prior distribution of the underlying population parameters with the likelihood of the data given the population parameters to produce a posterior distribution of the parameters given the data, which is then used in a decision theoretic framework where a threshold is chosen to minimize the posterior expected loss function. The combination of a multinomial likelihood function with a Dirichlet conjugate prior drastically improves computability. Simulating the posterior distribution of the key FPR/TPR parameters to construct a Bayesian credible set is facilitated by writing them as a vector function of the multinomial parameters whose posterior distribution is given analytically through the conjugate prior.

The empirical results suggest that the Bayesian method is less conservative and suggests replacement of more doctors by the machine algorithm than the frequentist method does.

Computing a Bayesian posterior distribution requires two key inputs: the likelihood of the data given the population value of $\theta_{H}$, and a prior distribution for the underlying population parameters. Each observation in the dataset consists of both the doctor diagnoses and the ground truth of pregnancy outcomes, and follows a multinomial distribution with four categories: diagnosed as risky and abnormal birth outcome; not diagnosed as risky and abnormal birth outcome; diagnosed as risky and normal birth outcome; not diagnosed as risky and normal birth outcome. The parameters of the multinomial probabilities are:

Since $t_1 + t_2 + t_3 + t_4 = 1$, we choose the first three as free parameters. The multinomial distribution is a completely specified parametric model and allows for exact likelihood Bayesian posterior distribution computation. The likelihood of the data given the free parameters $t=\(t_1, t_2, t_3\right)$ depends only on a set of sufficient statistics $\hat t=\left(\hat t_1, \hat t_2, \hat t_3\right)$ where

In particular, we can write the data likelihood as (where $\mathbb{D}$ refers to Data):

Given a prior distribution of $t$, denoted as $\pi\(t\right)$, the posterior distribution of the parameter $t$, denoted as $p\(t \vert \mathbb{D}\right)$ can usually be simulated or calculated analytically:

We are interested in the posterior distribution of $\theta_H=\left(\alpha_H, \beta_H\right)$, which can be written as functions of the underlying multinomial parameters: $h\(t\right) = \left(\alpha_{H}\(t\right), \beta_{H}\(t\right)\right)$. If we can simulate from the posterior distribution $p\(t \vert \mathbb{D}\right)$ and denote the simulated draws as $t_r, r=1,\ldots,R$, then the posterior distribution of $\theta_{H}$, denoted as $p\left(\theta_H \vert \mathbb{D}\right)$, can be estimated using the empirical distribution of $h\(t_r\right), r=1,\ldots,R$.

Computing the posterior distribution using a multinomial likelihood is facilitated by the Dirichlet conjugate prior. A Dirichlet prior on the K-dimensional simplex of $t_k,k=1,\ldots,K$, where $\sum_{k=1}^K t_k=1$, is specified by hyper-parameters $\gamma=\left(\gamma_k,k=1,\ldots,K\right)$. Its density is given by $\pi\(t\vert \gamma\right)=\frac{1}{B\left(\gamma\right)} \prod_{k=1}^K t_k^{\gamma_k-1}$, where $B\left(\gamma\right)$ is the multivariate Beta function. See for example kotz2019continuous. The case with $\gamma_k=\gamma, \forall k=1,\ldots,K$ is called a symmetric Dirichlet distribution. In one dimension, where $K=1$, the Dirichlet distribution specializes to a Beta distribution, which in turn includes the uniform distribution as a special case when $\gamma=1$.

The posterior distribution is also Dirichlet with parameters $\hat\gamma=\left(\hat\gamma_k,k=1,\ldots,K\right)$, where $\hat{\gamma}_k = \gamma_k + n \hat t_k$ for each $k=1,\ldots,K.$ Simulating from the posterior distribution of $\theta_{H} = \left(\alpha_{H}, \beta_{H}\right)$ can be accomplished by recomputing $\theta_{H,r} = h\(t_r\right)$ after drawing for $t_r$ from the posterior Dirichlet distribution with parameters $\hat\gamma$. Specifically, for each $t_r = \(t_{r,k}\right)^{K}_{k = 1}$, we calculate

The simulated draws $\theta_{H,r}, r=1,\ldots,R$ can be used to estimate posterior probabilities. For example, the posterior probability that $\theta_{H}$ lies below the ROC curve, denoted as

can be estimated by the fraction of times where $\theta_{H, r}$ lies below the ROC curve:

We are mainly interested in the {\it maximum} posterior probability of $\theta_{H}$ that are dominated simultaneously by a single point on the ROC curve, denoted as

and the corresponding maximizing point, denoted as $\theta_{max} = \left(\alpha_{max}, \beta_{max}=g\left(\alpha_{max}\right)\right)$, where

Both (ref) and (ref) can be estimated by the simulated posterior draws of $\theta_{H, r}, r = 1, \ldots, R$, by

More generally, decision-theoretic Bayesian analysis minimizes posterior expected losses. Let $\rho\left(\theta_{M}, \theta_{H}\right)$ denote a loss function that represents the disutility of replacing a human FPR/TPR pair $\theta_H$ by a machine FPR/TPR pair $\theta_M$. Consistent with Assumptions (ref) and (ref), we specify $\rho\left(\theta_{M}, \theta_{H}\right) = 0$ when $\alpha_{M} \leq \alpha_{H}$ and $\beta_{M} \geq \beta_{H}$, or when the machine FPR/TPR pair strictly dominates the human FPR/TPR pair. Otherwise, $\rho\left(\theta_{M}, \theta_{H}\right)$ may be strictly positive. Given the choice of $\rho\left(\theta_M, \theta_H\right)$, a point on the ROC curve can be chosen to minimize the posterior expected loss

which is also estimated using the simulated posterior draws of $\theta_{H,r}, r=1,\ldots,R$ as

In particular, (ref) is a special case of (ref) when $\rho\left(\theta_M, \theta_{H}\right) = 1 - \mathds{1}\left(\alpha_{H} \geq \alpha_{M}, \beta_{H} \leq \beta_{M}\right)$. It is also possible to explore more general loss functions. For example, we may weight the loss of replacing $\theta_H$ by $\theta_M$ by the distance between $\theta_H$ and the machine ROC curve:

where $\Vert\cdot\Vert$ is the Euclidean norm. Intuitively, the farther a human FPR/TPR pair $\theta_H$ is from the ROC curve, the smaller the loss (or the larger the benefit) of replacing it with a dominating point on the machine ROC curve. Alternatively, we can replace the distance from $\theta_H$ to the ROC curve by the distance of each $\theta_M$ to the complement of the set of FPR/TPR points that dominate $\theta_H$, and use the loss function:

The Euclidean distance from the machine ROC curve may also not fully capture the loss of not replacing a human FPR/TPR pair $\theta_H$ that lies under the machine ROC curve. For example, a diagonal ROC curve corresponds to completely randomized decision making without using feature information. Similarly, a ROC curve below the diagonal can be generated by a decision rule that awards rather than penalizes mis-classification errors. These considerations suggest that replacing a human FPR/TPR pair $\theta_H$ below the diagonal line with a dominating $\theta_M$ along the machine ROC curve should not entail losses. We can adopt a convention of normalizing the loss to be $1$ when $\theta_M$ does not dominate $\theta_H$, i.e., when $\alpha_M \geq \alpha_H$ or $\beta_M \leq \beta_H$.

We then specify the loss $\rho\left(\theta_M, \theta_H\right)$ when $\theta_M$ dominates $\theta_H$ and when $\theta_{H}$ lies between the machine ROC curve and the diagonal line. Each of such $\theta_H$ can be reproduced by a convex combination of a point on the machine ROC curve and a point on the diagonal line, using a decision rule that randomizes between the point on the machine ROC curve and the point on the diagonal. For example, $\left(\alpha_H, g\left(\alpha_H\right)\right)$ is generated by a machine based decision rule $\hat Y_{i} = \mathds{1}\(m\(X_{i}\right) > c_0\right)$ for some threshold $c_0$, while $\left(\alpha_H, \alpha_H\right)$ is generated by a fully randomized decision $\hat Y_{i} = \mathds{1}\(U_{i} \geq 1-\alpha_H\right)$ where $U_{i}$ is independently uniformly distributed on $\left(0,1\right)$. Then $\left(\alpha_H, \beta_H\right)$ can be generated by a lottery that places weight $\left(\beta_H-\alpha_H\right)/\(g\left(\alpha_H\right)-\alpha_H\right)$ on $\hat Y_{i} = \mathds{1}\(m\(X_{i}\right) > c_0\right)$ and the remaining weight on $\hat Y_{i} = \mathds{1}\(U_{i} \geq 1-\alpha_H\right)$. More precisely, if $V_{i}$ denotes an independent random variable uniformly distributed on $\left(0,1\right)$, the decision rule

generates the $\left(\alpha_H, \beta_H\right)$ FPR/TPR pair. The larger the weight $\lambda_{\theta_{H}} \equiv \frac{\beta_{H} - \alpha_{H}} {g\left(\alpha_{H}\right) - \alpha_{H}}$ placed on the machine ROC curve, the smaller the benefit, or the larger the loss, of replacing $\theta_H$ with a dominating pair $\theta_M$ on the machine ROC curve. The randomization scheme motivates using the weight $\lambda_{\theta_{H}}$ as a component of the loss function

Other randomization schemes can also be used. We refer to the above construction as a vertical decomposition. Similarly, a horizontal decomposition replaces the weights by $\lambda_{\theta_{H}} \equiv \frac{\beta_{H} - \alpha_{H}}{\beta_H - g^{-1}\left(\beta_{H}\right)}$. In addition, we experiment with decomposing the distance between $\theta_M=\left(\alpha_M, \beta_M\right)$ and the diagonal line into two parts. The first part is the distance from $\theta_M$ to the boundary of the set of FPR/TPR points that do not dominate $\theta_H$. The second part is the distance from this boundary to the diagonal line. The corresponding loss function that mirrors is

where distance can be measured either horizontally, implying that $\lambda_{\theta_{H},\theta_M}=\frac{\beta_M-\alpha_H}{\beta_M-\alpha_M}$, or vertically, implying that $\lambda_{\theta_{H},\theta_M}=\frac{\beta_H-\alpha_M}{\beta_M-\alpha_M}$.

When $\alpha_M < \alpha_H$ and $\beta_M > \beta_H$, replacing $\theta_H$ by $\theta_M$ is beneficial. The benefit, $b\left(\theta_M, \theta_H\right)$, is likely to be larger when $\theta_H$ is farther below the machine ROC curve. When either $\alpha_M > \alpha_H$ or $\beta_M < \beta_H$, we expect a cost $\ell\left(\theta_M, \theta_H\right)$. A general loss function consists of both a cost component and a benefit component:

For each loss function, a guardrail threshold is used to account for the status quo of human decision making. A human decision maker is only replaced by the machine algorithm when the posterior expected loss of the replacement is below a given prespecified level, or when the posterior expected benefit exceeds a certain level.

In addition to the deterministic decision rules of relying exclusively on either an individual doctor or the machine algorithm, randomized decision rules can be generated by mixing between a given point on the machine ROC curve and an individual doctor. Let $\omega$ denote the randomizing probability of using the machine algorithm, and $1-\omega$ the corresponding probability of relying on the human decision makers. The minimized expected posterior loss of the randomized decision rule can be written as

Without expert domain knowledge of medical diagnoses, it is difficult to implement (ref) and (ref). Investigating expert domain knowledge is beyond the scope of the current paper.

Our decision framework differs conceptually from the minimization of posterior expected loss function in Bayesian point estimation. A Bayesian point estimator of $\theta_H$, denoted as $\hat\theta_{H,B}$, is often defined as the minimizer of an expected posterior loss function:

The estimation loss function $\ell\left(\theta - \theta_H\right)$ typically depends only on $\vert\theta-\theta_H\vert$ and is increasing in $\vert \theta-\theta_H\vert$, which is different from our decision loss function $\rho\left(\theta_M, \theta_H\right)$.

Discussion: incentives and costs

The visual illusion shown in Figure (ref) that hampers the comparison between machine learning algorithms and human decision makers is an immediate artifact of Jensen's inequality. An optimal ROC generated by a correctly specified propensity score function is necessarily concave, since it is the primal function of a concave program. See for example feng2022properties. For the $j$th doctor, denote the FPR as $\alpha_j=\mathbb E \left(1-Y_{ij}\right)\hat Y_{ij}/\left(1 - p_j\right)$ and the TPR as $\beta_j=\mathbb E Y_{ij} \hat Y_{ij} / p_j$, where $p_j = \mathbb P\(Y_j =1\right)$. If all the $\left(\alpha_j, \beta_j\right)$ pairs lie on a strictly concave ROC curve $g\left(\cdot\right)$, these variables are related by $\beta_j = g\left(\alpha_j\right)$. When the patient cases across doctors are drawn from the same population, $p_j=p$ is a constant. It follows from Jensen's inequality that

when $\alpha_{j}$ are not all equal. This important observation appears to have gone largely unnoticed by the literature. Similarly, if each patient $i$ is diagnosed using a point on the human ROC curve $\left(\alpha_{i,j}, \beta_{i,j}\right)$ and if $\alpha_{i,j}, i=1,\ldots,n_j$, are not all equal, it is also the case that

Lemma (ref) in the supplementary appendix provides a set of primitive conditions under which the aggregate human decision maker FPR/TPR lies strictly below the machine ROC curve.

In general, without the restrictions in Assumptions (ref) and (ref), the decision maker $j$ may use a model $\hat y = \mathds{1}\(q_{j}\(x,u\right) > c_{j}\(x,u\right)\right)$. For decision maker $j$ minimizing expected loss of misclassification conditional on the feature $x$ and $u$, consider the following loss matrix:

where the misclassification losses $c_{01}$ and $c_{10}$ are now dependent on $x$ and $u$. Equivalently, decision maker $j$ also minimizes the ex ante expected loss:

The optimal decision rule is $\mathds{1}\(q_{j}\(x, u\right) > \frac{c_{01}\(x, u\right)}{c_{10}\(x, u\right) + c_{01}\(x, u\right)} \eqqcolon c_{j}\(x, u\right)\right)$. Consequently, for $\(x_{1}, u_1\right)$, $\(x_{2}, u_2\right)$ sharing the same propensity score $q_{j}\(x_{1}, u_1\right) = q_{j}\(x_{2}, u_2\right)$, the choice of the decision maker can be different due to cost differences: $c_{j}\(x_{1}, u_1\right) \neq c_{j}\(x_{2}, u_2\right)$. Such a decision rule is considered in elliott2013predicting. Even if all doctors have full information processing capacity, so that $q_j\(x,u\right)=p\(x,u\right)$ for a correctly specified propensity function $p\(x,u\right)$, an arbitrary FPR/TPR pair below the machine ROC curve can still be rationalized by some decision rule $\hat y = \mathds{1}\(p\(x,u\right) > c_j\(x,u\right)\right)$ corresponding to a reverse-engineered cost function $c_j\(x,u\right)$.

To illustrate using a synthetic example without $u$, let $x$ be uniformly distributed on $\left(0,1\right)$ and $p\(x\right) = x$. Let the cutoff threshold function be

Then $\hat y = \mathds{1}\(x < \frac12\right)$ and $(\text{TPR, FPR})=(1/8, 3/8)$, which lies strictly below the diagonal line. An example when the feature may enter into the cutoff threshold is bank lending, where $y \in \{0,1\}$ denotes whether a loan is paid back, and $\hat y \in \{0,1\}$ denotes whether a loan application is approved. The amount of principal and interests are features $x$ that are likely to affect the probability $p\(x\right)$ of defaulting by the borrower, but that are also likely to affect the cost perception $c\(x\right)$ by the lender. Assumption (ref) essentially excludes such difficulty of interpreting a FPR/TPR pair arising from incentive heterogeneity.

To understand whether the cost perception of doctors might vary systematically with the risk level of their patients, we use the FPR rank for each doctor to measure the conservativeness of their diagnoses. Doctors may be more willing to tolerate a higher FPR for higher risk patients when they perceive riskier patients as less healthy and more likely to incur a higher cost of underdiagnosis.

We then estimate the correlation between the FPR ranks and a pregnancy risk level indicator. In particular, we regress $q_{i,j}$, the predicted abnormal birth rate for patients diagnosed by doctor $j$ from a random forest algorithm described in section (ref), against $r_j$, the doctor's FPR rank:

The doctors' FPR rank are normalized to $\left[0, 1\right]$. If the FPR of doctor $j$ is higher than 80% of the doctors, then $r_{j} = 0.8$. The regression results for doctors with no less than 300 diagnoses and for doctors with no less than 500 diagnoses are summarized in Table (ref). The regression results suggest that there is no strong evidence that the FPR rank of doctors is associated with the risk level of their patients.

The use of statistical significance as a benchmark for comparing doctors with the machine algorithm is motivated by the difficulty of fixating a social cost function, even if we assume that informed human decision makers are fully rational and impose strong functional form assumptions. Designing a social loss function empirically relies on the domain knowledge of the specific decision maker. Our replacement algorithm allows for the choice of the cutoff threshold to be specific to each individual doctor, and does not require a homogeneous preference function to be applied to all doctors. The baseline comparison between true positive rates and false positve rates can also potentially mask the severity of individual cases, the identification of which requires a richer dataset with more precise measurements on the outcome variable. While these information are not available in our dataset, exploring richer data information can be a fruitful future direction of research.

A responsible phase-in protocol based on multiple years of data on performance observations can incorporate long term health and wellbeing outcomes into the definition of performance and quality for decision making. Our data is cross-sectional and does not follow doctors or patients over time. When a panel data becomes available in which doctors and patients can be observed over multiple time periods, future work can potentially generalize the current approach to allow for sequential testing and decision making. The challenge of accounting for longer term health outcomes is partially reflected by the current strategy that replaces a doctor’s diagnosis by the machine algorithm only when there is overwhelming statistical evidence suggesting the superiority of the machine decision.

Data and Algorithm Description

Our data came from the National Free Prepregnancy Checkups (NFPC) project. Starting from 2010, NFPC offers free health checkup for couples planning to conceive and is conducted across 31 provinces in China. The dataset contains more than 300 features for each observation, including age, demographic characteristics, results from medical examinations and clinical tests, disease and medication history, pregnancy history, lifestyle and environmental information of both wives and husbands. The data also include the birth outcome, which is denoted as normal ($y = 0$) or abnormal ($y = 1$). In addition, the dataset records doctors' IDs and diagnoses of pregnancy risk. The doctors' diagnoses are graded according to 4 levels: 0 for normal, 1 for high-risk wife, 2 for high-risk husband, and 3 for high-risk wife and husband. In this paper, we group the doctor's diagnoses into 2 categories, where a grade of 0 corresponds to normal pregnancy and any higher grade corresponds to a diagnosis of risky pregnancy.

Our original dataset includes 3,330,304 couples that have pregnancy outcomes between January 1, 2014, and December 31, 2015. We exclude duplicate features and samples for which either doctors' diagnoses or more than 50% of feature values are missing. We were left with 168 features of 1,137,010 couples who were diagnosed by 28,716 doctors. Of these observations, 61,184 couples (5.38%) had abnormal birth outcomes.

A basic measurement of the prediction quality of a binary classifier is accuracy, which is the proportion of correct predictions. However, accuracy itself is inadequate in many applications. As the adverse birth rate is about 5%, a naive classifier that categorizes all cases as low risk would achieve an accuracy of nearly 95%. This is clearly controversial. The doctors' overall accuracy is 73.63%, and 24.04% of pregnancies are diagnosed as high-risk. Doctors' aggregate false positive rate and true positive rate are 0.2379 and 0.2843. In contrast, the FPR and TPR of the naive predictor are both zero. In other words, doctors are willing to tolerate a higher FPR in order to achieve a higher TPR.

We focus on doctors who diagnosed more than 300 patients. There are 584 such doctors corresponding to 584,181 patient cases. Using a ratio of 7:3 and stratification methods, we randomly split our sample into a classification set and a performance set. We use the classification set to train a machine learning algorithm, and then we categorize a doctor as “replaced” or “retained” according to our statistical procedure. The performance set is used to compute the FPR/TPR of the combined diagnoses. The data cases are stratified into the four classes based on the actual birth outcomes and the doctors' diagnoses: true positive cases, true negative cases, false negative cases and false positive cases. The random splitting process is performed through each class of data with the same ratio of 7:3. Subsequently, the data in each class are merged to form the classification and performance sets. In total, there are 408,661 cases in the classification set and 175,520 cases in the performance set.

We further split the classification set into two subsets. The first subset is the training set used to estimate the parameters in a machine learning algorithm. The second subset is the validation set used to evaluate the performance of the algorithm and obtain the machine ROC curve. Splitting of the classification set into the training and validation subsets is done using a ratio of 4:3 and the same stratification method described above. There are 233,435 cases in the training set and 175,226 cases in the validation set.

We experimented with several algorithms. We begin with decision trees, which are widely used in many machine learning applications. However, a single tree method usually has high variance despite its low bias, and tends to overfit by generating results with substantial variation even when a small amount of noise is present in the data. In contrast, random forest (RF) is a commonly used ensemble learning algorithm proposed by breiman2001random, and overcomes these problems by constructing a collection of decision trees that are trained using different feature subspaces and bootstrapped samples. The predictions of each tree are aggregated to make predictions. Appendix (ref) provides more details about the random forest algorithm.\footnote{We choose the following parameter settings. The number of estimators $N$ is $100$. The number of max features per node $M$ is $50$. The minimum number of samples required to split is 50.}

Empirical Results for AI and doctor decision making

This section presents the empirical results from the heuristic frequentist approach described in section (ref) and the Bayesian approach described in section (ref). Doctors who are not replaced by the machine algorithm are called retained doctors hereafter.

Results of the heuristic frequentist approach

The frequentist approach identifies a dominating segment of the machine ROC for machine algorithm classification. The quality of machine algorithm classification is evaluated by averaging over a uniform grid of points on this dominating segment. For each doctor, we obtain the 95% elliptical confidence area of $\theta_{H}$ centered around the sample estimates using an estimate of the asymptotic covariance matrix based on 100 bootstrap samples. We find the largest TPR point $\left(\alpha_{\beta_{high}}, \beta_{high}\right)$ and the smallest FPR point $\left(\alpha_{low}, \beta_{{low}}\right)$ of the confidence set. The reference point of the doctor is then given by $P = \left(\alpha_{low}, \beta_{high}\right)$, as shown in Figure (ref), which is used to classify doctors into two groups. When a doctor's reference point $P$ is below the machine ROC curve, corresponding to case 1 in section (ref), the doctor is replaced by the algorithm. When a doctor's reference point $P$ is above the machine ROC curve, corresponding to cases 2 or 3 in section (ref), either because this doctor is more capable than the algorithm or this doctor's capability cannot be precisely measured due to fewer patient observations and a larger confidence set, the doctor is retained in decision making.

Among the 584 doctors, 175 doctors are classified by the frequentist approach as replaced by the algorithm. Consequently, patient cases in the performance set are diagnosed by 175 machine doctors (30.0%) and 409 (70.0%) human doctors. For each of the replaced doctors, $N = 100$ points on the dominating segment of the machine ROC curve are chosen corresponding to a uniform grid of the threshold value $c$ for the algorithm to make classification decisions, as summarized in Appendix (ref). Figure (ref) shows the results of this experiment. The aggregate FPR/TPR pair of all doctors on the performance set, represented by the blue point, is 0.2065 for FPR and 0.2264 for TPR. The ROC curve of the random forest model on the performance set is the green curve. The area under the curve (AUC) is 0.6851.

The yellow point uses a decision threshold $c$ corresponding to the upper endpoint of the dominating segment of the machine ROC curve associated with a highest FPR/TPR pair of 0.2017 and 0.3326. Compared to the raw data where all diagnoses are performed by doctors, the yellow point increases the TPR by 46.9% and reduces the FPR by 2.3%. The green point corresponds to the lower endpoint of the dominating segment of the machine ROC curve and is associated with a lowest FPR/TPR pair of 0.1836 and 0.2974. Compared to the aggregate doctor diagnoses, the green point increases the TPR by 31.4% and reduces the FPR by 11.1%.

The cyan interval in Figure (ref) reflects a trade-off between improving the TPR and reducing the FPR. Any point along the interval represents an enhancement over the aggregated doctors' FPR/TPR pair. Furthermore, the red point achieves the maximum F1 score\footnote{The F1 score is the harmonic average of precision and recall, and is often used as a single measurement that conveys the balance between precision and recall. It is widely used in evaluating the performance of algorithms (see for example baeza1999modern).} of 0.1309 along the dominating segment of the machine ROC curve. Figure (ref) is a scatter plot of the FPR/TPR pairs of both replaced and retained doctors against the ROC curve in the performance dataset. We see that two adjacent points can be of different replacement status. This can be due to different numbers of patients treated or the use of the test set for calculating the FPR/TPR pairs. An important caveat is that our test depends crucially on the sample size of the number of patients treated by each doctor, which introduces an implicit bias that favors experimenting with junior doctors until the null hypothesis that a doctor is not to be replaced is rejected with sufficient evidence from seeing a large number of patient cases.

When the quality of the stock of decision makers may evolve over time, allowing the juniors to learn from making potentially suboptimal decisions for their patients has the potential of increasing the future pool of high quality seniors. This is likely to be a very delicate issue involving complex welfare tradeoff between current and future generations of patients.

Results of the Bayesian approach

The prior distribution $\pi(t)$ is chosen to be a symmetry Dirichlet distribution with the parameters $\left(\gamma_1, \gamma_2, \gamma_3, \gamma_4\right)$ all set to $\gamma = 0.01$. For doctor $j$'s $i$th patient case, $\hat{Y}_{ij}$ is the doctor's diagnosis and $Y_{ij}$ is the ground-truth label of pregnancy outcome.

The likelihood for data is given in (ref) where $\hat Y_i$, $Y_i$ and $n$ in (ref) are replaced by $\hat Y_{ij}$, $Y_{ij}$ and $n_j$. The posterior distribution is also a Dirichlet distribution with parameters $(\hat{\gamma}_1, \hat{\gamma}_2, \hat{\gamma}_3, \hat{\gamma}_4)$ where $\hat{\gamma}_{k} = \gamma + n\hat{t}_k$ for $k = 1, 2, 3, 4$. The posterior distribution of $\theta_{H} = (\alpha_{H}, \beta_{H})$ can be simulated in a two step procedure. In the first step, $R$ samples of $\{t_r = (t_{1r}, t_{2r}, t_{3r}, t_{4r})\}_{r = 1}^{R}$ are drawn from the posterior Dirichlet distribution with parameters $(\hat{\gamma}_1, \hat{\gamma}_2, \hat{\gamma}_3, \hat{\gamma}_4)$. In the second step, we compute $\{\theta_{H, r} = \left(\alpha_{H}\(t_{r}\right), \beta_{H}\(t_{r}\right)\right)\}_{r = 1}^{R}$ with the simulated $R$ samples of $t_r, r=1,\ldots,R$, using (ref).

The simulated draws $\theta_{H, r}, r=1,\ldots,R$ are used to calculate (ref). If $\hat{q}_{max} \geq 0.95$, we mark the doctor as replaced. The threshold $c$ that corresponds to the point $\left(\hat{\alpha}_{max}, g\left(\hat\alpha_{max}\right)\right)$ given by (ref) is used as the decision threshold for the algorithm.

In Figure (ref), the maximum coverage rate $\hat q_{max} = 0.9657$ at the black point on the machine ROC curve. The doctor is replaced by the algorithm. In Figure (ref), the maximum coverage rate $\hat q_{max} = 0.3179$. There is not sufficient evidence to replace the doctor's decisions with the algorithm's.

Figure (ref) shows the experiment results for doctors who had diagnosed more than 300 cases. A total of 269 (46.1%) out of 584 doctors are classified as replaced. The remaining 315 (53.9%) are retained human doctors. The combined decisions by the retained human doctors and the algorithm's decisions indicated in the figure by the yellow point, result in an FPR of 0.1856 and a TPR of 0.3319, corresponding to an increase of 46.6% in the TPR and a reduction of 10.1% in the FPR.

Similar to Figure (ref), Figure (ref) is a scatter plot of the FPR/TPR pairs of both replaced and retained doctors against the ROC curve in the performance dataset.

We experiment with several alternative definitions described in equations (ref) to (ref) for the Bayesian posterior expected loss functions in (ref). These results are shown in Figure (ref), where panel \subref{fg:ai-doctors-bayesian-score-horizonal-lower-300} magnifies a portion of panel \subref{fg:ai-doctors-bayesian-score-horizonal-upper-300} containing the replacement paths. Ranking the doctors by their expected posterior losses allows us to replace only the doctors whose expected loss of being substituted by the machine algorithm is relatively low. Figure (ref) traces the dynamics of the aggregate FPR/TPR pairs of combining the doctor with the machine decisions, when the ratio of replaced doctors increases from 0% to 100%.

In Figure (ref) legends, {\it Bayesian Test} refers to the baseline construction in (ref); {\it Euclidean Distance to the ROC curve} refers to the loss function in (ref); {\it Complementary Set Boundary Euclidean Distance} refers to the loss function in (ref); {\it Diagonal Line Horizontal and Vertical Decompositions} refer to the loss function in (ref); {\it Complement Set Boundary Vertical and Horizontal Decompositions} refer to the loss function in (ref). No individual loss function demonstrates saliently stronger discriminatory power over the entire replacement path. Furthermore, the incremental benefit going beyond a 50% replacement rate is a small marginal reduction of FPR. In addition, the baseline 95% credible level Bayesian test implemented using (ref) is not dominated by any of the competing methods in the FPR/TPR comparison.

The null hypothesis of both the Bayesian analysis and the frequentist analysis is the status quo of human decision making. A human decision maker is replaced by the machine algorithm only when there is strong evidence from the data favoring the machine algorithm. An alternative conceptual paradigm is to treat the machine algorithm as the status quo null hypothesis, where the human decision makers are by default replaced by the machine algorithm unless there is strong data evidence validating the superiority of human decision making.

However, when we experiment with the alternative specification of the status quo, all except a few doctors are determined to be replaced by the machine algorithm. In the cross-point labeled as the “dominate method” in Figure (ref), a doctor is retained if there exists a point on the machine ROC curve that is dominated by at least 95% of the posterior distribution of $p\left(\theta_H \vert \mathbb{D}\right)$. A mere 6 doctors are retained in this method. In the cross-point labeled as the “above method” in Figure (ref), a doctor is retained if the posterior probability of $p\left(\theta_H \vert \mathbb{D}\right)$ above the machine ROC curve is more than 95%. Only 7 doctors are retained using this method. For doctors who are replaced by the machine algorithm, the replacement point on the ROC curve is determined by the baseline Bayesian tests in (ref).

At high replacement ratios, the FPR and TPR levels decrease simultaneously across all loss functions. Even when all the doctors are substituted by the machine algorithm, the resulting FPR/TPR pair still lies below the machine ROC curve. This is because different threshold cutoff values on the machine ROC curve are employed to replace different doctors and because of the concavity arguments formalized in Lemma (ref).

On the one hand, in the current empirical dataset, only a small fraction of doctors have FPR/TPR pairs above the machine ROC curve. We have not been able to combine doctors with the machine algorithm to generate a FPR/TPR pair above the machine ROC curve. On the other hand, using a data generating process that allows for a substantial portion of doctors with FPR/TPR pairs to be above machine ROC curve, reported in the synthetic data analysis section (ref), we are able to combine the doctors and the machine algorithm to generate a FPR/TPR pair that outperforms even the machine ROC curve.

Instead of deterministically replacing some doctors by the machine algorithm, Appendix (ref) reports the results of a randomized replacement experiment where each replaced doctor accepts the machine decision probabilistically according to a pre-specified acceptance rate $\lambda\in\left[0,1\right]$. As expected, the overall FPR/TPR pairs vary incrementally from when $\lambda=0$, corresponding to human decision making, to when $\lambda=1$, corresponding to deterministic replacement of doctors by the machine algorithm.

Appendix (ref) considers a more general form of substitution to replace a subset of patient cases for each doctor. We use patient case covariates to predict the likelihood that the doctor's diagnoses differ from the ground-truth outcome, and replace only those patient cases where the doctor is prone to make mistakes. While the resulting pattern of the replacement paths is quite different, the patient-case specific substitution approach does not show saliently stronger discriminatory power compared to the baseline Bayesian approach.

Characteristics of replaced doctors

Our paper identifies a subset of the doctors to be replaced by the machine algorithms according to FPR/TPR indicators. We create a dummy variable “replaced” that is set to 1 when a doctor is replaced by the algorithm and is set to 0 otherwise, and examine the partial correlation coefficient between the “replaced” dummy variable with several confounding factors including whether they practice in a clinic or hospital.

We regress the “replaced” indicator on three confounding factors. The first factor is county-level per capita GDP (in ten thousand Chinese yuans) collected manually from the statistical yearbooks of 2014 across cities and provinces, which is the starting year of our data\footnote{The statistical yearbook of year $A$ records the latest available data before year $A$.}. The second factor is the logarithm of the total number of patient cases diagnosed by each doctor. The third factor is represented by two dummy variables related to the practice history of each doctor. A doctor in our dataset may practice in more than one type of facility. The practice facility can be either a county hospital or a township clinic. Medical facilities in China are typically classified by the administrative level that they belong to. On the one hand, urban residents usually go to a county level hospital. On the other hand, rural residents often times go to a township level clinic, which is generally considered to be of lower quality than a county level hospital. The doctors in our data fall into three categories. Doctors who have practiced in both types of facilities are used as the reference level. The first dummy variable representing the third factor is set to $1$ for doctors who have practiced only in a county hospital. The second dummy variable representing the third factor is set to $1$ for doctors who have practiced only in a township clinic. There are additional variables in the dataset that are only available at a more aggregated the city or province level, such as birth rate and the average number of medical staff. We do not use these variables because, given how administrative units are divided in China, a municipal unit can include many county level units with heterogenuous economic conditions.

The data summary statistics are reported in Panel A of Table (ref). As shown in Panel A of Table (ref), doctors who have practiced at both a county hospital and a township clinic have seen the largest number of patient cases and have a high replacement ratio. Doctors who have practiced only at a township clinic have seen a smaller number of patient cases compared to doctors who have practiced only at a county hospital, but doctors who have practiced only at a township clinic have a higher replacement ratio than doctors who have practiced only at a county hospital have.

Panel B in Table (ref) reports a logistic regression of the replacement indicator on county level per capita GDP and the logarithm of the number of patients seen by the doctor. Column (2) adds an indicator of whether the doctor practices only in a township clinic, and an indicator of whether the doctor practices only in a county hospital. The reference level where both indicators are zero corresponds to doctors who have practiced in both a township clinic and a county hospital. Province fixed effects are included in the logistic regression. We use cluster robust standard errors in order to correct for spatial autocorrelation.

The logistic coefficient estimates do not contradict our a priori reasoning. While higher per capita GDP is associated with a lower likelihood of replacement, the corresponding coefficients are small and lack statistical significance. The coefficients of the log number of patients are positive and statistically significant, indicating the higher distinguishing power of the Bayesian test when a doctor sees more patients. The FPR/TPR pairs of doctors with more patient cases can be estimated more precisely. Consequently, it is more likely for these doctors whose FPR/TPR pairs are below the machine ROC curve to be identified as replaced by our statistical procedure. This finding is consistent with the lack of evidence contradicting Assumption (ref) that doctors accumulate experience from additional patient cases.

Doctors practicing only at township clinics have higher replacement probability than reference level doctors practicing at both township clinics and county hospitals, who in turn have higher replacement probability than doctors practicing only at county hospitals. Anecdotal evidence in China suggests that doctors with better medical school records are more likely to be assigned directly to county level hospitals than to have to work their ways up from township clinics into county hospitals.

To distinguish an analysis of the predictors of performance of doctors from the predictors of tests concluding whether they are to be replaced by the machine algorithm, we implement a kernel regression of TPR on FPR and a number of confounding features including county level per capita GDP, the logarithm of the number of patients seen by the doctor, an indicator of whether the doctor practices only in a township clinic, and an indicator of whether the doctor practices only in a county hospital.\footnote{We thank the associate editor and the review team for suggesting a direct analysis of the performance of doctors.} Among doctors achieving the same FPR, those achieving a higher TPR can be considered to perform better. A confounding variable with a positive coefficient enhances the TPR after controlling for FPR, and is interpreted as a variable that positively predicts performance.

Panel C in Table (ref) reports the results from this regression. The regression coefficients have signs that are consistent with a priori reasoning, but they are also mostly insignificant except for the coefficients on the indicator of whether the doctor practices only in a county hospital. For a given FPR, a higher TPR is positively associated with higher county level per capita GDP, a larger number of patients seen by the doctor and the indicator that the doctor practices only in a county hospital, but is negatively associated with the indicator that the doctor practices only in a township clinic.

The statistical insignificance of the coefficients on the logarithm of the number of treated patients in Panel C of Table (ref) and their significance in Panel B of Table (ref) provide further evidence that the replacement indicator is largely driven by the difference in the sample size. A larger sample size makes it easier to reject the null hypothesis that the machine algorithm does not outperform the doctor.

Synthetic Data Analysis

An example of human-algorithm complementarity

In our empirical dataset, only a small fraction of doctors lie above the machine ROC curve. Combining the machine algorithm and the doctor decisions results in aggregate FPR/TPR pairs that are still below the machine ROC curve. In addition, Jensen's inequality (elaborated in section (ref)) likely contributes to the combined aggregate FPR/TPR pair lying below the machine ROC curve. Even if we replace every doctor by the machine algorithm, unless the same point on the machine ROC curve is used to replace all doctors, the combined aggregate FPR/TPR pair is necessarily below the machine ROC curve. The end points of the replacement paths in Figure (ref) correspond to replacing all doctors by the machine algorithm. The algorithms that we implemented use different points on the machine ROC curve to replace doctors. These points also differ across different doctors. In order for the combined aggregate FPR/TPR pair to be above the machine ROC curve, it is necessary that a sizable portion of the original doctor diagnoses lie above the machine ROC curve.

While not the case in the NFPC dataset, other datasets might exist where a significant portion of human decision makers correctly use private information not available to the machine algorithm. Figure (ref) reports the results from a synthetic data analysis where the combined aggregate FPR/TPR pair between doctors and the machine algorithm lies above the machine ROC curve.

The data generating process of the simulation in Figure (ref) is as follows. The input features are three dimensional: $x_{i,j,1}$, $x_{i,j,2}$ and $u_{i,j}$. On the one hand, the first two features, denoted $x_{i,j,1}$ and $x_{i,j,2}$, are assumed to be observable both by the doctors and the machine algorithm. On the other hand, the last feature, denoted $u_{i,j}$, is only observable to the doctors and is not used by the machine algorithm. The three features are generated from independent normal distributions with pre-specified means and variances: $X_{i,j,1}\sim N\left(0,1.95\right)$, $X_{i,j,2} \sim N\left(0,0.25\right)$ and $U_{i,j} \sim N\left(0,2\right)$. The ground truth variable $Y_{i,j} \in\{0,1\}$ is generated by

and where $\delta_{i,j}$ is uniformly distributed between $0$ and $1$. Using this specification of the data generating process, we simulated 600,000 patient cases and allocate them evenly to 2000 doctors. Each doctor is assigned 300 simulated patient cases.

Doctors are partitioned into two groups according to their information processing capacities. The first group, consisting of 750 {\it more capable} doctors, use the correct full information propensity score $p\(x_{i,j,1}, x_{i,j,2}, u_{i,j}\right)$ with different incentives to diagnose patient cases. Their decision rule is given by, using $C_j$ that is uniformly distributed between $0.4$ and $1$ across doctors,

The second group of doctors, consisting of 1250 {\it less capable} doctors, use a misspecified propensity score to diagnose patients. Their decision rule is

is a misspecified propensity score function.

We divide the 600,000 simulated observations into a training subset consisting of 240,000 observations, a validation subset consisting of 180,000 observations and a test subset consisting of the remaining 180,000 observations. We estimate a random forest model using only the training subset and the first two features $x_{i,j,1}$ and $x_{i,j,2}$, and use the baseline Bayesian test in equation (ref) to combine doctors with the machine algorithm. The red point in Figure (ref) represents the combination of doctors and the machine algorithm. It lies above the ROC curve of the random forest model, and also strictly dominates the blue point of the aggregate FPR/TPR pair of all doctors. In summary, the synthetic data analysis represented by Figure (ref) illustrates the scientific principle of complementarity in human algorithm collaborations. See for example bansal2021does.\footnote{ We are grateful to an anonymous referee for pointing us to the science literature on human-algorithm collaboration complementary.}

Aggregating information by predicted doctor models

Our dataset contains doctor diagnoses information in addition to the eventual pregnancy outcomes. In this section we discuss whether estimating a model that predicts the label of doctors' diagnoses can assist in improving the FPR/TPR tradeoff of medical decisions. A model of predicted doctors can be intuitively interpreted as doctors' collective wisdom. The experience of senior doctors can provide guidance to junior doctors with limited clinical exposures.

In the NFPC dataset, a model of predicted doctors does not appear to improve on the aggregate FPR/TPR pair. However, in several of the following synthetic data examples, a predicted doctor model improves upon the diagnoses of individual doctors. Each doctors' diagnosis may encode information heterogeneity, or incentive heterogeneity, or both. Consider a model of incentive heterogeneity, where doctors' decision rules are given by $\hat y_{i} = \mathds{1}\(p\(x_{i}\right) > u_{i}\right)$ such that $p\(x_{i}\right)$ is the correctly specified propensity score, and $u_i$ is independently distribution with a strictly monotonic CDF $G\left(\cdot\right)$. On the one hand, the aggregate doctor FPR/TPR lies below the optimal ROC curve. On the other hand, the predicted doctor model, given by $\mathbb{E}\left[\mathds{1}\(p\(X_i\right) > U_i\right) \vert X_i\right] = \mathbb{P}\(U_i \leq p\(X_i\right)\right) = G\(p\(X_i\right)\right)$, is a monotonic transformation of the true propensity score function and recovers the optimal ROC curve.

We next focus on several information model without incentive heterogeneity. Consider a baseline example where doctors have access to private information $u_i$ in addition to the publicly observed features $x_i$ and are aware of the correct full information propensity score model $p\(x_i, u_i\right)$. If we could directly observe doctors' probabilistic assessment $p\(x_i, u_i\right)$ and use it as the label to train a machine learning model, then the resulting model will necessarily coincide with the ground-truth machine learned propensity score model:

However, in reality we rarely solicit doctors' probabilistic assessment of the likelihood of an illness. Instead, we only observe the binary diagnosis by the doctors, which corresponds to observing $\hat y_i=\mathds{1}\(p\(x_i, u_i\right) > c_0\right)$ for some threshold value $c_0$. When we use $\hat y_i$ as the label to train a machine learning model, the propensity score of the resulting predicted doctor model is

In general, $q\(X_i\right)$ is different from $p\(X_i\right)$. The ROC curve implied by $q\(X_i\right)$ coincides with $p\(X_i\right)$ only when $q\(X_i\right)$ is an increasing transformation of $p\(X_i\right)$: $q\(X_i\right)=\Lambda\(p\(X_i\right)\right)$, where $\Lambda\left(\cdot\right)$ is a strictly increasing function. There is no guarantee that this is necessarily the case.

Consider now three scenarios in which doctors employ misspecified models to process private information. In the first scenario, doctors process the full information $\(X_i, U_i\right)$ using an incorrect propensity score model $q\(X_i, U_i\right)$, but the ROC curve implied by the predicted doctor model $q\(X_i\right)$ still coincides with the ground-truth machine ROC curve. Figure (ref) illustrates this scenario using the following data generating process. The ground-truth propensity score model is $p\(X_{i}\right)=\exp\(X_{i}\right) / \left(1+\exp\(X_{i}\right)\right)$. The doctors use a misspecified propensity model of $q\(X_{i},U_{i}\right)=\exp\(X_{i}+U_{i}\right) / \left(1+\exp\(X_{i}+U_{i}\right)\right)$. Both $X_i$ and $U_i$ are uniformly and independently distributed: $X_{i} \sim U\left(-1, 1\right)$ and $U_{i} \sim U\left(-2, 2\right)$. The “$X_i$ only” ROC curve is generated by $p\(X_{i}\right)$; the “$X_i$ and $U_i$” ROC curve is generated by $q\(X_{i},U_{i}\right)$. The “Predicted Doctors on $X$” ROC curve is generated by $q\(X_i\right) \equiv \mathbb{E} \left[\mathds{1}\(q\(X_i, U_i\right) > c_0\right) \vert X_{i}\right].$

For any given cutoff threshold $c_0$, the predicted doctor model $q\(X_i\right)$ is an increasing function of $X_i$, as is $p\(X_i\right)$. In this particular case, both $p\(X_i\right)$ and $q\(X_i\right)$ correspond to the same ROC curve. This curve lies above the ROC curve generated by $q\(X_i, U_i\right)$ and the aggregate doctor FPR/TPR pair ($c_0=0.7$). Collective wisdom improves upon the misspecified model employed by individual doctors and results in higher diagnosing quality as represented by the ROC curves.

The second scenario, pictured in Figure (ref), is generated by a two dimensional observable feature model. The ground-truth propensity score model is

The doctors' (misspecified) information model is

$X_{i1}, X_{i2}$ and $U_i$ are independently distributed: $X_{i1} \sim N\left(0, 1\right), X_{i2} \sim N\left(0, 0.5\right), U_{i} \sim N\left(0,4\right)$.

In this scenario, doctors also process the full information $\(X_{i1}, X_{i2}, U_i\right)$ using an incorrect propensity score model. While the ROC curve corresponding to the predicted doctor model $q\(X_{i1}, X_{i2}\right)\equiv \mathbb{E} \left[\mathds{1}\(q\(X_{i1}, X_{i2}, U_i\right) > c_0\right) \vert X_{i1}, X_{i2}\right]$ lies below the optimal ROC curve corresponding to the correct propensity score model $p\(X_{i1},X_{i2}\right)$, it still lies above the ROC curve corresponding to the individual doctor misspecified information model $q\(X_{i1},X_{i2},U_{i}\right)$ and the aggregate doctor FPR/TPR pair. Collective wisdom in this case leads to limited improvement over the misspecified information model.

The third scenario is pictured in Figure (ref). The ground-truth propensity score model is $p\(X_{i}\right)=\exp\(X_{i}\right)/\left(1+\exp\(X_{i}\right)\right)$ but the doctors' incorrect information model is

where $X_{i}$ and $U_{i}$ are distributed as in the first scenario depicted in Figure (ref). In this model, doctors' information processing capacity is very limited. The predicted doctor model $q\(X_i\right) = \mathbb{E}\left[\mathds{1}\(p\(X_i, U_i\right) > c_0\right) \vert X_i\right]$ exacerbates the misspecification of the doctors' private information propensity score model, and leads to a ROC curve that lies strictly below the ROC curve corresponding to the individual doctor misspecified propensity score model $q\(X_{i},U_{i}\right)$.

The analysis in Figure (ref) shows that depending on the extent of misspecification of the doctors' information model, the predicted doctor model that aggregates the information among doctors using the subset of observable features available to the machine learning algorithm might either reduce or exacerbate the misspecification in the propensity score model. The resulting ROC curve of the predicted doctor model might lie above or below the ROC curve of the doctors' information model.

Replacement strategies initiated by doctors

For a given decision maker, recognizing that machine algorithms are more likely to be applicable in some candidate cases than in others is important for a replacement model to achieve human and machine complementarity. Formalizing such a general empirical replacement model is difficult because of the unknown diagnosing model employed by doctors in the data and because of our lack of knowledge of doctors' preferences for the machine algorithm. To illustrate these difficulties, we experiment with a set of simulations in which doctors decide when to employ machine learning algorithm.

In the simulation setup, each patient is associated with three independently generated normally distributed features $X_1, X_2, U$, where $X_1 \sim N(0, 0.5)$, $X_2 \sim N(0, 1.5)$ and $U \sim N(0, 1)$. The probability of each patient being ill is a logit function of the features:

The labels are generated according to the probability model $p\(X_1, X_2, U\right)$ for 30,000 patient cases. The diagnosis of doctors is modeled as a mixture between two decision rules. The first rule employs the correct probability model $p\(X_1, X_2, U\right)$. The second rule replaces the probability model $p\(x\right)$ by an uninformative uniformly distributed random variable $r \sim U\left(0,1\right)$. The second rule applies to a situation where the occasional lack of medical knowledge by doctors results in randomized diagnosis. Specifically, we simulate doctor diagnosis $\hat Y$ by

In the above, $c_d = 0.5$ is the decision threshold for doctor diagnosis. The simulated dataset is divided into training, validation and test subsets by the ratio of 4:3:3. They are used to estimate model parameters, compute the ROC curve and calculate FPR/TPR pairs respectively. We first train a random forest model using features $X_1$ and $X_2$, where $U$ is interpreted as unobserved heterogeneity that is known to the doctors but not to the machine algorithm.

We simulate four replacement rules. They are illustrated in Figure (ref). In the first rule, shown in Figure (ref), doctors know when their medical knowledge is inadequate (e.g. when $X_1 < 0$) and choose to defer to the machine algorithm in such a situation. This method achieves complementarity between the human doctor and the machine algorithm. It produces a FPR/TPR pair that is better than the Bayesian approach in wich either all or none of the patients for each doctor are replaced. In the second rule, shown in Figure (ref), doctors have partial but incomplete knowledge of when their medical knowledge is inadequate. Doctors defer to the machine algorithm when $X_1 < 0.5$. The second rule improves over the doctors, but does not outperform the Bayesian approach. Figure (ref) use a replacement rule of $X_2 \le -0.5$. In this case, the doctor does not know when they lack medical knowledge and the replacement reduces both FPR and TPR. In Figure (ref), doctors mistakenly defer to the machine algorithm when they do have medical knowledge (e.g. when $X_1 \geq 0$), but make their own decision when they are uninformed (e.g. when $X_1 < 0$). The resulting FPR/TPR pair is strictly dominated by the original doctor diagnosis.

Conclusion

In this paper, we propose statistical tests for replacing human decision makers with algorithms. Decision rules that robustly dominate diagnoses made by humans can be determined based on the machine ROC curves. We propose both a heuristic frequentist approach and an alternative Bayesian posterior loss function approaches. By replacing a subset of decision makers with algorithms, we can achieve higher quality decision making outcome. We experiment with the two approaches using a dataset of pre-pregnancy checkups, and find that the replacement of a subset of doctors using the Bayesian approach significantly improves the overall performance of risky pregnancy detection. Furthermore, our results indicate that doctors who practice only at township clinics are more likely to be replaced, while doctors who practice only at county hospitals are less likely to be replaced.

Our analysis shows that both stable preference and decision-making quality are important for comparing machine algorithm with human decision makers. While allowing the juniors to make potentially suboptimal decisions for their patients and learn from the results has the potential of increasing the pool of high quality seniors, this is a very delicate issue involving complex welfare tradeoff between current and future generations of patients. In panel datasets with rich dynamic information, human decision makers can engage in information updating and peer learning. Our current dataset is only cross-sectional. The eventual pregnancy outcomes are typically not available to doctors when they diagnose consecutive patient cases. Juniors might also benefit more generally from offline learning without having to make substantial and high stake decisions.

Acknowledgments

We thank the editors and two anonymous referees, Brendan Beare, Bo Honore, Xiaohong Chen, Ron Gallant, Bruce Hansen, Peter Hansen, Bentley MacLeod, Jack Porter, Adam Rosen, Andres Santos, George Tauchen, Valentin Verdier, and participants at various seminars and conferences for insightful comments. We also thank Yichuan Zhang, Xin Lin and Zhonghao Huang for excellent research assistance. This study was approved by the National Health Commission. Informed consents were obtained from all the NFPC participants. We acknowledge funding support from the National Science Foundation (SES 1658950 to Han Hong), the National Science Fund for Distinguished Young Scholars of China (71325007 to Ke Tang), the State’s Key Project of Research and Development Plan (2016YFC1000307 to Jingyuan Wang) and the National Natural Science Foundation of China (61572059 to Jingyuan Wang).

\FloatBarrier \phantomsection \addcontentsline{toc}{section}{References} { \setstretch{0.875} }

\processdelayedfloats