Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
108,710 characters · 19 sections · 46 citation commands
Designing Algorithmic Recommendations to Achieve Human--AI Complementarity
\xdef\@thefnmark\@footnotetext{ Bryce McLaughlin ([email removed]), Wharton Healthcare Analytics Lab, University of Pennsylvania and Jann Spiess ([email removed]), Graduate School of Business, Stanford University. We thank Maya Balakrishnan, Sanmi Koyejo, Jason Weitze, Emma Rockall, Adrienne Propp, Carl Meyer, Omer Shiran-Cohen, Haosen Ge, and audience members at the 2023 & 2024 INFORMS Annual Meetings, 2024 MSOM Conference, Stanford, and Wharton for helpful comments and suggestions. }
Algorithms often assist, rather than replace, human decision-makers. For example, in the United States, federal pretrial service officers set bail for defendants after observing a pretrial risk assessment cadigan_implementing_2011, rather than letting the algorithm take a bail decision automatically. Similarly, doctors may be assisted by clinical decision support systems throughout the patient care process sutton_overview_2020 and managers may be assisted by pre-employment assessments throughout the hiring process raghavan_mitigating_2020. In all of these cases, the human maintains the final decision authority. However, the design and analysis of these algorithms often focus on the prediction of outcomes as if algorithmic recommendations were implemented directly, rather than on how these recommendations affect human decisions.
The narrow focus of existing approaches on predicting outcomes rather than optimizing for human--AI complementarity becomes of particular concern in light of a growing body of evidence that shows that human decision-makers with access to algorithmic recommendations often make worse decisions than if these recommendations had been implemented directly and, in some cases, may even make worse decisions than they would have taken without access to the algorithm. hemmer_human-ai_2021 identifies 53 articles that report (i) unassisted human performance, (ii) algorithm performance, and (iii) algorithm-assisted human performance on a decision task. Only in 16 of these does the algorithm-assisted human decision-maker perform best. In many of these studies, decision-makers exhibit noisy and biased adoption of the algorithm's recommendations, which counteracts the gains associated with the additional information the algorithm provides. This suggests the necessity to incorporate decision-makers' adoption decisions explicitly when designing the algorithm.
This article formalizes the design of recommendation algorithms that assist humans in making binary treatment choices via an optimization problem that leverages the potential-outcomes framework from causal inference. To analyze recommendation algorithms, we employ the notion of “potential decisions” and make connections to the analysis of instrumental variables in causal inference with imperfect compliance. This view suggests that an optimal recommendation algorithm should take into account the final outcome of interest to which the decision is related as well as the decision-maker's choice of when to comply with a recommendation. We then show that this framework helps design better recommendation algorithms and make sense of their empirical performance through an online experiment. Finally, we derive robust recommendation algorithms that achieve minimax performance from limited training data. While our theorems and propositions explicitly hold for this stylized setup, the framework we develop is broadly applicable for designing algorithms that assist human decision-makers. Specifically, we connect algorithm design choices to the assumptions the designer is willing to make about the behavior of the human decision-maker, exposing the central role of human decision data to the design of these tools.
In (ref), we introduce a principal--agent model that formalizes the design of AI recommendation systems that assist a human in making binary treatment choices. As a motivating example, we consider a manager making hiring decisions on applicants evaluated by a pre-employment assessment. The principal in our problem is the company, which designs an algorithm that assists a hiring manager (the agent) in making the ultimate hiring decision. This model allows for complementarity in information and expertise between the principal's algorithm and the agent, thus moving beyond models in which perfect adherence to the algorithmic recommendation is optimal.
(ref) connects our problem setup to the potential-outcomes framework from causal inference. There, we introduce a monotonicity assumption that allows us to describe the effect of recommendations in terms of compliance, similar to the analysis of instrumental variables by imbens_identification_1994. The potential-outcomes framework formalizes how individual outcomes or actions change in response to treatment choices. We use this framework to capture how different recommendations given by the algorithm lead to different decisions by the human decision-maker. Using our “potential decisions”, we can state the optimal recommendation algorithm as minimizing an objective that separates into two parts, (1) the loss incurred by a direct implementation of the algorithm's recommendations and (2) the loss incurred (or avoided) due to how decision-makers deviate from the recommendations. Our monotonicity assumption then simplifies the second part since it implies that recommendations can only move the decision-maker toward the recommended action.
In (ref), we showcase our framework by applying it to an online experiment that involves a hypothetical hiring task. This experiment asks subjects to make 25 hiring decisions and randomly varies the recommendation algorithm across subjects. This setup allows us to evaluate how the structure of recommendation algorithms impacts ultimate task performance. Specifically, the separate recommendation algorithms are based on different assumptions about compliance behaviors. In line with our framework, we find that subjects perform better when the recommendations are designed to provide complementary information, rather than optimized for hiring decisions directly. The best-performing algorithm assumes that human decision-makers comply with the algorithm selectively, only impacting the cases where the human is uncertain. The decisions assisted by this algorithm outperform unassisted human decisions as well as the best performance by an algorithm on its own. We thus demonstrate that our framework can help design systems that improve decisions through human--AI complementarity.
Finally, in (ref), we leverage our approach to find robust recommendation algorithms from limited data. We focus on the realistic case where the principal may only have data about unassisted human decisions available and thus has to rely on assumptions about compliance and active decisions. Assuming that the decision-maker is more likely to comply in cases where they do not make good decisions themselves, we derive minimax optimal recommendation rules. Our result shows that learning successful recommendations is about learning from mistakes rather than from the relationship of outcomes to characteristics alone. We end by discussing the resulting robust optimization problem that can be implemented using machine learning.
Our work responds to an empirical literature that analyzes inefficiencies in the interactions between human decision-makers and prediction algorithms. This literature has identified cases where human decision-makers imperfectly use prediction algorithms in high-stakes decisions in criminal justice imai_experimental_2021,stevenson_algorithmic_2022,angelova_algorithmic_2023, healthcare agarwal_combining_2023,maron_artificial_2020, and child protective services fogliato_case_2022,mills_impact_2022, so much so that the assisted human decisions are often no better than unassisted human decisions. A more extensive review of this literature is detailed by lai_towards_2021. Of the works in this literature, our empirical approach is most related to that green_principles_2019, which posits ways human decision-makers should respond to algorithms and tests these hypotheses in the lab. Relative to this literature, we focus on designing complementary recommendations, rather than focusing on describing ways in which humans misuse algorithms.
We join a collection of works spanning the areas of operations, economics, and computer science that design algorithms to assist humans in making better decisions. bansal_beyond_2019 and donahue_human-algorithm_2022 propose improvements to collaborative decision processes based on the private information of each agent. balakrishnan_improving_2022 and orfanoudaki_algorithm_2022 find that humans often incorrectly assume linear relationships and then develop strategies to help human--algorithm decision systems overcome this issue. athey_allocation_2020 and mclaughlin_algorithmic_2022 model how effort costs and reference points, respectively, affect a human's adoption of algorithmic assistance. snyder_algorithm_2023 shows how workers' compliance with an algorithm's recommendation in a service system varies with the system's utilization. baek_policy_2023 and bastani_improving_2022 apply ideas from reinforcement learning to incorporate past human actions into the recommendations of assistive algorithms. caro_believing_2023 and sun_predicting_2022 use data on workers' deviations from algorithm advice to increase compliance. noti_learning_2023 focuses on when to provide assistance, and shows that strategically restricting advice can preserve the complementary strengths of the algorithm and human. Our approach differs from these prior works as it provides a causal framework to capture how a human's response varies with the underlying algorithm. Similar to the work of ibrahim_eliciting_2021 (where human predictors assist algorithmic decision-makers), our results highlight the importance of private information in designing impactful collaboration.
Our methods build upon a literature that optimizes the design of estimators that inform decision-making. We combine models of communication between heterogeneous agents kamenica_bayesian_2011,andrews_model_2021,spiess_optimal_2022 with approaches from causal inference with imperfect compliance angrist_identification_1996,vytlacil_independence_2002,mogstad_identification_2018 to design optimal recommendation algorithms in the prescriptive sense bertsimas_predictive_2020. This approach parallels work that transfers concepts from causal inference for use in decision-making manski_econometrics_2021,manski_probabilistic_2023, in particular the approach in saghafian_ambiguous_2023 of incorporating unobserved confounding using sets of causal models and the approach in ben-michael_safe_2022 of learning robust decision policies from non-stochastic treatment assignments.
We introduce a principal--agent model of joint human--algorithm decision-making
. In this model, the principal designs an AI system that can give recommendations to an agent. The agent is a human decision-maker who observes each recommendation and subsequently makes a binary treatment choice. As our main illustration, we apply this model to the problem a company (principal) faces when implementing an algorithmic pre-employment assessment to assist a manager (agent) in the hiring process. The company wants its managers to hire good workers and not hire bad workers. The company impacts the managers' decisions only through the recommendation algorithm they implement. We allow for the managers to have both unknown information and preferences, allowing for complex relationships between the algorithm the company implements and the decisions that managers take.
We consider the problem of taking a binary decision $D$, after which a binary outcome $Y$ is revealed. We capture the impact of this decision through a loss function $\ell$, which describes the loss from implementing $D$ compared to the optimal decision. For each problem instance, we assume access to characteristics $X$ that have predictive power for $Y$.
As a running example throughout this paper, we consider a hiring decision $D \in \{\text{{\normalfont\color{red}N}}, \text{{\normalfont\color{blue} H}}\}$ as the binary treatment choice with the ability $Y \in \{\text{{\normalfont\color{red} B}},\text{{\normalfont\color{blue} G}}\}$ capturing the quality of work produced by the applicant, which is unknown at the time of the decision. We measure the (dis-)utility associated with this decision via the loss function,
The type-I error cost $c_I > 0$ captures the cost to the company of not hiring a good ({\normalfont\color{blue} G}) worker (relative to hiring them). The type-II error cost $c_{II} > 0$ captures the cost of hiring a bad ({\normalfont\color{red} B}) worker (relative to not hiring them).
While worker ability $Y$ is ex-ante unknown, we assume that there is data $X$ collected about the worker that can be accessed by an algorithm ahead of the decision $D$ being made. We do not assume any specific structure of $X$, as the data available about applicants before hiring may include their resume, written question responses, video responses to interview questions, or the actions the worker took during a game li_algorithmic_2021. We do require that the information $X$ is collected, documented, and made available prior to making a hiring decision $D$ on each worker, and for simplicity assume throughout that $X$ has finite and full support.
We examine the decision $D$ through the lens of a principal--agent model with asymmetric information and potentially misaligned preferences. Specifically, we assume the decision $D$ is taken by an agent who (1) observes private information $U$, (2) does not observe characteristics $X$ directly, and (3) may not share the preferences represented by the loss function $\ell$. We allow for arbitrary overlap between the information in $X$ and $U$, so the model allows for some or even all of the characteristics $X$ to be available to the agent.
To improve the decisions by the agent, the principal (whose preferences are captured faithfully by $\ell$) implements an algorithm $f^{\textnormal{rec}}$ that provides recommendations $R = f^{\textnormal{rec}}(X)$ to the agent. Thus, similar to the information design literature kamenica_bayesian_2011,bergemann_information_2019, the principal has partial control over the information presented to the agent before the agent makes the final decision. We think of the algorithm as provided by an AI system that may learn from training data, but we abstract from the training process and focus on the resulting recommendation rule $f^{\textnormal{rec}}$ itself.
Returning to our hiring example, we assume the principal is a company that implements an algorithmic recommendation system $f^{\textnormal{rec}}$ that uses the characteristics $X$ on workers to generate hiring recommendations $R = f^{\textnormal{rec}}(X)$, which influence the hiring decisions of the agent, who is a manager at the company. The recommendations reduce the unstructured applicant data $X$ to a simpler form that may still be informative about the worker's ability $Y$. For simplicity, we limit the algorithm to either recommend a decision to the manager (not hire, $R =\text{{\normalfont\color{red}N}}$, or hire, $R =\text{{\normalfont\color{blue} H}}$) or not make a recommendation at all ($R = {{\color{gray}\ensuremath{\varnothing}}}$). In practice, recommendations may present more granular predictive information, such as a score that represents a scaled probability that a worker is good ($Y=\text{{\normalfont\color{blue} G}}$) and allows the managers to understand the algorithm's relative ranking of different workers. It may also include an explanation of the score, highlight pieces of $X$ the algorithm deems particularly relevant, or include certainty information about the confidence of its prediction raghavan_mitigating_2020.
The manager responds to the recommendation by making a hiring decision $D$ which may depend on the recommendation $R$ along with their private information $U$ and unknown private preferences. This information $U$ consists of all knowledge the manager has of the applicant, including potentially some of the information contained in $X$ (such as the candidate's resume) as well as new information the manager learns through their own conversations with the applicant (such as prior relationships with employees at the company or interest in areas the company is looking to expand). Thus, we allow arbitrary correlation between $X$ and $U$, but do not allow the manager to observe $X$ directly. This effectively captures the idea that $X$ and $U$ contain shared information, but that neither necessarily dominates the other. Additionally, the manager may exhibit preferences that do not directly align with $\ell$, preferring to hire individuals with whom they get along well (or have prior relationships) even when those applicants are less likely to be good than others. This flexibility allows us to capture the complex decisions managers exhibit empirically. We summarize the relationship between the elements of our model in (ref).
Throughout, we include the option that the algorithm does not make a recommendation, $R={{\color{gray}\ensuremath{\varnothing}}}$, forcing the manager to make an active decision. We believe that this design feature has formal advantages for modeling recommendations and practical relevance for achieving human--algorithm complementarity. First, we can consider unassisted decisions, those taken without any recommendation algorithm, as equivalent to those taken under a recommendation algorithm which always sends $R = {{\color{gray}\ensuremath{\varnothing}}}$, making unassisted decisions a special case of our framework. Second,
forcing active decisions by avoiding recommendations in specific cases may reduce distortions in cases where human decision-makers know better. These beliefs are further supported by the work of noti_learning_2023, which finds that partially withholding algorithmic assistance (i) increases humans' reliance on advice when given and (ii) can be leveraged strategically to improve humans' prediction accuracy.
Having set up a principal--agent problem for thinking about the design of recommendation algorithms, we now use the potential-outcomes framework from causal inference to discuss how the agent's decision depends on the recommendation algorithm. Viewing the agent's decision in this way suggests that the descriptive and prescriptive analysis of recommendations shares common features with the investigation of instrumental variables in inference problems. We strengthen this connection by applying a monotonicity assumption similar to that of imbens_identification_1994, which allows us to represent the remaining potential outcomes in terms of the active decision the agent makes when no recommendation is given and whether the agent complies with the recommendation. This leads to a decomposition of the recommendation algorithm's objective based on the agent's compliance choice. The objective captures the performance of the recommended action if the agent complies and the performance of the agent's active decision otherwise.
We now interpret the agent's decision in terms of the potential-outcomes framework of rubin_estimating_1974. Specifically, we write $D_U(r;f^{\textnormal{rec}})$ for the “potential decisions” that the agent would take when receiving the recommendation $r$ from the recommendation algorithm $f^{\textnormal{rec}}$ when the agent's information is $U$. Thus, the agent's decisions can be seen as the random variable $D = D_U(R;f^{\textnormal{rec}})$, where $R = f^{\textnormal{rec}}(X)$. This expanded framework is represented visually in (ref), which expands (ref) by including potential outcomes.
The potential decisions $D_U(r;f^{\textnormal{rec}})$ depend not only on possible recommendation values $r$ of $R$, they also depend on the whole mapping $f^{\textnormal{rec}}$ from characteristics $X$ to recommendations $R = f^{\textnormal{rec}}(X)$. This is because how a specific recommendation is interpreted depends on how recommendations are given overall. For example, if an algorithm recommends hiring for everybody, then it may change individual decisions differently from it recommending to hire only a few applicants. This is related to the concept of interference in causal inference, where potential outcomes may be functions not just of their own treatment assignment, but also of those of others. In (ref), the line between the recommendation algorithm and the potential decision clarifies that potential decisions may depend on the overall recommendation algorithm itself. This dependence highlights the challenge in estimating good recommendation algorithms under minimal assumptions or using limited data (which we return to in (ref)). Effective ways of forming recommendation algorithms will, therefore, rely on assumptions that limit the complexity of potential decisions, including their dependence on the overall recommendation algorithm. \footnote{ Without additional assumptions, this interference challenge implies, in our case, that the potential decision function $D_U(\cdot;f^{\textnormal{rec}})$ for a specific instance $(X,U)$ is only ever evaluated at its implied recommendation $R=f^{\textnormal{rec}}(X)$. However, by assuming that the potential decision function $D_U(\cdot;f^{\textnormal{rec}})$ only depends on $U$, this notation also captures variation in recommendations given for instances that are otherwise equivalent from the perspective of the human decision-maker. That is, for a given instance $U$, they capture how decisions vary across possible realizations of $f^{\textnormal{rec}}(X)$ stemming from variation in $X$. This formalization, therefore, implies that two instances that present with the same $U,R$ values are not distinguishable to the decision-maker, leading to the same decision even if their $X$ varies. (Here, any noise in decisions would be captured through $U$.) }
Our framework parallels instrumental variables that affect outcomes only through their impact on a treatment variable. In our hiring example, the algorithm's recommendations (corresponding to the instrument) impact the company's utility (playing the role of the outcome) only through managers' decisions (treatment). Rather than directly optimizing for the recommendation, the company must understand how the managers' decisions $D$ vary in response to receiving different recommendations from different algorithms. Specifically, as the manager's information $U$, the recommendation $R$, and the algorithm $f^{\textnormal{rec}}$ are sufficient information to determine $D$, and we control the recommendation algorithm $f^{\textnormal{rec}}$ (and through it $R$), we write the manager's decision as the potential outcome $D_U(r;f^{\textnormal{rec}})$ for each of the possible values $r \in \{\text{{\normalfont\color{red}N}},{{\color{gray}\ensuremath{\varnothing}}},\text{{\normalfont\color{blue} H}}\}$ of the recommendation. Since decisions are themselves binary and the recommendations take three values, we can enumerate these potential outcomes into eight triplets which we categorize into four response types based on how the realized recommendation impacts the assisted managers' decisions. These response types are similar in nature to the types developed by angrist_identification_1996 for the analysis of instrumental variables.
(ref) enumerates the potential outcomes associated with a manager's decision response to recommendations and provides the aforementioned categorization. If the manager maintains the same decision regardless of the recommendation, they $\textit{Ignore}{}$ the recommendation (similar to always-takers and never-takers in causal inference). If they adopt the recommendation regardless of whether the algorithm recommends not hiring ({\normalfont\color{red}N}) or hiring ({\normalfont\color{blue} H}) the applicant, then they $\textit{Comply}{}$ with the recommendation. If, instead, the manager always takes the decision that the algorithm does not recommend, then they $\textit{Defy}{}$ the recommendation. If none of these cases hold, then it must be the case that the manager will $\textit{Change}{}$ their decision in response to the algorithm sending a recommendation, taking one decision when the algorithm sends no recommendation and taking the opposite decision if either recommendation is sent. While we maintain all these response types for now, we will later simplify the analysis by ruling out the $\textit{Defy}{}$ and $\textit{Change}{}$ types, analogous to the monotonicity assumption of angrist_identification_1996.
Having introduced a framework to capture potential decisions in response to recommendations, we now consider the optimal recommendation algorithm as the solution to an optimization problem that minimizes the loss achieved by the decisions the algorithm induces. We show that the recommendation's objective can be separated into a triage term that assumes that recommendations are implemented directly and a term that captures the agent's endogenous response to the recommendation. We compare this optimal recommendation algorithm to solutions that instead directly target optimal decisions, and provide an intuition for why that optimal decision rule will rarely be the optimal recommendation rule.
In general, the optimal recommendation algorithm is the one that minimizes the expected loss of the decision taken by the human decision-maker after seeing the recommendation,
In our hiring example, the optimal recommendation algorithm for the company to implement is the mapping from applicant worker data to recommendations that leads to good hiring decisions by the manager. As this example illustrates, this optimization problem is generally quite difficult to solve, given the uncertainty about how managers will respond to recommendations. As a result, companies often implement algorithms that instead recommend the optimal decision based on the information available to the algorithm,
implicitly assuming that the recommendation $R=f(X)$ gets implemented directly ($D=R$) and ignoring the algorithm's ability to not send a recommendation ($R = {{\color{gray}\ensuremath{\varnothing}}}$). An extension suggested in previous work by raghu_algorithmic_2019 is to triage instances into those where algorithms outperform humans and those where humans outperform algorithms. We write $\ensuremath{D^0}_U = D_U({{\color{gray}\ensuremath{\varnothing}}};f^{{{\color{gray}\ensuremath{\varnothing}}}})$ where $f^{{\color{gray}\ensuremath{\varnothing}}} \equiv {{\color{gray}\ensuremath{\varnothing}}}$ for the decision taken by the manager in the absence of a recommendation algorithm. If we assume that the manager takes this decision when not given a recommendation ($f(X) = {{\color{gray}\ensuremath{\varnothing}}}$) and that otherwise the recommendation is implemented directly, then this objective and its solution correspond in our framework to
In practice, recommendations may not be followed perfectly. In addition, even the decision for $R = {{\color{gray}\ensuremath{\varnothing}}}$ may itself be different from the decision $\ensuremath{D^0}_U$ without a recommendation algorithm, and we may specifically hope it to improve since additional information and effort may be available for those cases raghu_algorithmic_2019,athey_allocation_2020.
We now compare these different objectives and solutions, and show that the optimal recommendation ($f^{\textnormal{rec}*}$) differs from triage ($f^{\textnormal{tri}*}$) in that it also incorporates the compliance behavior of the manager and considers how active decisions may differ from unassisted decisions. Specifically, the loss function that defines the optimal recommendation can be decomposed into the loss function of the triage problem (which we call the triage effect, TE) and the response effect (RE) from managers deviating from the triage decision,
In addition to the triage effect from (ref), this decomposition has a second part representing deviations from recommendations and unassisted decisions. It can represent both improvements (in cases where the human decision-maker uses their private information to override a suboptimal recommendation or adjusts their active decision based on the knowledge they receive through the recommendation algorithm) as well as mistakes (when the manager goes against a correct hiring recommendation or takes an active decision that is worse than in the case without any recommendations).
The two components also highlight the challenge of learning good recommendations from data. The triage effect (TE) is straightforward to estimate, provided the company has access to historical data through which they can learn the joint distribution of characteristics $X$, ability $Y$, and unassisted manager decisions $\ensuremath{D^0}_U$. On the other hand, the response effect depends on the potential outcomes associated with the managers' decisions, which complicates its evaluation for any $f^{\textnormal{rec}}$ under which decision data has not already been collected. Understanding $D_U(r;f^{\textnormal{rec}})|X,Y$ may require more than simple experimental evaluation based on local changes in recommendations. These decision distributions may vary arbitrarily with changes to $f^{\textnormal{rec}}$. Solving for the optimal recommendation then requires understanding this distribution across a set of reasonable recommendation algorithms $\mathcal{F} \subseteq \{f; f:X \mapsto R\}$. This makes designing an optimal recommendation algorithm an intractable task without additional assumptions (even when we just want to robustly outperform the unassisted decisions $\ensuremath{D^0}_U$).
The relationship between algorithms that solve for the different objectives above is illustrated in (ref). Plotting a set of candidate algorithms ($\mathcal{F}$) according to their triage effect (TE) and response effect (RE) clarifies, in particular, the relationship between optimal triage and optimal recommendation. The optimal triage ($f^{\textnormal{tri}*}$) minimizes the triage effect without considering the response effect related to each algorithm. The optimal recommendation ($f^{\textnormal{rec}*}$) trades off additional loss through the triage effect with loss incurred due to the response effect. In practice, this means the optimal recommendation may be sub-optimal if applied directly and instead relies on the manager strategically overriding recommendations to make better decisions. However, both $f^{\textnormal{rec}*}$ and $f^{\textnormal{tri}*}$ lie on the Pareto frontier that simultaneously minimizes the triage and response effects. This does not have to be true of the optimal algorithmic decision rule $f^{\textnormal{dec}*}$, which generally incurs a higher triage effect TE than the optimal triage solution $f^{\textnormal{tri}*}$. This is because the triage solution $f^{\textnormal{tri}*}$ comes from minimizing expected triage loss (ref) subject to a less strict constraint than the decision solution $f^{\textnormal{dec}*}$. Meanwhile, the difference in the response effect between the $f^{\textnormal{dec}*}$ and $f^{\textnormal{tri}*}$ rules may be positive or negative.
Above, we have proposed a potential-outcomes framework for agent decisions in response to recommendations and used this framework to discuss the design of optimal recommendation algorithms. We now introduce assumptions that limit the potential decision to simplify their analysis. We then use these simplifications to shed further light on the structure of the optimal recommendation algorithm in terms of agent compliance and active decisions.
In the context of the four response types introduced in the hiring example and repeated in (ref), manager behavior consistent with the Ignore and Comply responses is easier to make sense of than behavior consistent with the Defy and Change responses. If the manager believes the algorithm made a bad recommendation given their private information $U$ about the applicant, they are likely to $\textit{Ignore}{}$ the recommendation. Conversely, if they believe the algorithm makes a good recommendation given their private information $U$ about the applicant, they will $\textit{Comply}{}$. Using recommendations as a threshold requirement for considering an applicant, as done by hiring managers interviewed by li_algorithmic_2021, also generates potential outcomes consistent with only $\textit{Ignore}{}$ and $\textit{Comply}{}$ responses. On the other hand, $\textit{Defy}{}$ and $\textit{Change}{}$ behaviors are counter-intuitive and typically need bad recommendation algorithm designs, perverse correlation between $X$ and $U$, or adversarial managerial incentives to be justified rationally. Consequently, we now make an assumption to eliminate the potential outcomes associated with $\textit{Defy}{}$ and $\textit{Change}{}$ responses, similar to the monotonicity assumption of angrist_identification_1996 in the context of instrumental variables and its extensions to multivariate discrete treatments lee2018identifying.
At a high level, this assumption restricts the direction of the managers' response to follow the algorithm's recommendation for the applicant. Hire recommendations ($R =\text{{\normalfont\color{blue} H}}$) can turn not-hire decisions into hire decisions ($D = \text{{\normalfont\color{red}N}} \to D = \text{{\normalfont\color{blue} H}}$) but cannot turn hire decisions into not-hire decisions ($D = \text{{\normalfont\color{blue} H}} \not\to D = \text{{\normalfont\color{red}N}}$). Not-hire recommendations ($R = \text{{\normalfont\color{red}N}}$) do the exact opposite.
The monotonicity restriction exactly rules out the bottom four rows of (ref). It can be considered a restriction on plausible recommendation rules $\mathcal{F}$. Specifically, it rules out that recommendation rules are used that would make sense to Defy (such as a rule that suggests hiring bad candidates) or that would make sense to observe the Change behavior for (such as a rule that only gives a recommendation for bad candidates and never gives one for good candidates).
When the manager's decision behavior is limited by (ref), we can fully describe their behavior for some specific recommendation rule $f^{\textnormal{rec}}$ in terms of their active decisions $\ensuremath{D^{\varnothing}}_U = D_U({{\color{gray}\ensuremath{\varnothing}}};f^{\textnormal{rec}})$ which they exhibit in response to the algorithm $f^{\textnormal{rec}}$ being implemented, along with their compliance behavior $\ensuremath{C}_U$:
(ref) visualizes (ref) by restating the remaining potential-decision types in terms of $(\ensuremath{D^{\varnothing}}_U,\ensuremath{C}_U)$ once (ref) is applied. The only information determining the decision $D$ outside of the recommendation $r$ is the active decision $\ensuremath{D^{\varnothing}}_U$ and whether the response type $\ensuremath{C}_U$ of the manager is $\textit{Ignore}{}$ or $\textit{Comply}{}$. The proposition follows directly from an inspection of (ref). For completeness, a formal proof is given in (ref).
As an immediate consequence of the simple representation of response types in (ref), we can decompose the objective of the recommendation algorithm in terms of compliance:
Similarly, this explicit notation allows us to shed further light on the response effect from (ref) in terms of compliance and deviations between active decisions and unassisted decisions:
The first term in this decomposition expresses deviations from recommendations when they are ignored, while the second term captures the change in behavior for those instances where no recommendation is given that comes from the presence of the recommendation algorithm.
Even with the simplifications introduced by (ref), it is not generally possible to learn the optimal recommendation algorithm from past data $(Y,X,\ensuremath{D^0}_U)$ on outcomes, characteristics, and unassisted human decisions. This is because the joint distributions of active decisions and compliance with outcomes and characteristics, $(Y,X,\ensuremath{D^{\varnothing}}_U,\ensuremath{C}_U)$ cannot be identified from such limited data without further assumptions. However, we can now formulate conditions on these unobserved quantities to derive optimal learnable recommendation algorithms in specific cases, thus providing a formal analog and extension of the discussion in (ref):
At the same time, more nuanced assumptions or data on compliance behavior may be necessary to achieve complementarity in practical cases. This may include the cases below:
In our experiment below, we combine information about the point distribution of information $(Y,X,U)$ with assumptions on active decisions $\ensuremath{D^{\varnothing}}_U$ and compliance $C_U$ in order to solve for sensible recommendation algorithms.
\FloatBarrier
We now apply our framework to the design and analysis of recommendation algorithms in a controlled trial. In a pre-registered online experiment, study subjects participate in an incentivized hiring game. \footnote{The pre-analysis plan for this experiment can be found at \href{https://www.socialscienceregistry.org/trials/11857}{www.socialscienceregistry.org/trials/11857}.} This empirical application showcases our framework's ability to map realistic assumptions about agent behavior to the design of practical recommendation algorithms that realize complementarity between humans and algorithms.
In the experiment, we collect hypothetical hiring decisions while varying the structure of the recommendation algorithm at the subject level. We apply different assumptions on how subjects will behave within our framework to generate these recommendation algorithms. We find that subjects, on average, improve their decisions with algorithms whose recommendations are focused on providing suggestions that are complementary to the subjects' private information. Moreover, subjects given these focused recommendations perform better than subjects who see naive recommendations based on optimal algorithmic predictions and outperform decisions by the human or algorithm alone.
We implement a hiring game as part of an online experiment to test subjects' performance when using recommendation algorithms in a setting where both the subject and algorithm have private information. Specifically, we label 25 hypothetical applicants as good or bad based on the role they are applying for and their personality type, and ask subjects to only hire good applicants. The role is only known to the human subject and the personality type is only known to the algorithm. We measure the change in the accuracy of subjects' hiring decisions in response to different recommendation algorithms (treatments) that vary at the subject level.
For each subject, the experiment proceeds as follows. Subjects arrive at a Qualtrics survey from the recruiting site Prolific and learn they will make hiring decisions on 25 hypothetical applicants, of which there are good and bad types (referred to as the applicant's ability). They learn that each applicant's ability $Y$ is determined by the relationship of the applicant's personality type $X$ (denoted by letters A--E) to their intended role $U$ (Engineering, Sales, or Communications), as shown in (ref). Then they answer three comprehension questions about how ability $Y$ is determined and, conditional on answering correctly, we randomize them into a treatment.
Subjects receive recommendations from algorithms to assist them with making hiring decisions, except for those subjects in the control arm. Unlike the study subjects, these algorithms can access the applicants' personality types, but not their role. \footnote{Subjects then answer three more comprehension questions about the specific algorithm they will interact with, but continue in the experiment regardless of their answers. We leverage these additional questions for robustness checks below.} Subjects then make 25 hiring decisions with access to the distribution of applicants as represented in (ref), the recommendations of their assigned algorithm, and an explanation of that specific recommendation algorithm. An example of a hiring decision is given in (ref). When making these hiring decisions, subjects know the applicant's role, but are unaware of the applicant's personality type beyond any information conveyed by the recommendation.
We incentivize subjects to hire good applicants and to not hire bad applicants. Specifically, as compensation for participating in our experiment, subjects receive \$2.04 for their time (estimated to be 12 minutes) and the chance to earn a \$2.00 bonus as a performance incentive. They earn this bonus if their hiring decision on a randomly selected applicant is correct (hire if good, not hire if bad). Note that this induces equal type I and type II losses ($c_I = c_{II}$) from the perspective of our subjects, and we analogously evaluate their performance using the percentage of correct decisions taken. Subjects see all 25 applicants in a random order without replacement to make their hiring decisions but don't learn whether they made correct decisions.
The joint distribution of applicant role, personality type, and ability (pictured in (ref)) is designed to test subjects' willingness to $\textit{Comply}{}$ with recommendations across applicant roles while allowing the recommendation to provide useful information in cases the applicant role is relatively uninformative. When subjects review an applicant for an Engineering role, they know the worker is good. Subjects are much more uncertain for Sales applicants (60% of which are good) and Communications applicants (40% of which are good). The algorithm's private information (personality type) varies significantly across the cases in which subjects are uncertain, thus containing useful information to help them make their decisions. Moreover, both the subject and algorithm's private information are of equal value in that a rational agent with access to either the role or the personality type would take the correct decision $76\%$ of the time.
The recommendation algorithm a subject receives represents the treatment in our experiment and is randomized at the subject level. In addition to four treatment arms representing different recommendation algorithms, there is a control arm in which no recommendation is given. These recommendation algorithms differ in the way they map personality types $X$ to recommended actions $R = f^{\textnormal{rec}}(X) \in \{\text{{\normalfont\color{red}N}},{{\color{gray}\ensuremath{\varnothing}}},\text{{\normalfont\color{blue} H}}\}$, where the recommendations follow our earlier example and correspond to not hiring, no recommendation, and hiring the applicant, respectively. We now describe the four different algorithms and link them to the implicit assumptions about compliance and active decisions that motivate their design. Additionally, (ref) lists the recommendations that these algorithms provide across personality types.
We summarize the logic behind these algorithms by examining their implicit assumptions about the subjects' compliance choices and active decisions. Specifically, (ref) presents each algorithm as a solution to a specific combination of assumptions. For compliance choices, we distinguish between perfect compliance (that is, following the recommendation and only making an active decision when no recommendation is given) and selective compliance. By selective compliance, we mean that subjects always make an active decision for Engineering candidates and comply otherwise. For active decisions, we distinguish between completely random decisions and sophisticated decisions. Sophisticated decisions assume that the subject uses their private role information effectively when they take active decisions, that is, only hiring Engineering and Sales candidates, which is optimal absent recommendations and across each of our treatment arms. For example, if compliance is perfect and decisions are sophisticated, then we assume that subjects follow all recommendations and correctly decide in the absence of a recommendation when they do not receive one.
Using these combinations of behaviors, we can interpret the above treatment arms as optimal recommendation algorithms. First, the Predictive algorithm is optimal with perfect compliance and random active decisions. Second, the assumptions under which the Triage algorithm is optimal are that compliance is perfect and active decisions are sophisticated. In this case, the subject follows the recommendation when it is given and otherwise relies on their private information to decide. Third, the Complementary algorithm is optimal when subjects comply for non-Engineering candidates only, and make random decisions when no recommendation is sent. Fourth, the Complementary Triage recommendations achieve optimal outcomes when subjects do not comply for (and always hire) Engineering candidates, and effectively use their private information to make a sophisticated decision when they do not receive a recommendation. If the human were replaced by a rational agent, this treatment arm would achieve the best performance on average, with only one mistake (not hiring a type-E communications candidate).
Although final recommendations only depend on the personality type $X$, we note that constructing these algorithms requires an understanding of the joint distribution of $X$ with the outcomes $Y$ as well as the role $U$. Whether such a joint distribution can be learned in practice depends on the type and quality of data that is available at the time of the construction of the algorithm. Here, we focus instead on an analysis of the relative performance of different recommendation algorithms in a setting where we (and the study subjects) know the joint distribution of $(Y,X,U)$. In (ref), we will return to the question of how to design recommendations when the distribution of $U$ remains unknown and our only insight into the humans' behavior is through their unassisted decisions ($\ensuremath{D^0}_U$).
Before analyzing the results of the experiment, we describe the data collection process, final sample sizes for each treatment, and basic performance statistics. Among 1675 surveys begun, about 35% failed to answer at least one of the initial comprehension questions and were screened out, leaving us with 1083 subjects that completed our study. Subjects were split approximately equally between all arms except Control, which received half as many observations as the other treatments. Specifically, each of the four treatment arms contained approximately 240 participants who, like the approximately 120 Control subjects who did not get recommendations, each took 25 hiring decisions for a total of around 6,000 decision responses per treatment and 3,000 for Control. Overall, the subjects performed well in the experiment and within the estimated time frame. The subjects of our experiment made the correct decision 78.1% of the time. The average duration of the experiment was around twelve minutes. However, both the performance (standard deviation of 13 percentage points) and duration (standard deviation of almost eight minutes) show significant heterogeneity across subjects.
We compare treatments by average subject performance, with the main results reported in (ref). Subjects in the Predictive treatment, who received recommendations from an algorithm that was trained to predict outcomes, got around 76% of hiring decisions correct, significantly above the around 70% in the Control group that was not supplied with any recommendations. The Triage, Complementary, and Complementary Triage treatments were all designed to provide recommendations that complement human decisions, and subjects in these groups on average outperformed subjects in the Predictive treatment. Hence, the data confirms that the algorithm designed to take optimal decisions by itself (given by the \textit{Predictive} treatment) is not the optimal recommendation algorithm in this case. Across the treatments, subjects assigned to the \textit{Complementary} algorithm perform best, with over 81% optimal decisions (although this performance is statistically indistinguishable from the \textit{Complementary Triage} subjects). Overall, the results suggest that subjects tend to make smart compliance choices and frequently (and correctly) override the algorithm for Engineering candidates. At the same time, decisions are inefficiently noisy.
We compare subject performance to useful reference points in (ref) wu_rational_2023. Throughout, we focus on the fraction of cases in which subjects make the optimal decision on an applicant (hire if good, not hire if bad). Within each treatment, we compare subject performance to the performance when following the algorithm perfectly (corresponding to the Triage Effect of (ref)), and plot the impact of subject deviations (Response Effect). We also provide the best achievable performance by an algorithm acting alone (to benchmark which treatments were able to outperform algorithmic automation) as well as the performance achievable by a rational agent acting with the subject's information (to act as a performance upper bound). The decomposition reveals that subject performance tends to improve by deviating from the algorithm (at least in cases other than the Predictive arm, in which case subject performance is equivalent to perfectly following the algorithm). Interestingly, machine-assisted human performance is better for the somewhat simpler Complementary algorithm than for the theoretically optimal Complementary Triage. Overall, we think that the patterns in (ref) are in line with good compliance choices, noisy active decisions, and a penalty for overly complex recommendation algorithms.
Our experiment documents complementarity between humans and algorithms when recommendations are designed well. Subjects in the Triage, Complementary, and Complementary Triage treatments all performed significantly better (at the 5% significance level) than humans by themselves (Control) or the algorithm by itself, thus achieving human--AI complementarity, while subjects in the Predictive treatment could only match the performance humans or algorithms could (theoretically) achieve by themselves.
To understand performance differences across treatment arms further, we separate subjects' propensity to hire applicants across roles and personality types in (ref), and relate this behavior to the recommendations subjects receive. From the figure, we can identify that subjects' unassisted decisions mostly match the average quality of candidates (which would be in line with a probability-matching model where subjects match the probability they hire an applicant with the probability they are good). Their compliance choices are generally sophisticated (in that they correctly override the algorithm for Engineering candidates), although somewhat random. Note that our framework predicts that the Complementary algorithm is optimal if subjects make somewhat noisy active decisions but make selective compliance choices. This is in line with our empirical findings, where the Complementary algorithm is the best among those algorithms we tested and likely the optimal recommendation algorithm we could have implemented.
As an additional robustness check for our results in this section, we exploit the additional comprehension questions we asked subjects about the recommendation algorithm they used. Specifically, 376 of the 961 study subjects who were given a recommendation did not answer one of the comprehension questions correctly about their specific algorithm. We reproduce all analysis on both the sub-population of subjects who answered all recommendation comprehension questions correctly and those who answered at least one question wrong in (ref). Our results strengthen on the sub-population who answered all comprehension questions correctly, while, as expected, results are very noisy and largely insignificant on the sub-population of subjects who answered at least one question wrong. These results support conclusions by bansal_beyond_2019 that a subject's understanding of an algorithm is vital to achieving complementarity.
Given our descriptive framework connecting recommendation algorithm design to the human agent's counterfactual decisions, an immediate prescriptive question arises: how should we design recommendation algorithms in practice based on observed data only? Above, we showed the value of designing recommendation algorithms that achieve complementarity in a lab experiment where we know the joint distribution of the human's information and the algorithm's information. We now study how our approach can be helpful in the realistic case where we want to design recommendations based on baseline data. To do so, we first outline challenges that arise when we train recommendation algorithms when only unassisted decision data is available. We then propose a minimax approach to address these challenges. Our minimax solution has a particularly intuitive structure: it yields an algorithm that learns where the human decision-maker is likely to make a mistake and then suggests an alternative course of action in those cases. We link that solution back to the experiment from the previous section before discussing machine-learning implementations.
We consider the task of learning a good recommendation algorithm from baseline training data in which we observe characteristics $X$, outcome labels $Y$, as well as unassisted human decisions $\ensuremath{D^0}_U$, taken in the absence of algorithmic recommendations (corresponding to the Control group in the experiment). We abstract from the task of learning that distribution and assume for now that we know the full joint distribution of
Knowing this distribution, how can we design a good recommendation algorithm?
In general, finding an optimal recommendation algorithm from this baseline data alone is not feasible since relevant parts of the potential-decisions distribution are not identified based on this data alone. Just assuming monotonicity ((ref)) does not alleviate the problem. Specifically, we face the following three practical challenges:
These challenges imply that we are not generally able to solve for optimal recommendation algorithms based on baseline data alone. Instead, we have to combine assumptions on compliance, active decisions, and interference. In the spirit of our overall framework, we aim to make relatively weak assumptions. This means there is remaining uncertainty, which we resolve by searching for minimax solutions that make worst-case assumptions about unidentified parts of the distribution of potential decisions.
To determine plausible recommendation algorithms from baseline data about unassisted decisions $\ensuremath{D^0}_U$, characteristics $X$, and outcomes $Y$, we now make additional assumptions about how compliance and active decisions relate to unassisted decisions and their performance in the baseline. We then solve for minimax optimal solutions to resolve additional uncertainty about the full distribution of potential outcomes.
Recall that the optimal recommendation algorithm under monotonicity ((ref)) minimizes
under the continued assumptions from (ref), as in (ref). We now address these two components separately. For the second part, we note that we do not directly observe active decisions $\ensuremath{D^{\varnothing}}_U$, since these may differ from entirely unassisted decisions $\ensuremath{D^0}_U$ through additional information provided by the recommendation algorithm. For example, even the (non-)recommendation $R = {{\color{gray}\ensuremath{\varnothing}}}$ may act as a signal that a human decision-maker may leverage together with their private information. To handle such cases, we assume that active decisions cannot be any worse than unassisted decisions:
As a direct consequence, we can bound the loss of implementing an arbitrary recommendation algorithm $f^{\textnormal{rec}}$ relative to the loss in the unassisted baseline case by
Minimizing worst-case loss then corresponds to minimizing the remaining loss stemming from compliers. To restrict compliance, we consider its relationship to the decision-maker's mistakes in the baseline data. We define
and call $M$ a (baseline) mistake, which covers the mutually exclusive cases of a type-I error ($M_I$) and a type-II error ($M_{II}$). Were we to assume that, for a specific baseline decision and recommendation, compliance is positively correlated with baseline mistakes,
then the decision-maker would be weakly more likely to change their decision in response to a recommendation in cases where they would have made a mistake without any assistance. Thus, those cases in which a lot of the same mistakes happen could be improved by suggesting an action that goes in the opposite direction of those mistakes. Specifically, we will later establish that the recommendation rule
is minimax optimal (assuming for simplicity that only one of the two conditions is fulfilled at a time). In words, this rule checks how many mistakes were made in baseline data for a given $X$ and unassisted decision $\ensuremath{D^0}_U$. If the probability of a mistake is so high that taking the opposite decisions would improve performance, then (ref) will ensure that recommending that alternative plan of action cannot make things any worse.
We now express the idea that compliance is related to mistakes in the baseline data in a more flexible way. Specifically, we use an assumption that expresses how strongly compliance correlates with baseline mistakes. This allows for both expressing assumptions that are weaker than the one in (ref) as well as assuming that the relationship between compliance and mistakes is stronger.
This assumption generalizes (ref), which is the special case $\kappa = 1$. The assumption is not restrictive at all for $\kappa = 0$, and for $0 < \kappa < 1$ only assumes that compliance is not creating a large number of new mistakes without also correcting some old mistakes. For $\kappa > 1$, it assumes that compliance is more likely if a mistake would have been committed in the baseline and is indicative of the decision-maker using their private information to comply selectively. As $\kappa$ grows large, more and more mistakes are overridden by compliant decisions, while relatively fewer new mistakes are added. To give this assumption bite in those cases, we also assume that compliance is positive (at least conditional on mistakes):
Putting all these assumptions together, we can now derive a minimax optimal recommendation algorithm that generalizes the recommendation rule above.
This algorithm learns from mistakes: it sends a recommendation only if the unassisted decisions lead to so many mistakes that giving the opposite recommendation can only improve decisions. The structure of the algorithm may look complicated, but is intuitive. The first two rows check whether recommending $\text{{\normalfont\color{red}N}}$ or $\text{{\normalfont\color{blue} H}}$ are guaranteed to improve loss relative to unassisted decisions, respectively. If neither leads to guaranteed improvements in expected loss, no recommendation is given; if both lead to improvements, then the one that leads to the larger guaranteed decrease in loss is selected.
We close this part by comparing the resulting algorithm to the alternatives we discuss in (ref). Specifically, the automated decision from (ref) that an algorithm could take is to hire whenever $\mathbb{P}(Y{=}\text{{\normalfont\color{blue} G}}|X{=}x) \geq \frac{c_{II}}{c_I + c_{II}}$. While this algorithm ignores human decisions, our proposed approach focuses instead on mistakes that capture the relative performance of human vs machine decisions. The triage rule from (ref) decides whether to override human decisions based on whether they perform better or worse than automated algorithmic decisions, and is a special case of (ref).
The intuition behind this equivalence can be seen by comparing (ref) to the following representation of the triage rule,
where $M_I$ and $M_{II}$ express baseline human type-I and type-II errors as above. Specifically, both formulations show that for a recommendation to be sent, the baseline data must contain a sufficient number of mistakes. If a large number of those mistakes are type-I errors, then both the probability of a mistake given we do not hire and the probability the applicant is good will be high, leading both algorithms to send a hire recommendation. A similar argument holds for type-II errors.
Through (ref), our approach delivers insights into the connection between recommendations and triage:
Overall, these distinctions showcase that the central questions in designing algorithms for complementarity are around how the decision-maker leverages their private information to decide when to comply with a recommendation and how their active decision changes in response to changes in the recommendation algorithm.
Above, we have derived minimax algorithms that lead to a guaranteed improvement in expected loss under assumptions on compliance and active decisions. We now apply this family of recommendation algorithms to the experiment in (ref). This application demonstrates both the features and the limitations of the agnostic minimax approach.
For our application, we assume that we do not have any of the information about private information and its relationship to outcomes available, and instead only rely on information about the joint distribution of decisions, characteristics, and outcomes from the baseline (Control) arm. We see this as a conservative reference point that represents the case where we have no or little information about the setting for which we design a recommendation algorithm, making the assumptions from above appropriate.
Per our result above, the main input to finding minimax optimal recommendations using baseline data is the distribution of mistakes across characteristics. To calculate the relative improvement of different recommendations, we also require information about unassisted decisions and the distribution of characteristics. These data are represented in (ref). Together, these numbers completely describe the (empirical) baseline distribution. Here, we take this empirical distribution as given and derive the (sample analog of the) algorithms from (ref) for varying values of $\kappa$.
The resulting minimax optimal algorithms are represented in (ref) for varying values of $\kappa$. Even in the very conservative case of $\kappa \rightarrow 0$, the algorithm recommends hiring applicants of personality type D, since any deviation can only be a mistake. In this regime, no other recommendations are given. As $\kappa$ increases, hiring recommendations are also given for types B, C, and E. Around $\kappa = 1$, this leads to the triage solution from (ref). As the minimum correlation between compliance and baseline mistakes increases further, type A is recommended not to be hired (recovering the Predictive algorithm), and the recommendation for type E is eventually switched from hiring to not hiring (recovering the Complementary algorithm that performs best in the experiment). This connects to the analysis of different algorithms in the experiment in (ref) and especially in (ref): if we are willing to assume that compliance is selective (which we interpret here as a high $\kappa$), then we expect the Complementary algorithm to work well.
Note the approach in this section does not have access to any of the information about complementarity that the algorithms in (ref) relied on. Learning such additional information and reasoning about compliance and active decisions beyond the arguments above would require access to data about how decisions change with different recommendations, such as those from the treatment arms other than the Control.
Overall, this application shows both strengths and limitations of the approach behind (ref). Even with limited data, our framework can derive sensible recommendation algorithms. At the same time, the approach is inherently limited since it does not rely on any learning about how decisions change in response to recommendations in the given setting and instead is sensitive to specific assumptions. As a result, we obtain both algorithms that improve only somewhat over the baseline (like the Predictive algorithm we recover for intermediate $\kappa$) or improve strongly (the Complementary algorithm for high $\kappa$), but could not have told based on baseline data alone which algorithm would have done better. Ultimately, finding better recommendation algorithms will involve adaptive experimentation that learns about the structure of compliance and active decisions.
We close this section by discussing the practical implementation of recommendation algorithms using machine learning. We assumed above knowledge of the joint distribution of unassisted decisions, characteristics, and outcomes. In practice, these relationships would have to be learned from data. Fortunately, the algorithm above is instructive about how this learning could be implemented effectively. Specifically, the main ingredients to constructing (ref) are the probability of mistakes conditional on characteristics and unassisted decisions as well as the baseline decisions. These correspond to the solution of straightforward classification problems that estimate
Based on estimates $\hat{m}_\text{{\normalfont\color{red}N}}, \hat{m}_\text{{\normalfont\color{blue} H}}, \hat{h}$, a plug-in rule is
An alternative is to estimate the rule directly via cost-weighted classification by solving the sample analog of
where $M_I$ and $M_{II}$ are type-I and type-II errors, respectively.
This article formalizes the analysis and optimal design of recommendation algorithms within a principal--agent model that employs the potential-outcomes framework from causal inference. We draw similarities between the impact that recommendation algorithms have on human decisions in algorithmic design and the effect that instrumental variables have on individual treatment choices in inference problems. We use this connection to decompose the objective function of a recommendation algorithm into algorithmic performance and human response. Making additional assumptions on reasonable human responses to recommendations, our approach allows us to express recommendation-assisted decisions in terms of compliance and active decisions. We can then reason about optimal recommendation algorithms in terms of assumptions on each component individually. We demonstrate the utility of our framework in a controlled online experiment and a minimax optimization application, both of which reinforce that decision-maker responses should be considered when designing recommendation algorithms. Specifically, we document in the experiment that subjects perform better using recommendations that are designed to provide complementary information, relative to algorithms that are instead designed to automate decisions.