Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
39,693 characters · 11 sections · 40 citation commands
Assessing Disparate Impacts of Personalized Interventions: Identifiability and Bounds
The expanding use of predictive algorithms in the public sector for risk assessment has sparked recent concern and study of fairness considerations propublica,bs14,barocas-hardt-narayanan. One critique of the use of predictive risk assessment argues that the discussion should be reframed to instead focus on the role of positive interventions in distributing beneficial resources, such as directing pre-trial services to prevent recidivism, rather than in meting out pre-trial detention based on a risk prediction bdivz17; or using risk assessment in child welfare services to provide families with additional childcare resources rather than to inform the allocation of harmful suspicion shroff2017predictive,eubanks2018automating. However, due to limited resources, interventions are necessarily targeted. Recent research specifically investigates the use of models that predict an intervention's benefit in order to efficiently target their allocation, such as in developing triage tools to target homeless youth triage-rice-13,kube-das-19. Both ethics and law compel such personalized interventions to be fair and to avoid disparities in how they impact different groups defined by certain protected attributes, such as race, age, or gender.
The delivery of interventions to better target those individuals deemed most likely to respond well, even if a prediction or policy allocation rule does not have access to the protected attribute, might still result in disparate impact (with regards to social welfare) for the same reasons that these disparities occur in machine learning classification models chen-js18. (See (ref) for an expanded discussion on our use of the term “disparate impact.”) However, in the problem of personalized interventions, the “fundamental problem of causal inference,” that outcomes are not observed for interventions not administered, poses a fundamental challenge for evaluating the fairness of any intervention allocation rule, as the true “labels” of intervention efficacy of any individual are never observed in the dataset. Metrics commonly assessed in the study of fairness in machine learning, such as group true positive and false positive rates, are therefore conditional on potential outcomes which are not observed in the data and therefore cannot be computed as in standard classification problems.
The problem of personalized policy learning has surfaced in econometrics and computer science manski05,kt15, gaining renewed attention alongside recent advances in causal inference and machine learning athey17,wager17,dell2014. In particular, bd12 analyze optimal treatment allocations for malaria bednets with nonparametric plug-in estimates of conditional average treatment effects, accounting for budget restrictions; davis-heller use the generalized random forests method of wager2017estimation to evaluate heterogeneity of causal effects in a program matching at-risk youth in Chicago with summer jobs on outcomes and crime; and kube-das-19 use BART hill2011bayesian to analyze heterogeneity of treatment effect for allocation of homeless youth to different interventions, remarking that studying fairness considerations for algorithmically-guided interventions is necessary.
In this paper, we address the challenges of assessing the disparate impact of such personalized intervention rules in the face of unknown ground truth labels. We show that we can actually obtain point identification of common observational fairness metrics under the assumption of monotone treatment response. We motivate this assumption and discuss why it might be natural in settings where interventions only either help or do nothing. Recognizing nonetheless that this assumption is not actually testable, we show how to conduct sensitivity analyses for fairness metrics. In particular, we show how to obtain sharp partial identification bounds on the metrics of interest as we vary the strength of violation of the assumption. We then show to use these tools to visualize disparities using partially identified ROC and xROC curves. We illustrate all of this in a case study of personalized job training based on a dataset from a French field experiment.
We suppose we have data on individuals $(X,A,T,Y)$ consisting of:
Our convention is to identify $T=1$ with an active intervention, such as job training or a homeless prevention program, and $T=0$ with lack thereof. Similarly, we assume that a positive outcome, $Y = 1$, is associated with a beneficial event for the individual, e.g., successful employment or non-recidivation. Using the Neyman-Rubin potential outcome framework ir15, we let $Y(0), Y(1) \in \{0,1\}$ denote the potential outcomes of each treatment. We let the observed outcome be the potential outcome of the assigned treatment, $Y=Y(T)$, encapsulating non-interference and consistency assumptions, also known as SUTVA rubin1980randomization. Importantly, for any one individual, we never simultaneously observe $Y(0)$ and $Y(1)$. This is sometimes termed the fundamental problem of causal inference. We assume our data either came from a randomized controlled trial (the most common case) or an unconfounded observational study so that the treatment assignment is ignorable, that is, $Y(1), Y(0) \indep T \mid X,A$.
When both treatment and potential outcomes are binary, we can exhaustively enumerate the four possible realizations of potential outcomes as $(Y(0),Y(1))\in\{0,1\}^2$. We call units with $(Y(0),Y(1))=(0,1)$ responders, $(Y(0),Y(1))=(1,0)$ anti-responders, and $Y(0)=Y(1)$ non-responders. Such a decomposition is also common in instrumental variable analysis angrist96 where the binary outcome is take-up of treatment with the analogous nomenclature of compliers, never-takers, always-takers, and defiers. In the context of talking about an actual outcome, following manski1997monotone, we replace this nomenclature with the notion of response rather than compliance. We remind the reader that due to the fundamental problem of causal inference, response type is unobserved.
We denote the conditional probabilities of each response type by $$p_{ij}=p_{ij}(X,A)=\mathbb P(Y(0)=i,Y(1)=j\mid X,A).$$ By exhaustiveness of these types, $p_{00}+p_{01}+p_{10}+p_{11}=1$. (Note $p_{ij}$ are random variables.)
We consider evaluating the fairness of a personalized intervention policy $Z=Z(X,A)\in\{0,1\}$, which assigns interventions based on observable features $X,A$ (potentially just $X$). Note that by definition, the intervention has zero effect on non-responders, negative effect on anti-responders, and a positive effect only on responders. Therefore, in seeking to benefit individuals with limited resources, the personalized intervention policy should seek to target only the responders. Naturally, response type is unobserved and the policy can only mete out interventions based on observables.
In classification settings, minimum-error classifiers on the efficient frontier of type-I and -II errors are given by Bayes classifiers that threshold the probability of a positive label. In personalized interventions, policies that are on the efficient frontier of social welfare (fraction of positive outcomes, $\Prb{Y(Z)=1}$) and program cost (fraction intervened on, $\Prb{Z=1}$) are given by thresholding ($Z=\indic{\tau\geq\theta}$) the conditional average treatment effect (CATE):
where the latter equality follows by the assumed ignorable treatment assignment. Estimating $\tau$ from unconfounded data using flexible models has been the subject of much recent work wager2017estimation,shalit-johansson-sontag-17,hill2011bayesian.
We consider observational fairness metrics in analogy to the classification setting, where the “true label” of an individual is their responder status, $R = \indic{Y(1)>Y(0)}$. We define the analogous true positive rate and true negative rate for the intervention assignment $Z$, conditional on the (unobserved) events of an individual being a responder or non-responder, respectively:
The use of predictive models to deliver interventions can induce disparate impact if responding (respectively, non-responding) individuals of different groups receive the intervention at disproportionate rates under the treatment policy. This can occur even with efficient policies that threshold the true CATE $\tau$ and can arise from the disparate predictiveness of $X,A$ of response type (i.e., how far $p_{ij}$ are from $0$ and $1$). This is problematic because the choice of features $X$ is usually made by the intervening agent (e.g., government agency, etc.).
We discuss one possible interpretation of TPR or TNR disparities in this setting when the intervention is the bestowal of a benefit, like access to job training or case management. From the point of view of the intervening agent, there are specific program goals, such as employment of the target individual within 6 months. Therefore, false positives are costly due to program cost and false negatives are missed opportunities. But outcomes also affect the individual's utility. Discrepancies in TPR across values of $A$ are of concern since they suggest that the needs of those who could actually benefit from intervention (responders) in one group are not being met at the same rates as in other groups. Arguably, for benefit-bestowing interventions, TPR discrepancies are of greater concern. Nonetheless, from the point of view of the individual, the intervention may always grant some positive resource (e.g., from the point of view of well-being), regardless of responder status, since it corresponds to access to a good (and the individual can gain other benefits from job training that may not necessarily align with the intervener's program goals, such as employment in 1 year or personal enrichment). If so, then TNR discrepancies across values of $A$ imply a “disparate benefit of the doubt” such that the policy disparately over-benefits one group over another using the limited public resource without the cover of advancing the public program's goal, which may raise fairness and envy concerns, especially since this “waste” is at the cost of more slots for responders.
Beyond assessing disparities in TPR and TNR for one fixed policy, we will also use our ability to assess these over varying CATE thresholds in order to compute xAUC metrics kz-19 in (ref). These give the disparity between the probabilities that a non-responder from group $a$ is ranked above a responder from group $b$ and vice-versa. Thus, they measure the disproportionate access one group gets relative to another in any allocation of resources that is non-decreasing in CATE.
We emphasize that the identification arguments and bounds that we present on fairness metrics are primarily intended to facilitate the assessment of disparities, which may require further inquiry as to their morality and legality, not necessarily to promote statistical parity via adjustments such as group-specific thresholds, though that is also possible using our tools. We defer a more detailed discussion to (ref) and re-emphasize that assessing the distribution of outcome-conditional model errors are of central importance both in machine learning hardt2016equality,barocas-hardt-narayanan,mitchell-potash-barocas and in the economic efficiency of targeting resources pmt-2016,pmt-2016-brown,berger.
madras-creager-pz19 consider estimating joint treatment effects of race and treatment under a deep latent variable model to reconstruct unobserved confounding. For evaluating fairness of policies derived from estimated effects, they consider the gap in population accuracy $\mathrm{Acc}_a = \Prb{Z = Z^*\mid A=a}$, where $Z^*= \mathbb{I}[\tau(X) > 0]$ is the (identifiable) optimal policy. In contrast, we highlight the unfairness of even optimal policies and focus on outcome-conditional error rates (TPR, TNR), where the non-identifiability of responder status introduces challenges regarding identifiability.
The issue of model evaluation under the censoring problem of selective labels has been discussed in situations such as pretrial detention, where detention censors outcomes lkllm17,kz18-2. Sensitivity analysis is used in jsfg18 to account for possible unmeasured confounders. The distinction is that we focus on the targeted delivery of interventions with unknown (but estimated) causal effects, rather than considering classifications that induce one-sided censoring but have definitionally known effects.
Our emphasis is distinct from other work discussing fairness and causality that uses graphical causal models to decompose predictive models along causal pathways and assessing the normative validity of path-specific effects klrs17,hardtscholkopf-disc17, such as the effect of {probabilistic} hypothetical interventions on race variables or other potentially immutable protected attributes. When discussing treatments, we here consider interventions corresponding to allocation of concrete resources (e.g., give job training), which are in fact physically manipulable by an intervening agent. The correlation of the intervention's conditional average treatment effects by, say, race and its implications for downstream resource allocation are our primary concern.
There is extensive literature on partial identification in econometrics, e.g. manski2003partial. In contrast to previous work that analyzes partial identification of average treatment effects when data is confounded and using monotonicity to improve precision manski2003partial,balke1997bounds,beresteanu2012partial, we focus on unconfounded (e.g., RCT) data and achieve full identification by assuming monotonicity and consider sensitivity analysis bounds for nonlinear functionals of partially identified sets, namely, true positive and false positive rates.
Since the definitions of the disparate impact metrics in (ref) are conditioned on an unobserved event, such as the response event $Y(1) > Y(0)$, they actually cannot be identified from the data, even under ignorable treatment. That is, the values of $\op{TPR}_a,\op{TNR}_a$ can vary even when the joint distribution of $(X,A,T,Y)$ remains the same, meaning the data we see cannot possibly tell us about the specific value of $\op{TPR}_a,\op{TNR}_a$.
Essentially, (ref) follows because the data only identifies the marginals $p_{10}+p_{11},\,p_{01}+p_{11}$ while $\op{TPR}_a,\op{TNR}_a$ depend on the joint via $p_{01}$, which can vary even while marginals are fixed. Since this can vary independently across values of $A$, discrepancies are not identifiable either.
We next show identifiability if we impose the additional assumption of monotone treatment response.
(ref) says that anti-responders do not exist. In other words, the treatment either does nothing (e.g., an individual would have gotten a job or not gotten a job, regardless of receiving job training) or it benefits the individual (would get a job if and only if receive job training), but it never harms the individual. This assumption is reasonable for positive interventions. As k19-monotonicity points out, policy learning in this setting is equivalent to the binary classification problem of predicting responder status.
Since the quantities on the right hand sides in (ref) are in terms of identified quantities (functions of the distribution of $(X,A,T,Y)$), this proves identifiability. Given a sample and an estimate of $\tau$, it also provides a simple recipe for estimation by replacing each average or probability by a sample version, since both $A$ and $Z$ are discrete.
Thus, (ref) provides a novel means of assessing disparate impact of personalized interventions under monotone response. This is relevant because monotonicity is a defensible assumption in the case of many interventions that bestow an additional benefit, good, or resource, such as the ones mentioned in (ref). Nonetheless, the validity of (ref) is itself not identifiable. Therefore, should it fail even slightly, it is not immediately clear whether these disparity estimates can be relied upon. We therefore next study a sensitivity analysis by means of constructing partial identification bounds for $\op{TPR}_a,\op{TNR}_a$.
We next study the partial identification of disparate impact metrics when (ref) fails, i.e., $p_{10}\neq0$. We first state a more general version of (ref). For any $\eta=\eta(X,A)$, let
Since the anti-responder probability $p_{10}$ is unknown, we cannot use (ref) to identify $\op{TPR}_a, \op{TNR}_a$. We instead use (ref) to compute bounds on them by restricting $p_{10}$ to be in an uncertainty set. Formally, given an uncertainty set $\mathcal U$ for $p_{10}$ (i.e., a set of functions of $x,a$), we define the simultaneous identification region of the TPR and TNR for all groups $a\in\mathcal A$ as: $$ \Theta=\braces{\prns{\rho^{\op{TPR}}_a(\eta),\rho^{\op{TNR}}_a(\eta)}_{a\in\mathcal A}\;:\:\eta\in\mathcal U}\subseteq\R{2\times\abs{\mathcal A}}. $$ For brevity, we will let $\rho_a(\eta)=\prns{\rho^{\op{TPR}}_a(\eta),\rho^{\op{TNR}}_a(\eta)}$ and $\rho(\eta)=(\rho_a(\eta))_{a\in\mathcal A}$.
The set $\Theta$ describes all possible simultaneous values of the group-conditional true positive and true negative rates. As long as $\forall\eta\in\mathcal U$ we have $0\leq\eta(X,A)\leq \min\prns{\Prb{Y=1\mid T=0,X,A},\Prb{Y=0\mid T=1,X,A}}$ (which is identified from the data) by (ref) this set is necessarily sharp manski2003partial given only the restriction that $p_{10}\in\mathcal U$. (In particular, this bound on $\eta$ can be achieved by just point-wise clipping $\mathcal U$ with this identifiable bound as necessary.) That is, given a joint on $(X,A,T,Y)$, on the one hand, every $\rho\in\Theta$ is realized by some full joint distribution on $(X,A,T,Y(0),Y(1))$ with $p_{10}\in\mathcal U$, and on the other hand, every such joint gives rise to a $\rho\in\Theta$. In other words, $\Theta$ is an exact characterization of the in-fact possible simultaneous values of the group-conditional TPRs and TNRs.
Therefore, if, for example, we are interested in the minimal and maximal possible values for the true (unknown) TPR discrepancy between groups $a$ and $b$, we should seek to compute $ \inf_{\rho\in\Theta}\ \rho^{\op{TPR}}_a-\rho^{\op{TPR}}_b$ and $\sup_{\rho\in\Theta}\ \rho^{\op{TPR}}_a-\rho^{\op{TPR}}_b. $ More generally, for any $\mu\in\R{2\times\abs{\mathcal A}}$, we may wish to compute
Note that this, for example, covers the above example since for any $\mu$ we can also take $-\mu$. The function $h_{ \Theta }$ is known as the support function of ${ \Theta }$ rockafellar2015convex. Not only does the support function provide the maximal and minimal contrasts in a set, it also exactly characterizes its convex hull. That is, $\op{Conv}\prns{ \Theta }=\braces{\rho:\mu^\top\rho\leq h_{\Theta}(\mu)\ \forall\mu}$. So computing $h_{ \Theta }$ allows us to compute $\op{Conv}\prns{ \Theta }$.
Our next result gives an explicit program to compute the support function when $\mathcal U$ has a product form of within-group uncertainty sets:
which leads to $\Theta=\prod_{a\in\mathcal A}\Theta_a$ where $\Theta_a=\braces{\rho_a(\eta_a):\eta_a\in\mathcal U_a}$.
For a fixed value of $t_a$, the above program is a linear program, given that $\mathcal U_a$ is linearly representable. Therefore a solution may be found by grid search on the univariate $t_a$. Moreover, if $\mu_a^{\op{TPR}}=0$ or $\mu_a^{\op{TNR}}=0$, the above remains a linear program even with $t_a$ as a variable charnes1962programming. With this, we are able to express group-level disparities through assessing the support function at specific contrast vectors $\mu$.
We next consider the implications of the above for the following relaxation of the monotone treatment response assumption:
Note that (ref) with $B=0$ recovers (ref) and (ref) with $B=1$ is a vacuous assumption. In between these two extremes we can consider milder or stronger violations of monotone response and the partial identification bounds they corresponds to. This provides us with a means of sensitivity analysis of the disparities we measure, recognizing that monotone response may not hold exactly and that disparities may not be exactly identifiable. For the rest of the paper, we focus solely on partial identification under (ref). Note that (ref) corresponds exactly to the uncertainty set $\mathcal U_B=\braces{\eta:0\leq\eta(X,A)\leq \min\prns{B,\Prb{Y=1\mid T=0,X,A},\Prb{Y=0\mid T=1,X,A}}}$.\break We define $\Theta_B=\prod_{a\in\mathcal A}\Theta_{B,a}$ to be the corresponding identification region.
Under (ref), our bounds take on a particularly simple form. Let
and define
We discuss diagnostics to summarize possible impact disparities across a {range} of possible policies.
\paragraph{TPR and TNR disparity.} Discrepancies in model errors (TPR or TNR) are of interest when auditing classification performance on different groups with a given, fixed policy $Z$. Under (ref), they are identified by (ref). Under violations of (ref), we can consider their partial identification bounds. If the minimal disparity remains nonzero, that provides strong evidence of disparity. Similarly, if the maximal disparity is large, a responsible decision maker should be concerned about the possibility of a disparity.
Under (ref), (ref) provides that the sharp identification intervals of $\op{TPR}_a-\op{TPR}_b$ and $\op{TNR}_a-\op{TNR}_b$ are, respectively, given by
Given effect scores $\tau$, we can then use this to plot disparity curves by plotting the endpoints of (ref) for policies $Z=\mathbb{I}[\tau \geq \theta]$ for varying thresholds $\theta$.
\paragraph{Robust ROC Curves} We first define the analogous group-conditional ROC curve corresponding to a CATE function $\tau$. These are the parametric curves traced out by the pairs $(1-\op{TNR}_a,\op{TPR}_a)$ of policies that threshold the CATE for varying thresholds. To make explicit that we are now computing metrics for different policies, we use the notation ${\rho}(\eta; \tau \geq \theta)$ to refer to the metrics of the policy $Z=\indic{\tau\geq\theta}$. Under (ref), (ref) provides point identification of the group-conditional ROC curve:
When (ref) fails, we cannot point identify $\op{TPR}_a,\op{TNR}_a$ and correspondingly we cannot identify ${\op{ROC}}_a(\tau)$. We instead define the robust ROC curve as the union of all partially identified ROC curves. Specifically:
Plotted, this set provides a visual representation of the region that the true ROC curve can lie in. We next prove that under (ref), we can easily compute this set as the area between two curves.
This follows because the extremes are simultaneously achievable as noted in (ref). We highlight, however, that the lower (resp., upper) ROC curve may not be simultaneously realizable as an ROC curve of any single policy.
\paragraph{Robust xROC Curves} Comparison of group-conditional ROC curves may not necessarily show impact disparities as, even in standard classification settings ROC curves can overlap despite disparate impacts kz-19,hardt2016equality. At the same time, comparing disparities for fixed policies $Z$ with fixed thresholds may not accurately capture the impact of using $\tau$ for rankings. kz-19 develop the $\op{xAUC}$ metric for assessing the bipartite ranking quality of risk scores, as well as the analogous notion of a $\op{xROC}$ curve which parametrically plots the TPR of one group vs. the FPR of another group, at any fixed threshold. This is relevant if effect scores $\tau$ are used for downstream decisions by different facilities with different budget constraints or if the score is intended to be used by a “human-in-the-loop” exercising additional judgment, e.g., individual caseworkers as in the encouragement design of behncke-evaluation.
Under (ref), we can point identify $\op{TPR}_a,\op{TNR}_a$, so, following kz-19, we can define the point-identified xROC curve as
Without (ref), we analogously define the robust xROC curve as the union of all partially identified xROC curves:
This follows because $\mathcal U_B$ takes the form of a product set over $a\in\mathcal A$.
We consider a case study from a three-armed large randomized controlled trial that randomly assigned job-seekers in France to a control-group, a job training program managed by a public vendor, and an out-sourced program managed by a private vendor behaghel-2014. While the original experiment was interested in the design of contracts for program service delivery, we consider a task of heterogeneous causal effect estimation, motivated by interest in personalizing different types of counseling or active labor market programs that would be beneficial for the individual. Recent work in policy learning has also considered personalized job training assignment wager17,kt15 and suggested excluding sensitive attributes from the input to the decision rule for fairness considerations, but without consideration of fairness in the causal effect estimation itself and how significant impact disparities may still remain after excising sensitive attributes because of it.
We focus on the public program vs. control arm, which enrolled about 7950 participants in total, with $n_1 = 3385$ participants in the public program. The treatment arm, $T=1$, corresponds to assignment to the public program. The original analysis suggests a small but statistically significant positive treatment effect of the public program, with an ATE of $0.023$. We omit further details on the data processing to (ref). We consider the group indicators: nationality ($0,1$ denoting French nationals vs. non-French, respectively), gender (denoting woman vs. non-woman), and age (below the age of 26 vs. above). (Figures for gender appear in (ref).)
In (ref), we plot the identified “disparity curves” of (ref) corresponding to the maximal and minimal sensitivity bounds on TPR and TNR disparity between groups. Levels of shading correspond to different values of $B$, with color legend at right. We learn $\tau$ by the Generalized Random Forests method of wager2017estimation,athey2019generalized and use sample splitting, learning $\tau$ on half the data and using our methods to assess bounds on $\rho^{\op{TPR}}, \rho^{\op{TNR}}$ and other quantities with out-of-sample estimates on the other half of the data. We bootstrap over 50 sampled splits and average disparity curves to reduce sample uncertainty.
In general, the small probability of being a responder leads to increased sensitivity of TPR estimates (wide identification bands). The curves and sensitivity bounds suggest that with respect to nationality and gender, there is small or no disparity in true positive rates but the true negative rates for nationality, gender, and age may differ significantly across groups, such that non-women would have a higher chance of being bestowed job-training benefits when they are in fact not responders. However, TPR disparity by age appears to hold with as much as -0.1 difference, with older actually-responding individuals being less likely to be given job training than younger individuals. Overall, this suggests that differences in heterogeneous treatment effects across age categories could lead to significant adverse impact on older individuals.
This is similarly reflected in the robust ROC, xROC curves ((ref)). Despite possibly small differences in ROCs, the xROCs indicate strong disparities: the sensitivity analysis suggests that the likelihood of ranking a non-responding young individual above a responding old individual (xAUC kz-19) is clearly larger than the symmetric error, meaning that older individuals who benefit from the treatment may be disproportionately shut out of it as seats are instead given to non-responding younger individuals.
We presented identification results and bounds for assessing disparate model errors of causal-effect maximizing treatment policies, which can lead disparities in access to those who stand to benefit from treatment across groups. Whether this is “unfair” would naturally rely on one's normative assumptions. One such is “claims across outcomes,” that individuals have a claim to the public intervention if they stand to benefit, which can be understood within adler's axiomatic justification of fair distribution. There may also be other justice-based considerations, e.g. minimax fairness. We discuss this more extensively in (ref).
With the new ability to assess disparities using our results, a second natural question is whether these disparities warrant adjustment, which is easy to do given our tools combined with the approach of hardt2016equality. This question again is dependent both on one's viewpoint and ultimately on the problem context, and we discuss it further in (ref). Regardless of normative viewpoints, auditing allocative disparities that would arise from the implementation of a personalized rule must be a crucial step of a responsible and convincing program evaluation. We presented fundamental identification limits to such assessments but provided sensitivity analyses that can support reliable auditing.