Leave No One Undermined: Policy Targeting with Regret Aversion

Leave No One Undermined: Policy Targeting with Regret Aversion

\sloppy

\newgeometry{verbose,tmargin=1in,bmargin=1in,lmargin=1.25in,rmargin=1.25in,footskip=1cm}

Introduction

Personalization and policy targeting gained wide recognition in social and medical sciences. Yet, practice of complete personalization is rare. For example, as noted by manski2022patient, President's Council of Advisors on Science and Technology (PCAST) defines “personalized medicine” as:

“...the tailoring of medical treatment to the specific characteristics of each patient. In an operational sense, however, personalized medicine does not literally mean the creation of drugs or medical devices that are unique to a patient...”

Indeed, even though a rich set of observables $X$ are present in the data, policy makers (PM) often only form allocation rules based on a few covariates $W$, a subset of and not as rich as $X$. If treatment responses are significantly different in $X$ even after conditioning on $W$, how should the PM design its optimal treatment policy?

For instance, consider the mentoring program studied by resnjanskij2024can that aims to improve the labor market prospects for disadvantaged adolescents in Germany. resnjanskij2024can collected a rich set of covariate information on the participants of their randomized control trial. Their study highlights drastically different treatment responses depending on the social economic status (SES, classified based on answers to questions like how many books are at home, whether the adolescent is a first-generation migrant or has a single parent, etc.) of the adolescents: Low SES adolescents respond positively to the mentoring program while higher SES adolescents respond negatively (see left panel of Figure (ref)).

Motivated by their heterogeneity analyses, imagine a PM who decides what fraction of the population should be treated. However, suppose the PM cannot condition their decision on SES, because including it may be perceived as illegal or unfair, or measuring it precisely at the decision making stage is too costly. This corresponds to a scenario in which $X$ refers to SES and there is no $W$. Since the full population average treatment effect is positive, the common approach that aims to maximize the average welfare (e.g., manski2004statistical,kitagawa2018should,AW20,MT17) would inform the PM to treat all adolescents, even though higher SES adolescents lose from the program. Is the common approach still reasonable? And if not, what alternative decision criterion can reflect the heterogeneous treatment responses along SES?

In this paper, we answer these questions by studying the problem of policy targeting for a regret averse PM. The PM aims to find an optimal allocation rule $\delta$ that maps subset characteristics $W$ to $[0,1]$ indicating treatment fractions, with a loss function $L(\delta):=\mathbb{E}\left[Reg^{\alpha}(X,\delta)\right]$, where $Reg(x,\delta)$ refers to the welfare regret of group $X=x$ when applied with $\delta$, i.e., the efficiency loss compared to its best achievable welfare, $\alpha\geq1$ and $\mathbb{E}[\cdot]$ denotes expectation with respect to the marginal distribution of $X$. The case of $\alpha>1$ is what we call regret aversion, while $\alpha=1$ corresponds to the common approach, i.e., minimizing mean welfare regret (equivalent to maximizing mean welfare).

We link the regret-averse loss to the PM's aversion to unequal distributions of regret among groups defined by covariates that are not allowed to use as input to the treatment rule.\footnote{liang2021algorithm provide microfoundation for preference types that yield a preference for fairness, which in turn can be interpreted as inequality aversion. auerbach2024testing,liu2024inference conduct statistical inference based on the theoretical characterization of the fairness-accuracy frontier in liang2021algorithm.} For example, viewing the index of labor market prospects as a proxy for welfare, the right panel of Figure (ref) plots distributions of welfare regrets for three hypothetical rules (treat all, treat 88%, and treat 82%). Treating all benefits and induces no regret for low SES adolescents, but hurts and generates a high regret for higher SES adolescents.\footnote{As explained by resnjanskij2024can, mentoring may actaully crowd out more useful inputs offered by higher SES families, such as parental attachment or participation in other useful activities.} The other two fractional rules enjoy a more equal distribution of regret between the two groups, although they also have higher mean regrets. If $\alpha = 1$, PM displays no aversion to inequality of regrets and evaluates rules solely based on the mean, indicating treating all is indeed optimal. However, if $\alpha>1$, the policymaker dislikes and penalizes rules with higher inequality. We formally quantify such aversion via an Atkinson inequality index applied to regret (calculated in Table (ref) according to (ref)).\footnote{atkinson1970measurement's measure of inequality is at an individual level rather than group level. We may interpret $X$ in our setup as individuals in his framework.} The higher the value of $\alpha$, the more the PM dislikes inequality. In fact, treating 88% (82%) of the population is optimal if $\alpha=2$ (3). As one of the fundamental goals of sustainable development policy is to “reduce the inequalities (\ldots) that (\ldots) undermine the potential of individuals”\footnote{United Nations Sustainable Development Group, \url{https://unsdg.un.org/2030-agenda/universal-values/leave-no-one-behind}.}, and a wide literature in program evaluation \citep*[e.g.,][]{angrist2012benefits,angrist2022marginal,gray2023long} documents significant subgroup treatment heterogeneity along gender, race and other costly or sensitive attributes, our approach develops a new perspective in the current policy learning literature.

In general, both $W$ and $X$ can be continuous and multidimensional. We show the key insights persist: If $\alpha=1$, the optimal rule is to treat everyone or no one in the same $W$ group, depending on the sign of the corresponding CATE($W$), even if significantly heterogeneous treatment responses remains beyond $W$. Whenever $\alpha>1$, the optimal rule is fractional, if the treatment effect heterogeneity is severe enough to alter the sign of CATE($X$) within the same $W$.\footnote{We stress that the mechanism for fractional rules is different from that in kitagawa2022treatment, in which fractional rules arise due to sampling uncertainty. Fractional rules also show up with partially identified welfare with Manski2000,manski2005social,manski2007identification, Manski2007 or without stoye2012minimax,yata2021,manski2022identification,montielolea2023decision, kitagawa2023treatment true knowledge of the identified set, with nonlinear welfare manski2007admissible,manski20092009, when the decision maker targets a functional of the outcome distribution that is not quasi-convex \citep*{kock2022functional,kock2023treatment}, or when agents respond with strategic behavior munro2020learning.} This insight carries over analogously even with a capacity constraint.

The preceding example ignores the statistical uncertainty in the estimation of heterogeneous treatment effects. Focusing on $\alpha=2$ for tractability, we propose an empirical squared regret minimization approach with debiasing and cross fitting to properly account for the estimation uncertainty from training data. Our procedure accommodates a wide range of black-box machine learning methods for estimating the heterogeneous treatment effects. We develop new asymptotic upper and lower bounds for the excess risk of our proposal. In the case of a correctly specified linear sieve policy class, our procedure achieves a fast convergence rate of $O(1/n)$ and is in fact asymptotically efficient. In terms of computation, our squared regret approach is attractive due to the weighted least squares structure of the objective function. It will be straightforward to accommodate policy classes with convex constraints, compared to the mean regret approach (which often relies on maximum score type optimization).

We further illustrate the value of our approach using two real datasets. For the National JTPA Study that measured the benefit and cost of employment and training programs, we consider a PM who designs treatment policies based on pre-program years of education and earnings only, even though a wider range of characteristics like gender, race and marital status are available in the data. For the International Stroke Trial data that assessed the effect of aspirin treatment for patients with acute ischemic stroke, we considered a hypothetical scenario in which a doctor determines whether a patient should be treated with aspirin based on their age only, even though the training data contains many covariates of the patients, including their demographic and medical history. In both datasets, the estimated optimal policy fractions with our squared regret approach reveal considerable treatment effect heterogeneity for some population subgroups, which can lead to significant regret inequality should a singleton policy be applied instead. These exercises suggest that our squared regret approach reveals additional important information that may not be assessed with the mean regret paradigm alone.

The treatment choice literature has become an area of active research since the pioneering works of Manski2000, manski2002treatment,manski2004statistical and Dehejia2005. When the policy maker cannot differentiate individuals based on observable characteristics, finite and asymptotic results are developed by schlag2006eleven, stoye2009minimax, HiranoPorter2009, tetenov2012statistical, masten2023minimax, and chen2024note. manski2014quantile and guggenberger2024minimax in point-identified situations, and by Manski2000,manski2005social,manski2007identification, Manski2007, manski20092009, stoye2012minimax, christensen2020, ishihara2021, yata2021, manski2022identification, ishihara2023bandwidth and montielolea2023decision in partially-identified settings. When the policy maker is able to condition on individual characteristics, studies on personalization and policy targeting include manski2004statistical, BhattacharyaDupas2012, kitagawa2018should, KT21, MT17, AW20, sun2021empirical, adjaho2022externally, han2023optimal, cui2023individualized, terschuur2024locally, viviano2024policy and viviano2024fair, among others, for analyses in different settings.

Our squared regret criterion coincides with a quadratic surrogate criterion for a cost-sensitive classification problem to predict the sign of CATE($X$) with cost being squared CATE($X$). See Zhang_2004, Bartlett_et_al_2006, and references therein. Note, however, that our approach fundamentally differs from classification with the quadratic surrogate since our approach takes the squared regret as the ultimate objective function to minimize rather than a surrogate for the binary classification loss. As a result, our analysis can allow constrained $W$-individualized rules without raising the inconsistency issue of the constrained classification studied in KST21.

The rest of the paper is organized as follows: Section (ref) sets the stage, motivates our regret-averse loss function via inequality aversion and studies the population optimal allocation rule. Section (ref) presents our main proposal. Section (ref) develops the statistical performance guarantee. Section (ref) discusses the case with capacity constraint. Empirical applications are in Section (ref). Additional proofs, lemmas and technical results are reserved in the Appendix and Online Supplement.

Setup

Consider a policymaker who has access to a random sample of size $n$: $\ensuremath{Z^{n}:=\left\{ Z_i\right\} {}_{i=1}^{n}\in\mathcal{Z}^{n}}$, where $Z_{i}:=\{X_{i},D_{i},Y_{i}\}$, $X_{i}\in\mathcal{X}\subseteq\mathbb{R}^{d_X}$ is the observed pre-treatment characteristics (covariates) of unit $i$, e.g., their gender, race, pre-treatment education level, etc., $D_{i}\in\left\{ 0,1\right\} $ is the binary treatment indicator ($D_{i}=1$ means unit $i$ is under treatment and $D_{i}=0$ means under control), and $Y_{i}\in\mathbb{R}$ is the observed outcome of interest of unit $i$, generated as

in which $Y_{i}(1),Y_i(0)\in\mathcal{Y}\subseteq\mathbb{R}$ are the potential outcomes under treatment and control, respectively. Denote by $P\in\mathcal{P}$ the joint distribution of $\left\{ X_{i},D_{i},Y_{i}(1),Y_{i}(0)\right\} $. Then, the random sample $Z^{n}$ follows a joint distribution written as $P^{n}\in\mathcal{P}^n$, determined jointly by $P$, $n$ and (ref). In this paper, we maintain the following unconfoundeness and overlap assumptions:

The policymaker wishes to allocate a binary treatment $D\in\left\{ 0,1\right\} $ to a future population that shares the same marginal distribution of $\left\{ X_{i},Y_{i}(1),Y_{i}(0)\right\} $ induced by $P$. For each subpopulation group $X=x$ in which the covariate takes a specific value $x\in\mathcal{X}$, write

where $\mathbb{E}[\cdotp|\cdotp]$ denotes the conditional expectation under $P$. Write $\tau(x):=\gamma_{1}(x)-\gamma_{0}(x) $ as the conditional average treatment effect (CATE) for subgroup $X=x$. Under Assumption (ref), the CATE $\tau(x)$ is point identified for all $x\in\mathcal{X}$. We focus on the situation when --- at the decision-making stage --- the set of covariates that the policymaker can actually condition on, denoted by $W\in\mathcal{W}\subseteq\mathbb{R}^{d_W}$, is only a subset of and not as rich as $X$, i.e., we may partition $X=\{W,X_1\}$ for some $X_1\in \mathbb{R}^{d_{X_1}}$.

A regret averse planner's problem

We start from the planner's problem without sample data $Z^{n}$. Since the policymaker can only allocate policy decisions conditional on $W$, we call their action plan a $W$-individualized decision rule, i.e., a mapping $ \delta:\mathcal{W}\rightarrow[0,1]$ from the support of $W$ to the unit interval. Here, $\delta(w)$ is the fraction of the subpopulation $W=w$ to be treated. For example, $\delta(w)=0.5$ means half of the subpopulation with $W=w$ will be treated, leaving the rest untreated. Although the policy rule of the planner can only condition on $W$, treatment effect heterogeneity may still vary at the more refined level $X$. For each group with $X=x$, let its corresponding covariate $W$ take a value at $W=w$. Suppose that applying a generic $W$-individualized rule $\delta$ to $X=x$ yields a linearly additive welfare for the planner:

Note the form of the welfare in ((ref)) implies that the optimal level of the welfare for $X=x$ is achieved by the infeasible rule $\mathbf{1}\left\{ \tau(x)\geq0\right\} $. Then, for $X=x$, define the regret of rule $\delta$ as the welfare gap between $\delta$ and $\mathbf{1}\left\{ \tau(x)\geq0\right\}$:

We consider a regret-averse policy maker who chooses an optimal $W$-individualized policy rule by minimizing a nonlinear transformation of regret, a notion advocated by kitagawa2022treatment and axiomatized by hayashi2008regret,stoye2011axioms. Specifically, the policy maker aggregates regrets among different subpopulation groups via the average nonlinear regret loss:

where $\alpha\geq1$ is the degree of regret aversion, $F_{X}(\cdotp)$ denotes the marginal distribution of $X$ induced by the population distribution $P$, and $\mathbb{E}[\cdotp]$ is the expectation operator under $P$. A nonlinear regret optimal decision rule $\delta^{*}$ then solves

The rule $\delta^{*}$ characterizes an optimal action plan (conditional on $W$) for the planner if $P$ were known. Since $P$ in fact is unknown and needs to be learned from data $\ensuremath{Z^{n}\in\mathcal{Z}^{n}}$, the decision of the planner becomes statistical, i.e., selecting a $W$-individualized statistical decision rule: \[ \hat{\delta}:\mathcal{Z}^{n}\times\mathcal{W}\rightarrow[0,1] \] that instructs an action for each subgroup $W=w$ given each possible realization of data $Z^{n}=z^{n}\in\mathcal{Z}^{n}$. Denote by $\mathbb{E}_{P^{n}}[\cdotp]$ the expectation with respect to the randomness of $Z^{n}\sim P^{n}$. The planner's ultimate goal is to find a “good” rule $\hat{\delta}$ from the training data so that

is small and converges to zero at a fast rate (hopefully the fastest) uniformly across a set of possible distributions $P^{n}\in\mathcal{P}^{n}$.

Regret aversion as inequality aversion

We argue that our regret aversion loss $L(\delta,\tau)$ has baked in an aversion to regret inequality in the population.\footnote{Regret measures the welfare loss of a group compared to what could have achieved in terms of its best potential. Thus, our $L(\delta,\tau)$ reflects the preference of a planner who cares about to what extent personalized policies are equally fulfilling each sub-population's potential.} More concretely, write $Reg_{\delta}(x):=Reg(x,\delta,\tau)$. Inspired by the seminal work of atkinson1970measurement, let

be the Atkinson inequality measure of the regret distribution $Reg_{\delta}(\cdotp)$ in the population induced by rule $\delta$.\footnote{We may take $I_\alpha(Reg_\delta)=0$ if $\mathbb{E}[Reg_{\delta}(X)]=0$ as a convention.} Call $\left\{ \mathbb{E}[Reg_{\delta}(X)^{\alpha}]\right\} ^{1/\alpha}$ as the equally distributed equivalent level of regret --- the level of regret, if equally distributed to each subgroup in $X$, would yield the same level of the loss as the actual distribution $Reg_{\delta}(\cdotp)$. As $\alpha\geq1$, $I_{\alpha}(Reg_{\delta})\geq0$. The index $I_{\alpha}(Reg_{\delta})$ then measures how much larger the equally distributed equivalent is compared to the actual mean of regret $\mathbb{E}[Reg_{\delta}(X)]$. A larger value of $I_{\alpha}$ indicates a higher degree of regret inequality. In this context, the regret aversion coefficient $\alpha$ can be alternatively interpreted as a degree of inequality aversion of the policymaker. A larger value of $\alpha$ means the planner is more averse to regret inequality among the population. From ((ref)), we may rewrite our nonlinear regret loss as

The mean regret paradigm corresponds $\alpha=1$: $I_{1}(Reg_{\delta})=0$ for all distributions of regret, meaning the policy maker displays no aversion to regret inequality and ranks each distribution of regret only by their mean. If $\alpha>1$, the planner is averse to regret inequality and penalizes rules that lead to large inequality among the population.

Population analysis

We say a $W$-individualized rule $\delta$ is a singleton rule if for almost all $w\in\mathcal{W}$, it holds $\delta(w)\in\left\{ 0,1\right\} $; otherwise, we say $\delta$ is fractional. With a slight modification of notation, we write $\tau(w):=\mathbb{E}[Y(1)-Y(0)\mid W=w]$.

Proposition (ref) shows that a regret-averse planner concerned about regret inequality will often prefer a fractional $W$-individualized rule. They would prefer a singleton rule if, for almost all $w\in\mathcal{W}$, CATE($x$) shares the same sign for all $x$ with the same value of $w$, which also nests the case when $W=X$. Our results offer a novel justification of implementing fractional rules at the population level: Treatment effect heterogeneity at the $X$ level plus a concern for regret inequality induces a planner to “diversify” their treatment allocation. We illustrate the optimal rule of Example (ref) in Figure (ref).

When $\alpha=2$, $L(\delta,\tau)$ becomes the average squared regret, and the planner's problem becomes a weighted least squares problem: \[ \min_{\delta:\mathcal{W}\rightarrow[0,1]}\mathbb{E}\left\{ \tau^{2}(X)\left[\mathbf{1}\left\{ \tau(X)\geq0\right\} -\delta(W)\right]^{2}\right\} . \] Moreover, the associated optimal rule also has an explicit form

whenever $\mathbb{E}\left[\tau^{2}(X)\mid W=w\right]\neq0$.\footnote{If $\mathbb{E}\left[\tau^{2}(X)\mid W=w\right]=0$ for some $w\in\mathbb{R}^{d_W}$, then $\delta^*(w)\in[0,1]$, i.e., any action in $[0,1]$ is optimal for those $w$ values. Our theory accommodates this “non-uniqueness” situation to some extent. See Assumptions (ref) and (ref) and discussions therein. } (ref) shows that the fractional nature of the optimal rule $\delta^*$ depends not only on the sign and magnitude of the average treatment effect $\tau(x)$, but also the conditional distribution of $X$ given $W$. The optimal treatment assignment would be more fractional (i.e., closer to 0.5) if the values of $\int_{\tau(x)>0}\tau^{2}(x)dF_{X|W}(x\mid w)$ and $\int_{\tau(x)<0}\tau^{2}(x)dF_{X|W}(x\mid w)$ are closer to each other.

Main proposal

From now on and for the rest of the paper, we focus on $\alpha=2$ for tractability. Other values of $\alpha>1$ may be analyzed analogously with more technicalities. Our proposal for learning a good rule $\hat{\delta}$ from training data involves two steps. The first step is the efficient estimation of the loss function for each fixed $\delta$, i.e.,

Once $L(\delta,\tau)$ is efficiently estimated from data, the second step is to minimize the estimated loss among a class of $W-$individualized rules that must be specified by the researcher.

To allow for a wide range of ML algorithms in the first step and to potentially improve the statistical qualities of the second step, we consider “debiasing” (in a sense we make precise below) the loss function together with cross-fitting. \footnote{See Remark (ref) for discussions on the connections with more direct “plug-in” approaches that do not involve debiasing.} Note although the nuisance function $\tau$ appears inside the indicator function, the loss $L(\delta, \tau)$ is still continuously differentiable in $\tau$ (though not twice continuously differentiable). This is due to the squaring of the term $[\mathbf{1}\{\tau(X) \geq 0\} - \delta(W)]$, which smooths out the discontinuity (See Figure (ref)). Furthermore, we may view $L(\delta,\tau)$ as a finite dimensional parameter $\theta_0\in\mathbb{R}$ in a moment condition

Therefore, following newey1994asymptotic, we can still derive the efficient influence function for any regular and asymptotically linear estimator of $L(\delta,\tau)$. Let $\eta_{0}:=(\gamma_{1},\gamma_{0},\omega_{1},\omega_{0})$, where $\omega_{1}(x):=\frac{2\left(\gamma_{1}(x)-\gamma_{0}(x)\right)}{\pi(x)}$, $\omega_{0}(x):=\frac{2\left(\gamma_{1}(x)-\gamma_{0}(x)\right)}{1-\pi(x)}$. Write \[\xi(Z,\eta_{0}):=\left[\gamma_{1}(X)-\gamma_{0}(X)\right]^{2}+\left[D\omega_{1}(X)(Y-\gamma_{1}(X))-\left(1-D\right)\omega_{0}(X)(Y-\gamma_{0}(X))\right].\]

As $Var(\psi(Z))$ defines the semiparametric efficiency bound for estimating $L(\tau,\delta)$, we think it makes sense to exploit the structure of $\psi(Z)$ and define our modified loss function as:\footnote{Moreover, with the margin condition in Assumption (ref) and other regularity conditions, $L^o(\delta,\eta_0)$ can also be verified to satisfy the Neyman orthogonal condition (c.f., chernozhukov2018double and references therein).} \[ L^{o}(\delta,\eta_{0}):=\mathbb{E}\left[\xi(Z,\eta_{0})\left(\mathbf{1}\{\gamma_{1}(X)-\gamma_{0}(X)\geq0\}-\delta(W)\right)^{2}\right]. \]

Our theory does not restrict how the additional nuisance functions, $\omega_{1}$ and $\omega_{0}$, should be estimated. However, we note they feature the following balancing property (see, e.g., \citealt*{hainmueller2012entropy,zubizarreta2015stable,athey2018approximate}): for all $g(\cdotp)$ such that $\mathbb{E}[g^{2}(X)]<\infty$,

which may facilitate their estimation without the need to calculate propensity score. For example, to construct an estimator for $\omega_{1}$, denote by $b(x):=(b_{1}(x),\ldots,b_{\text{dim}(b)}(x))^{\prime}$ a vector of $\text{dim}(b)$ basis functions. Let $\left\Vert \cdot\right\Vert$ be the vector $l_{2}$ norm. Note (ref) implies

suggesting we may estimate $\omega_1$ by solving the following minimum distance estimator with a Tikhonov penalty (c.f., chen2012estimation,qiu2022approximate):

where $\hat{\gamma}_{1},\hat{\gamma}_{0}$ are estimated versions of $\gamma_{1}$ and $\gamma_{0}$, \[ \Theta_{n}=\left\{ f:\mathcal{X}\rightarrow\mathbb{R}\mid f(x)=a^{\prime}b(x),a\in\mathbb{R}^{\text{dim}(b)}\right\} , \] and $\lambda_{1,n}\geq0$ is a tuning parameter specified by the researcher.\footnote{In the empirical application, we use cross validation to select the tuning parameter $\lambda_{1,n}$. See Appendix (ref) for additional computational details.}

We now describe our cross-fitting procedure to estimate $L^{o}(\delta,\eta_0)$ from data $Z^{n}=\left\{ X_{i},D_{i},Y_{i}\right\} {}_{i=1}^{n}$. Let $[n]:=\{1,\dots,n\}$ be the observation index set. Randomly partition $[n]$ into approximately equal-sized $K\geq2$ folds $\left(I_{k}\right)_{k=1}^{K}$. Without loss of generality, we assume each fold is of sample size $m:=n/K$. For each $k\in[K]:=\{1,\dots,K\}$, let $I_{k}^{c}:=[n]\backslash I_{k}$ only include observations $\textit{not}$ from fold $I_{k}$. For each $I_{k}$, $k\in[K]$, we estimate $\eta_{0}$ by $\hat{\eta}^{k}:=(\hat{\gamma}_{1}^{k},\hat{\gamma}_{0}^{k},\hat{\omega}_{1}^{k},\hat{\omega}_{0}^{k})$, where $\hat{\gamma}_{1}^{k}:=\hat{\gamma}_{1}\left(\left(Z_{j}\right)_{j\in I_{k}^{c}}\right)$, $\hat{\gamma}_{0}^{k}:=\hat{\gamma}_{0}\left(\left(Z_{j}\right)_{j\in I_{k}^{c}}\right)$, $\hat{\omega}_{1}^{k}:=\hat{\omega}_{1}\left(\left(Z_{j}\right)_{j\in I_{k}^{c}}\right)$ and $\hat{\omega}_{0}^{k}:=\hat{\omega}_{0}\left(\left(Z_{j}\right)_{j\in I_{k}^{c}}\right)$, i.e., $\hat{\eta}^{k}$ is constructed only using data in $I_{k}^{c}$. Then, for each $\delta$, an estimator of $L^{o}(\delta,\eta_{0})$ is \[ \hat{L}_{n}^{o}(\delta):=\frac{1}{n}\sum_{i=1}^{n}\hat{\xi}(Z_{i})\left(\mathbf{1}\{\hat{\tau}(X_{i})\geq0\}-\delta(W_{i})\right)^{2}, \] where

are estimated versions of the weight $\xi(Z_{i},\eta_{0})$ and CATE $\tau(X_{i})$ for each $i\in[n]$. Next, let $p(w):=(p_{1}(w),p_{2}(w),\ldots p_{d_p}(w))^{\prime}$ be a vector of basis functions with dimension $d_p:=d_p(n)$ that may grow as $n\rightarrow\infty$. Write

Our final estimated policy is defined as:

where

and ${(\cdot)}^{-}$ denotes the Moore-Penrose inverse.\footnote{Our cross-fitted procedure is considered as DML2 chernozhukov2018double. One may also consider a different cross-fitting approach, in which we solve for the optimal rule in each fold before taking the averages over all folds. It would be interesting to compare these two approaches in policy learning problems, in light of the recent progress of velez2024asymptotic for estimation problems.}

The rationale behind (ref) is as follows. Given $\hat{L}_{n}^{o}(\delta)$, one may consider finding an optimal rule by solving

where

is a linear sieve policy class.\footnote{It is a common practice to use a class of linear functions to approximate a probability function. See, e.g., \citet*{chen2008semiparametric}. Our theory in fact can also be extended to other policy classes, e.g., a class of logit functions, with more technicalities.} (ref) may be viewed as a weighted least squares (empirical projection) problem, in which we predict an estimated outcome $\mathbf{1}\{\hat{\tau}(X_i)\geq0\}$ in space $\mathcal{D}$ with an estimated weight $\hat{\xi}(Z_i)$. Due to the presence of the adjustment term to “debias”, the weight $\hat{\xi}(Z_i)$ may be negative, and the Hessian matrix $\hat{A}_n$ may also not be positive semidefinite. As a result, the problem in (ref) is not necessarily convex in finite sample. However, our theory shows that, whenever $\hat{\eta}^{k}$ is of high quality and $n$ is sufficiently large (in a sense we make precise), the probability of $\hat{A}_{n}$ not being positive definite is exponentially small. In addition, on the event that $\hat{A}_{n}$ is positive definite, (ref) has a unique solution (ref), in which the Moore-Penrose inverse reduces to a standard inverse.\footnote{Therefore, (ref) also has an interesting IV interpretation with an outcome of interest $\mathbf{1}\{\hat{\tau}(X_i)\geq0\}$, a vector of endogenous variables $p(W_i)$, and a vector of instrument $\hat{\xi}(Z_i)p(W_i)$.} Finally, to guarantee the estimated policy is indeed a valid decision rule and also for technical tractability, (ref) takes a trimmed form.\footnote{See, e.g., newey1994series,newey1999nonparametric for examples, in other contexts, of using trimming to improve the theoretic performances of certain statistics.} Note (ref) is well-defined irrespective of whether $\hat{A}_n$ is positive definite or not.

Performance guarantee

Let $e_{1}:=Y(1)-\gamma_{1}(X)$, $e_{0}:=Y(0)-\gamma_{0}(X)$ and $A:=\mathbb{E}[\tau^{2}(X)p(W)p^{\prime}(W)]$. We first impose the following regularity conditions.

Under Assumptions (ref) and (ref), there exists some $C_{\xi}$ such that $\sup_{z\in\mathcal{Z}}\left|\xi(z,\eta_0)\right|\leq C_{\xi}$, and denote by $\bar{\lambda}<\infty$ and $\underline{\lambda}>0$ the maximum and minimum eigenvalues of $A$. Notably, even if $\mathbb{E}\left[\tau^{2}(X)\mid W=w\right]=0$ for some $w\in\mathbb{R}^{d_W}$, Assumption (ref)(ii) may still hold, thus allowing $\delta^*(w)$ to be non-unique for some $w$ values. Next, we impose the following statistical quality requirements on the learners of $\eta_0$. Let $ \mathbb{E}_{k}\left[\cdotp\right]:=\mathbb{E}_{P^{n}}\left[\cdotp\mid\left\{ Z_{j}\right\} _{j\in[n]\setminus I_{k}}\right]$.

By Assumption (ref)(i), our cross-fitted learners of $\eta_0$ are mean square consistent with certain convergence rates. Moreover, Assumptions (ref), (ref) and (ref)(ii) together imply that there exists some $\tilde{C}_{\xi}$ such that for all $k\in[K]$, $\sup_{z\in\mathcal{Z}}\left|\hat{\xi}^k(z)\right|\leq\tilde{C}_{\xi}$ conditional on $\left\{ Z_{j}\right\} _{j\in[n]\setminus I_{k}}$.

We now present a high-level stability condition of $\hat{\delta}$ useful for deriving fast convergence rates of our proposal.

Assumption (ref) essentially requires that the solution to the weighted least squares problem (ref) satisfies a stability property with respect to the sup norm. \footnote{See, for example, the sup-norm stability property of empirical $L_2$ projections using certain basis functions (e.g., splines and wavelets), which has been exploited by huang2003local,belloni2015some,chen2015optimal for sharp bias control in least squares series estimation. Our weighted least squares problem (ref), however, differs from those studied in the preceding literature due to the presence of estimated weights and outcomes.} With Assumptions (ref)-(ref), we verify in Appendix (ref) that Assumption (ref) holds if $\mathcal{D}$ is constructed with B-spline basis functions. Finally, we consider the following margin condition that concerns the distribution of $\tau(X)$ in the neighborhood of $\tau(X)=0$ (see also tsybakov2004optimal,kitagawa2018should):

Note Assumption (ref) rules out $P\{\tau(X)=0\}>0$, implying that $\delta^*(w)$ will be unique a.e. with respect to the marginal distribution of $W$. We are now ready to state our main result. Denote by $d_p^*$ the dimension of the basis functions for $\{f^2:f\in\mathcal{D}\}$, where $\mathcal{D}$ is defined in (ref), and write $\zeta_{p}:=\sup_{w\in\mathcal{W}}\left\Vert p(w)\right\Vert$.\footnote{The magnitudes of $d_p^*$ and $\zeta_p$ depend the choice of the basis functions. An upper bound of $d^*_p$ is $d_p^2$, but $d_p^*$ may be as small as $O(d_p)$, e.g., for B-splines. The quantity of $\zeta_p$ is a key object of interest in the series estimation literature. It is well known that $\zeta_p=O(\sqrt{d_p})$ for general spline basis functions (see, e.g., newey1997convergence). For B-splines, its structural properties imply that in fact, $\zeta_p\leq1$. See Appendix (ref) for additional treatments.}

Theorem (ref) provides an upper bound for the uniform excess risk whenever $n$ is sufficiently large. As long as $R_{n,O}$, $R_{n,B}$ and $R_{n,V}$ all go to zero at a polynomial rate as a function of $n$, the exponential term $R_{n,F}$ will be asymptotically negligible, implying immediately that if Assumptions (ref)-(ref) hold, \[ \sup_{P_{n}}\mathbb{E}_{P_{n}}\left[L(\hat{\delta}^{\mathcal{T}},\tau)-L(\delta^{*},\tau)\right]=O(R_{n,O}+R_{n,B}+R_{n,V}), \] and with the additional Assumption (ref), \[ \sup_{P_{n}}\mathbb{E}_{P_{n}}\left[L(\hat{\delta}^{\mathcal{T}},\tau)-L(\delta^{*},\tau)\right]=O\left(R_{n,O}+R_{n,B}+R_{n,V}\left(n^{-r_{\gamma_{1}}}+n^{-r_{\gamma_{0}}}\right)^{\frac{\alpha}{\alpha+2}}\right). \] Each part of (ref) and (ref) is interpretable. Term $R_{n,F}$ controls the excess risk even when $\hat{A}_n$ and its oracle version $A_n:=\frac{1}{n}\sum_{i=1}^{n}\xi(Z_{i},\eta_0)p(W_{i})p(W_{i})^{\prime}$ do not “behave nicely” (i.e., when they are not positive definite). When they do “behave nicely”, consider the following oracle “empirical risk minimization” (ERM) problem with known $\eta_0$:

where

The oracle excess risk is of $O\left(R_{n,O}\right)$, containing an approximation error term $\sup_{P^{n}}\inf_{\delta\in\mathcal{D}}\left[L(\delta,\tau)-L(\delta^{*},\tau)\right]$ due to using $\mathcal{D}$ to approximate $\delta^*$.\footnote{This approximation quality term may depend on whether Assumption (ref) is imposed or not, and may be further analyzed provided with suitable smoothness conditions on $\delta^*$, which we leave for future research. } Since $\eta_0$ is in fact unknown and needs to be estimated, we have to pay an additional price from the “remainder estimation error”. Interestingly, the asymptotic order of this remainder error depends on whether the margin condition is imposed or not. Without margin condition, the “remainder estimation error” is $O\left(R_{n,B}+R_{n,V}\right)$, where $R_{n,B}$ is a bias term while $R_{n,V}$ is a variance term. If the margin condition is imposed with some $\alpha>0$, the rate for the variance term improves to $O(R_{n,V}\left(n^{-r_{\gamma_{1}}}+n^{-r_{\gamma_{0}}}\right)^{\frac{\alpha}{\alpha+2}})$.

The optimality of our proposal depends on the complexity of $\delta^*$. If there exists some $\delta\in\mathcal{D}$ that solves (ref) with $d_p$ fixed, we say $\delta^{*}$ is parametric. If no rule in $\mathcal{D}$ solves (ref), we say $\delta^{*}$ is nonparametric. The following proposition suggests that, when $\delta^*$ is parametric, our procedure is asymptotically optimal in terms of the rate with Assumptions (ref)-(ref). Moreover, it is also asymptotically semiparametrically efficient with the additional Assumption (ref).

Note if $\delta^*$ is parametric, Assumption (ref)(ii) implies a unique $\beta^*\in\mathbb{R}^{d_p}$ such that $\left(\beta^{*}\right)^{\prime}p(w)$ solves (ref). The study of $\beta^*$ has a known semiparametric efficiency bound $\Omega$. See e.g., newey1994asymptotic,ackerberg2014asymptotic. By Proposition (ref)(ii), our procedure is asymptotically equivalent to the oracle that solves (ref). In particular, $\sqrt{n}\left(\hat{\beta}-\beta^{*}\right)\overset{d}{\rightarrow}N(0,\Omega)$, achieving the semiparametric efficiency bound asymptotically. Moreover, with a parametric $\delta^*$, \[n\left(L(\hat{\delta},\tau)-L(\delta^{*},\tau)\right) =n\left(\hat{\beta}-\beta^{*}\right)^{\prime}A\left(\hat{\beta}-\beta^{*}\right).\] The asymptotic efficiency of $\beta^*$ translates to that of $L(\hat{\delta},\tau)$, implying that our procedure is asymptotically efficient as well.\footnote{A parametric $\delta^*$ is not necessarily restrictive. Even if $\delta^*$ is nonparametric, one may wish to target the “second best” rule, i.e., $\delta^{SB}\in\arg\inf_{\delta\in\mathcal{D}}L(\delta,\tau)$, for which Proposition (ref) can be shown to still apply.}

When $\delta^*$ is nonparametric, the discussion of the optimality of our procedure is more involved. In Appendix (ref), we derive a minimax lower bound for $\delta^*$ in the style of stone1982optimal, which we suspect is attainable by our procedure for certain high smoothness class of $\delta^*$ when $d_p$ grows sufficiently slowly. We leave the verification of this conjecture, as well as the asymptotic distribution of $(L(\hat{\delta},\tau)-L(\delta^{*},\tau))$ for future research.

Capacity constraint

In this section, we consider a decision maker facing convex constraints for the allocation rules. As a leading case, suppose $W$ is discrete and a capacity constraint exists on how many people in the population can get treatment. With such capacity constraint, the problem is convex with differentiable objective and constraint functions, and the Slater's condition can be verified to hold. Therefore, the optimal solution is characterized by the well-known KKT condition (e.g., boyd2004convex, Chapter 5, p.244), as we show below.

Proposition (ref) highlights an interesting insight: with a capacity constraint, a regret-averse decision maker would reduce the fractional treatment for all groups, possibly with some groups with smallest $b_j$ not treated at all if the capacity constraint is too severe. In contrast, when $\alpha=1$, the decision maker always prioritizes treating the $W$ groups with the largest positive average treatment effect until the capacity constraint is filled, possibly with fractional allocation for the marginal group. In the hypothetical policy question from resnjanskij2024can considered in the introduction, suppose we have a capacity constraint that at most a $t\leq1$ fraction of the population can be offered with the mentoring program. Since there is only one $W$ group whose average treatment effect is positive, the optimal constrained rule is easy to calculate (see Table (ref)). For example, if $\alpha=1$, the optimal rule is to treat $t$ fraction of the population; if $\alpha=2$, the optimal rule is to treat $t$ fraction of the population if $t<0.88$ and to treat 0.88 of the population if $t\geq0.88$ (as 0.88 is the unconstrained optimal which does not violate the capacity constraint). In this simple case with one $W$ group, $\alpha=1$ and $2$ would share the same optimal rule if $t<0.88$.

In the setup of Proposition (ref), we can still learn the optimal constrained rule from data by solving (ref) and incorporating additional constraints:\footnote{We do not need to impose the constraints that $\delta(w_{j})\leq1$ for $j=1,...,J$, as they will be non-binding at the true population constrained optimal rule.}

which is still a convex program with differentiable objective and constraints and can be efficiently computed. However, establishing the statistical performance guarantee is more involved due to the known technical difficulty associated with not knowing whether the constraints in (ref) are binding or not in general.

Empirical applications

Job Training Partnership Act (JTPA) Study

We revisit the experimental dataset of the National JTPA Study that aimed to measure the benefit and cost of employment and training programs. Our sample consists of 9223 observations, in which the treatment $D$ was randomized to generate the applicants' eligibility for receiving a mix of training, job-search assistance, and other services provided by the JTPA. The outcome of interest $Y$ is the total individual earnings in the 30 months after program assignment.\footnote{We take the intention-to-treat perspective. One may also consider an net-of-cost outcome, which would further deduct 774 dollars for each of those assigned to treatment.} The study also collected a variety of the applicants' background information ($X$), some of which might be perceived as sensitive, e.g., gender, race and marital status. Following kitagawa2018should, we consider a scenario in which a policymaker can only design treatment policies based on pre-program years of education (“education”) and the pre-program earnings (“income”) --- these two variables become the $W$ in our setup. As an illustration of our debiased approach, we choose $K=5$ and estimate $\gamma_1$ and $\gamma_0$ via lasso with 10-fold cross-validation, with all interactions and squared terms of $X$. We estimate $\omega_1$ and $\omega_0$ with the minimum distance estimator with a Tikhonov penalty (ref), where the tuning parameter is selected via cross validation. See Section (ref) for computational details and our algorithm to calculate $\hat{\xi}(Z_i)$.

To start with, suppose the policymaker is interested in implementing a simple rule based on nine pre-determined income and education brackets (defined in the note of Figure (ref)). In this case, $W$ is discrete, and the optimal rule can be solved bracket-by-bracket. Figure (ref) reports the results for our squared regret debiased approach, a squared regret plug-in approach, as well as two linear regret approaches. Although the majority of the estimated CATEs conditional on $W$ are positive, the fractional nature of our estimated policies reveals plausible and considerable treatment effect heterogeneity at the $X$ level for some brackets, demonstrating the value of our approach compared to the standard mean regret paradigm. For example, for those units in education bracket 3 and income bracket 3, the debiased CATE estimate is slightly positive (41), implying all units shall be treated. However, an IPW estimate of the same CATE (-3361) would imply that no-one should be treated. For this group of workers, our squared regret debiased optimal policy is 0.63, indicating that workers in the high-education and high-income bracket may display drastically different treatment effects from each other, which can lead to a high regret-inequality should a non-fractional policy be applied. The pattern of the squared-regret policy estimates between the plug-in and debiased approaches are similar for many brackets, although some disparities do exist.

Next, we consider a policymaker designing a class of linear sieve policies based on education and income. As an illustration, for each of the education and income variables, we create cubic B-splines with a total of 5 degrees of freedom. The multivariate B-splines are then generated as tensor products of the two. We present estimated policies of the debiased and plug-in approaches for selected values of the income and education variables in Figure (ref). Both approaches again indicate considerable effect heterogeneity in the population, although disparities remain in the exact fitted values for some $W$ groups.

International Stroke Trial

As a second application and to demonstrate the value of our approach in medical studies, we analyze the International Stroke Trial (IST, international1997international) that assessed the effect of aspirin and other treatments for patients with presumed acute ischemic stroke. Following yadlowsky2025evaluating, we focus on the treatment of aspirin only on the outcome of whether there is death or dependency at 6 months. This leaves us with a sample of 18304 patients from over 30 countries. For each patient, we also observe a vector of 39 covariates ($X$), including their gender, age as well as some of their medical history and geographical information. In this exercise, we consider a hypothetical scenario in which a doctor determines whether a patient should be treated with aspirin only based on their age ($W$). The aim is to assess whether our approach would generate significantly different treatment fractions compared to the mean regret approach.

We estimate the nuisance parameters with the same methodology described in Section (ref). For the age variable, we create a cubic B-spline with a total degree freedom of 6. Figure (ref) reports our estimated optimal treatment fractions for patients with age between 39 and 92. As the CATEs are all positive for all considered age groups, a linear regret approach will recommend to treat everyone for all age groups. In sharp contrast, the estimated treatment fractions with our debiased approach is between 25% and 75% for most age values, revealing considerable treatment heterogeneity among those sharing the same ages. The treatment proportion is especially close to 0.5 for patients with age between 75 to 85, suggesting that a singleton “treat everyone” rule would potentially harm significantly some of those patients, leaving some of them with large regrets. The fitted curve with the plug-in approach shares the same downward sloping pattern as the debiased approach, although the estimated treatment proportions is slightly higher for all age groups. In light of these findings, we think that our squared regret approach to policy learning reveals additional important information that cannot be assessed with the mean regret approach alone.