Statistical Inference of Optimal Allocations I: Regularities and their Implications

The exact contents of citations.db main_text.text for this paper — one flattened LaTeX string, title through conclusion, appendix excluded, unmodified except for removing email addresses. This is what our citation measures are computed over.

260,151 characters

Statistical Inference of Optimal Allocations 1: Regularities and their Implications


\maketitle

\begin{abstract}
  \noindent {\sc Abstract.}
In this paper, we develop a functional differentiability approach for solving
statistical optimal allocation problems.
We derive Hadamard differentiability of the value functions through
analyzing the properties of the sorting operator using tools from
geometric measure theory.
Building on our Hadamard differentiability results, we apply
the functional delta method to obtain the asymptotic properties of the value function process for the binary constrained optimal allocation problem and the plug-in ROC curve estimator.
Moreover, the convexity of the optimal allocation value functions facilitates demonstrating the degeneracy of first order derivatives
with respect to the policy.
We then present a double / debiased estimator for the value functions.
Importantly, the conditions that validate Hadamard differentiability
justify the margin assumption from the statistical classification literature
for the fast convergence rate of
plug-in methods.


  \vspace{15pt}
  \noindent {\sc Keywords}:  Optimal treatment allocation, Functional differentiability and delta method, Plug-in classification, ROC curve, Double / debiased machine learning

  \vspace{15pt}
  \noindent {\sc JEL Codes}: C14, C38, C44
\end{abstract}

\begin{comment}

  \begingroup
  \renewcommand{}\footnote{This paper is developed based on the ideas from
an earlier work-in-progress project circulated under the title
``Decision Making with Machine Learning and ROC Curves''.
The work-in-progress project
latter has been developed into a working paper series, where
this manuscript
serves as the beginning of the sequence.
A large portion of the reported results of this paper was
drafted during the first author's visit to
Stanford University.
We acknowledge helpful and encouraging conservations with
collaborators and colleagues
including Timothy Armstrong, Xiaohong Chen, Hongjun Li, Jessie Li,
Thomas MaCurdy. We are most grateful to Yuhao Xue for enlightening us
with his knowledge of geometry measure theory.
Financial support was provided by the Tsinghua University Short-Term
Overseas Visit Funds.}
  \addtocounter{footnote}{-1}
  \endgroup

\end{comment}

\newpage

\section{Introduction}\label{introduction}

Characterizing individual heterogeneity is crucial
in the optimal allocation of scarce resources especially as a consequence of the rapid digitization of social-economic data.
Examples include
medical treatment assignments when doctors adopt
different therapeutic approaches to different patients,
pretrial bails when judges decide where defendants will await trials,
college admissions when committees assign students to various educational programs.
In order to address the heterogeneity inherent in such problems,
where the potential effects of a treatment on
an outcome vary among individuals,
personalized allocation is more of a norm than an exception.
Personalized allocation is compelling in the presence of the inherent individual heterogeneity
in the potential effects of a treatment on an outcome.

Based on past experimental or observational data,
the core objective of optimal allocation problem is to find a
strategy that can perform well out of sample.
An effective approach advocated by the literature is to make use of information provided by
prediction algorithms. See for example \cite{QJEbail}, \cite{agrawal2019artificial} and \cite{babina2024artificial}.
A common decision framework is to form
a nonrandom treatment rule using an indicator function that
benchmarks
a first step regression estimate of individual level treatment effect  against
certain thresholds, representing a plug-in strategy
in a weighted classification problem.

Such a plug-in strategy is based on solving a population
optimal allocation problem.
For a joint distribution of $\(X, Y\right)$ where $Y = \(Y_{j}, j=0,\ldots,J\right)$,
consider the \textit{social welfare potential function}\footnote{Appendix \ref{frechet differentiability section}  provides a mathematical justification
for the term ``potential function''.}
\begin{align}\begin{split}\label{optimal allocation general problem}
\gamma\left(\lambda, g\left(\cdot\right)\right) \coloneqq
\max_{\phi \in \Phi} \mathbb{E}\left[\sum_j \lambda_j \phi_j\(X\right) Y_j\right]
= \max_{\phi \in \Phi} \mathbb{E}\left[\sum_j \lambda_j \phi_j\(X\right) g_j\(X\right)\right].
\end{split}\end{align}
In the above, $\Phi$ is the set of multi-critical functions, i.e.
$\phi = \left(\phi_{j}, j=0,\ldots,J\right)$,
$\phi_j: \Omega\rightarrow \mathbb{R}$, with $\left(\Omega, F, \mu\right)$ being
a Borel probability space,  such that
$\phi_j \in L^\infty\left(\mu\right), 0 \leq \phi_j\leq 1, \forall
j\in \{0,\ldots,J\}$, and such that  $\sum_{j=0}^J
\phi_j\(x\right) \equiv 1$ for all $x \in \Omega$;
$g\left(\cdot\right) = \(g_j\left(\cdot\right), j=0,\ldots,J\right)$
and $g_j\(x\right) = \mathbb{E}\(Y_j \vert X = x\right)$.
Under very general conditions, the problem posed in
\eqref{optimal allocation general problem} admits an intuitive solution
where the $j$th-category receives all allocation
when $\mathbb{E}\left(\lambda_j Y_j \vert X\right)$ is the unique maximum.
If there exists some $l$ such that
$\mathbb{E}\left(\lambda_j Y_j \vert X\right) = \mathbb{E}\left(\lambda_l Y_l \vert X\right)$,
then
all allocation can be divided among the maximal indexes.

In this paper, we are interested in differentiating $\gamma\left(\lambda, g\right)$ in \eqref{optimal allocation general problem} functionally
with respect to both $\lambda$ and $g\left(\cdot\right)$.
The solution
can
provide inference methods for the value of optimal treatment allocation in multiple applications.
In section \ref{constrained and roc},
we derive the asymptotic distributions and justify bootstrapping for binary optimal allocation subject to a resource constraint and for the receiver operating characteristic (ROC) curve in binary classification.
Section \ref{fast convergence rates} provides asymptotic regret bounds and the asymptotic distribution of the value function for unconstrained multi-class optimal allocation.


Our paper attempts to synthesize several strands of literature.
It is widely-known
in the plug-in classification literature that the error of the second step classification is smaller than that of
the first step regression
\citep{devroye1996probabilistic, audibert2007fast}.
Our paper builds on seminal works by \cite{chernozhukov2018sorted} and \cite{chen2003estimation}
to provide conditions supporting $\sqrt{n}$ value function convergence rates and tight regret bounds in multiclass and constrained optimal allocations, covering
settings both for classical empirical processes and for recent advances in Neyman orthogonality and sample splitting \citep{chernozhukov2018double, chernozhukov2022locally}.
Subsequent works following \cite{manski2004statistical} have defined regret as the difference between the social
welfare potential and the expected utility achieved by an estimated decision rule.

\begin{comment}
Our paper is an attempt to synthesize the methodology and perspective of several strands of literature.
Hereafter, the core nature of issues,
as will be readily apparent, comes from the statistical treatment rule literature.
in the sense that the former usually converges faster than the latter
\citep{devroye1996probabilistic, audibert2007fast}.
Built on seminar works \cite{chen2003estimation} and \cite{chernozhukov2018sorted}
on method of moments allowing nonsmoothness,
especially the latter,
our paper can provide sets of conditions to support $\sqrt{n}$ and faster convergence rates of both the regret and other objective of interest.
Here, the regret is defined as the difference between the social
welfare potential and the expected utility achieved by an estimated decision rule, following \cite{manski2004statistical} and the subsequent works.
It will be shown that our differentiation results allow for
asymptotic analysis under settings for both classical empirical processes and recent advances in double / debiased machine learning \citep{chernozhukov2018double, chernozhukov2022locally}.
Our paper is an attempt to synthesize the methodology and perspective of several strands of literature.
Hereafter, the core nature of issues,
as will be readily apparent, comes from the statistical treatment rule literature.
A widely-known surprising phenomenon in the plug-in classification literature is that the second step classification is simpler than the first step regression, in the sense that the former usually converges faster than the latter
\citep{devroye1996probabilistic, audibert2007fast}.
Built on seminar works \cite{chen2003estimation} and \cite{chernozhukov2018sorted}
on method of moments allowing nonsmoothness,
especially the latter,
our paper can provide sets of conditions to support $\sqrt{n}$ and faster convergence rates of both the regret and other objective of interest.
Here, the regret is defined as the difference between the social
welfare potential and the expected utility achieved by an estimated decision rule, following \cite{manski2004statistical} and the subsequent works.
It will be shown that our differentiation results allow for
asymptotic analysis under settings for both classical empirical processes and recent advances in double / debiased machine learning \citep{chernozhukov2018double, chernozhukov2022locally}.
\end{comment}


In the binary case of $J=1$, $\lambda = \left(\lambda_0, \lambda_1\right)$.
Consider the
finite dimensional derivatives of $\gamma\left(\lambda, g\right) =\gamma\left(\lambda_0, \lambda_1, g\right)$ with respect to $\lambda_0$.
Assuming $\mathbb{P}\{\lambda_1 g_1\(X\right)=\lambda_0 g_0\(X\right)\}=0$,  we can write
\begin{align}\begin{split}\label{binary indicator sorting operator}
\gamma\left(\lambda_0,\lambda_1, g\right)
=\mathbb{E}\left[1\left(\lambda_0 g_0\(X\right) > \lambda_1 g_1\(X\right) \right)\lambda_0 g_0\(X\right)\right]
+\mathbb{E}\left[1\left(\lambda_1 g_1\(X\right) > \lambda_0 g_0\(X\right) \right)\lambda_1 g_1\(X\right)\right].
\end{split}\end{align}
Given the definition of $\gamma\left(\lambda_0,\lambda_1, g\right)$,  we expect based on intuitive reasoning that
\begin{align}\begin{split}\label{intuition for lambda finite dimensional}
&\frac{\partial \gamma\left(\lambda_0, \lambda_1, g\right)}{\partial \lambda_0}
= \mathbb{E}\left[1\left(\lambda_0 g_0\(X\right) > \lambda_1 g_1\(X\right) \)g_0\(X\right)\right].
\end{split}\end{align}
In a rigorous proof of \eqref{intuition for lambda finite dimensional}
using direct calculation,
the terms of major difficulties
are
\begin{align}\begin{split}\label{binary indicator sorting operator heuristic derivative}
\lim_{h \to 0} & \frac{
	\mathbb{E} \left[1\left(\left(\lambda_0+h\right) g_0\(X\right) > \lambda_1 g_1\(X\right)\right)
	- 1\left(\lambda_0 g_0\(X\right) > \lambda_1 g_1\(X\right)\right) \lambda_0 g_0\(X\right)\right]
	}{
h} \quad \text{and} \\
\lim_{h \to 0} & \frac{
	\mathbb{E} \left[1\left(\lambda_1 g_1\(X\right) > \left(\lambda_0 +h \right) g_0\(X\right)\right)
	- 1\left(\lambda_1 g_1\(X\right) > \lambda_0 g_0\(X\right)\right) \lambda_1 g_1\(X\right)\right]
	}{
h
	}.
\end{split}\end{align}
We will see later in section \ref{Weighted sorted effect} that under mild conditions,
the two terms in \eqref{binary indicator sorting operator heuristic derivative} cancel out because
they converge to the same integral on the level set
$\{x: \lambda_0 g_0\(x\right)=\lambda_1 g_1\(x\right)\}$ with opposing signs.
As a consequence, the desired partial derivatives exist and take the displayed form
\eqref{intuition for lambda finite dimensional}.

Our approach in section \ref{Weighted sorted effect} is greatly inspired by the \textit{integration on manifold} approach in \cite{chernozhukov2018sorted}.
In particular, we reapproach and generalize their results based on the concept of Hausdorff measure and powerful
area and coarea integration formulas from geometric measure theory.
We show that the Hadamard derivative of $\gamma\left(\lambda, g\right)$ contains terms that are integrals under certain Hausdorff measure.
The Hadamard differentiability results are derived under mild primitive conditions instead of being directly assumed.
See for example \cite{sherman1993limiting} and \cite{luedtke2016optimal}.

In the general multi-class setting, the social welfare potential function
in \eqref{optimal allocation general problem}
is not only subadditive but also positive homogeneous of degree one with respect to both $\lambda$ and $g\left(\cdot\right)$, respectively.
Consequently, while $\gamma\left(\lambda, g\right)$ is not necessarily jointly convex in $\left(\lambda, g\right)$,
$\gamma\left(\lambda, g\left(\cdot\right)\right)$ is convex in $g\left(\cdot\right)$ given $\lambda$ and is convex in $\lambda$ given $g\left(\cdot\right)$.
Interestingly, this simple observation seems to have gone
unnoticed by the existing literature.
As discussed in section \ref{frechet differentiability section},
profound results from geometric functional analysis imply that under nearly no assumptions, the social welfare potential
function $\gamma\left(\lambda, g\right)$ is Fr\'{e}chet differentiable on a \textit{large} set and admits an application of the envelope  theorem.

The implied envelope theorem allows us to verify the degeneracy of the first order Hadamard derivative of a multi-discrete social welfare potential function.
In section \ref{fast convergence rates},
we show that the primitive conditions in section \ref{Weighted sorted effect} can also be used to generalize fast convergence
rates
for the regret of binary plug-in classifiers \citep{audibert2007fast} to multi-discrete allocation problems.
The same methodology can be combined with recent developments of double/debiased machine learning \citep{chernozhukov2018double,chernozhukov2022locally} to speed
up the estimation of $\gamma\left(\lambda, g\right)$.


\section{Literature}\label{literature}

A multidisciplinary literature has studied
the construction of optimal decision rules from experimental and observational data in the forms of treatment allocation, policy learning and
individual treatment rule (ITR).
A large body of the literature focuses on empirical risk minimization (ERM).
See for example
\cite{manski2004statistical}, \cite{qian2011performance}, \cite{zhao2012estimating}, \cite{swaminathan2015batch}, \cite{zhou2017residual}, \cite{kitagawa2018should}, \cite{rai2018statistical}, \cite{luedtke2020performance}, \cite{athey2021policy}, \cite{mbakop2021model}, \cite{zhou2023offline}, \cite{ben2024policy} and \cite{ai2024data}, among others.
An important objective is to provide probabilistic bounds for
\begin{align}\begin{split}\nonumber
R\left(\hat{\phi}, \phi^{*}\right) & = \mathbb{E}_{X, Y}\left[\sum_{j}\phi^{*}_{j}\(X\)Y_{j}\right] -
\mathbb{E}_{X, Y}\left[\sum_{j}\hat{\phi}_{j}\(X\)Y_{j}\right], \\
\text{ for } \hat{\phi} & \in \mathop{\arg\max}\limits_{\phi \in \Phi_{0}}
 \sum_i \sum_{j} \phi_j\(X_i\right)	 F_j\(X_{i}, D_{i}, Y_{i}\right),
\end{split}\end{align}
where $\phi^{*}$ solves the population optimization program \eqref{optimal allocation general problem}.
The function $F\(X, D, Y\right)$ usually takes the inverse propensity weighting (IPW) or doubly robust (DR) form
and $\Phi_{0} \subset \Phi$ is not too complex in an entropy sense.
 See for example \cite{kitagawa2018should} and \cite{athey2021policy}.

A statistical difficulty in the optimal allocation problem arises
from the indicator function in the population solution \citep{qian2011performance}. To address
such a problem, we build on a divergent literature motivated by very different purposes.
To restore monotonicity in conditional quantile estimation,
the functional derivative of the sorting operator in the univariate case
is studied by \cite{chernozhukov2010quantile} and subsequently
extended to the multivariate cases
by \cite{chernozhukov2018sorted}
using calculus on manifold techniques. \cite{kim1990cube} hinted at a rudimentary form
of calculus on manifold in deriving the large sample properties of cube root consistent estimators.
\cite{sasaki2015quantile} incorporated Hausdorff measure tools in fluid mechanics to characterize the information content of quantile partial derivatives for general structural functions.

Optimal allocation problems appear in different but related settings.
Examples include asymptotically minimax optimal decision under the limits of experiments framework \citep{hirano2009asymptotics}, optimal decision under minimax regret \citep{stoye2009minimax, tetenov2012statistical, ben2024policy}, uniform confidence interval \citep{armstrong2023inference}, allocation with spillover effects \citep{kitagawa2023individualized, kitagawa2023should, viviano2025policy},
as well as binary constrained optimal allocation \citep{bhattacharya2012inferring, luedtke2016optimal} for which
our general functional Hadamard derivative results in section \ref{Weighted sorted effect} are applicable.

Closely related is the receiver operating characteristic (ROC) curve widely used in binary classification.
In the economics and finance literature, the plug-in ROC curves are used for example in \cite{berge2011evaluating}, \cite{vallee2019marketplace}, \cite{sadhwani2021deep}, \cite{feng2025statisticalMS} to evaluate prediction and decision making performance.
A previous literature studying the inference of the ROC curves, including \cite{hsieh1996nonparametric}, \cite{lloyd1998using}, \cite{li1999semiparametric}, \cite{hall2004nonparametric} and \cite{bertail2008bootstrapping},
did not account for the sampling error in the estimated propensity score.
An exception is \cite{luckett2021receiver} which
derives uniform asymptotics for the estimated ROC curve
using a support vector machine (SVM) classifier.
Unlike previous studies,
we derive the asymptotic properties of
the ROC curves obtained by plugging-in
the propensity score estimators,
which is consistent with the prevalent empirical usage of
predictive classifiers.




The methodology in this paper also has an intriguing connection with the faster convergence phenomenon in the classification literature \citep{devroye1996probabilistic, audibert2007fast}.
Consider, especially, the margin assumption (MA) evoked by works including \cite{mammen1999smooth}, \cite{tsybakov2004optimal}, \cite{boucheron2005theory}, \cite{audibert2007fast}, \cite{kitagawa2018should}, \cite{luedtke2020performance}
and \cite{semenova2023debiased}:
\begin{align}\begin{split}\label{marginal assumption MA}
\mathbb{P}\left\{\vert p\(X\right) - c\vert < t\right\} \leq Ct^{\alpha}
\end{split}\end{align}
for fixed $c$ and some constant $C > 0$ and $\alpha \geq 0$.
Our conditions in section \ref{Weighted sorted effect} directly imply the margin
assumption with $\alpha \geq 1$.
This connection can be further combined with the recent double/debiased machine learning approach \citep{chernozhukov2018double, chernozhukov2022locally} to form a transparent cross-validation and plug-in based method to complement
the optimal allocation literature.



\section{Hadamard differentiability}\label{Weighted sorted effect}

Following the seminal work by
\cite{chernozhukov2018sorted},	we consider \eqref{binary indicator sorting operator}
as a functional mapping
and are interested in the infinitesimal change in  $\gamma\left(\lambda, g\left(\cdot\right)\right)$ induced by a
marginal change in $\left(\lambda, g\left(\cdot\right)\right)$. More precisely, we derive the
Hadamard derivative of
$\gamma\left(\lambda, g\left(\cdot\right)\right)$ with respect to $\left(\lambda, g\left(\cdot\right)\right)$, which justifies and generalizes the calculations in \eqref{binary indicator sorting operator heuristic derivative}.
We assume that the underlying distribution of $X$,
denoted as $\mu$, is absolutely continuous with density
function $f\(x\right)$ with respect to $\mathcal{L}_{n}$, the $n$-dimensional Lebesgue measure. Then we write
\begin{align}\begin{split}\nonumber
\mathbb{E}\left[\sum_{j=0}^J \lambda_j \phi_j\(X\right) g_j\(X\right)\right]
= \sum_{j=0}^J \int \lambda_j \phi_j\(x\right) g_j\(x\right) d\mu\(x\right)=
\sum_{j=0}^J \int \lambda_j \phi_j\(x\right) g_j\(x\right) f\(x\right) d \mathcal{L}_{n} x.
\end{split}\end{align}
Typically, the optimizing policy function takes the form of
$\phi_j\(x\right) = \prod_{l \neq j} 1\left(\lambda_j g_j\(x\right) > \lambda_l g_l\(x\right) \right)$.
For simplicity we abbreviate $\lambda_j g_j\(x\right)$ as
$g_j\(x\right)$. The main object of study is
\begin{align}\begin{split}\nonumber
\int g_j\(x\right) f\(x\right) \prod_{l\neq j} 1\(g_j\(x\right) > g_l\(x\right)\right) d \mathcal{L}_{n} x,
\end{split}\end{align}
which can be
generalized to a form of the following sorting operator (as defined by \cite{chernozhukov2018sorted})
for vector-valued function $h\(x\right)$ and vector-valued cutoff threshold $c$,
\begin{align}\begin{split}\label{sorting operator in general form}
\int g_j\(x\right) f\(x\right) \prod_{l} 1\(h_l\(x\right) > c_{l}\right) d
\mathcal{L}_{n} x.
\end{split}\end{align}
The number of terms in the product should be
less than $n$, the dimension of $X$.


\subsection{Hausdorff measure and integration formulas}

The following discussions are based on standard references by
\cite{federer2014geometric} and \cite{evans2018measure}.
In order to extend the analysis in \cite{chernozhukov2018sorted},
this section first
defines extrinsically the
notion of length, area and volume.
For this purpose, the Carath\'{e}odory criterion is used to
construct a measure space from a $\sigma$-subadditive set function (outer measure) on power set of a space.
More specifically, we introduce the series of Hausdorff's
outer measures
(See definition
\ref{basic outer measure} for outer measure) which
does not require any local parameterization or regularity
assumption on these sets.




\begin{definition}{(Hausdorff outer measure)}
Let $d$ be a metric on $O$.
For arbitrary subset
$E \subset O$, define its diameter as
$\mathrm{diam}\ E = \sup_{x_{1}, x_{2} \in E}
d\(x_{1}, x_{2}\right)$ with
$\mathrm{diam}\ \emptyset = 0.$
Let
\begin{align}\begin{split}\nonumber
\alpha_{k} = \frac{\pi^{\frac{k}{2}}}
{\Gamma\left(\frac{k}{2} + 1\right)},
\quad k \in \mathbb{N}=\{0,1,\ldots\},
\end{split}\end{align}
where $\Gamma\left(\cdot\right)$ is the gamma function.	For arbitrary $E \subset O$,
and for $\delta > 0$,
define
\begin{align}\begin{split}\nonumber
\mathcal{H}^{*}_{k, \delta}\(E\right) = \inf\left\{
\sum_{j \geq 1}\alpha_{k}\left(
\frac{\mathrm{diam}\ B_{j}}{2}\right)^{k} :
E \subset \bigcup_{j \geq 1}B_{j},
\mathrm{diam}\ B_{j} \leq \delta
\right\}.
\end{split}\end{align}
The $k$-dimensional Hausdorff outer measure
of $E$ is defined as
$\mathcal{H}^{*}_{k}\(E\right) = \lim_{\delta \downarrow 0}
\mathcal{H}^{*}_{k, \delta}\(E\right)$, and can be verified to
be a metric outer measure on the metric space $\(O,d\right)$.
\end{definition}

\begin{definition}\label{def Hausdorff measure}
{(Hausdorff measure)}
For $k \in \mathbb{N}$, let $\mathcal{M}_{k} =
\mathcal{M}_{k}\(O\right)$ be the $\sigma$-algebra
generated by $\mathcal{H}^{*}_{k}$ by the
Carath\'{e}odory criterion (See Theorem
\ref{Caratheodory's extension theorem}).
The restriction of $\mathcal{H}^{*}_{k}$ to
$\mathcal{M}_{k}$,
$\mathcal{H}_{k} =
\mathcal{H}^{*}_{k}\vert_{\mathcal{M}_{k}}$, is
called Hausdorff measure on $\mathcal{M}_{k}$.
\end{definition}

In $\mathbb{R}^{n}$,
  $\mathcal{H}^{*}_{n} = \mathcal{L}^{*}_{n}$, $\mathcal{H}_{n} = \mathcal{L}_{n}$,
  where $\mathcal{L}^{*}_n$ denotes the
  Lebesgue outer measure and $\mathcal{L}_{n}$
  denotes the Lebesgue measure on
  $\mathbb{R}^{n}$.
$\mathcal{H}^{*}_{0}\(E\right)$ measures the cardinality of the set
$E$:
  \begin{align}\begin{split}\nonumber
  \mathcal{H}^{*}_{0}\(E\right) = \mathcal{H}_{0}\(E\right) = \left\{
  \begin{array}{ll}
    \text{number of elements in } E, &
    E \text{ is empty or finite,} \\
    + \infty, & E \text{ is infinite}.
  \end{array}\right.
  \end{split}\end{align}
We focus on the Hausdorff measures  $\mathcal{H}_{k}$
 defined on
Euclidean spaces $\mathbb{R}^{n}$ for $k \leq n$.

\begin{comment}
\begin{definition}\label{def Hausdorff measure}
{(Hausdorff Measure)}
Let $k \in \mathbb{N}$, $\mathcal{M}_{k} =
\mathcal{M}_{k}\(O\right)$ is the $\sigma$-algebra
generated by $\mathcal{H}^{*}_{k}$ in Theorem
\ref{Caratheodory's extension theorem}.
Then the restriction of $\mathcal{H}^{*}_{k}$ to
$\mathcal{M}_{k}$,
$\mathcal{H}_{k} = \mathcal{H}^{*}_{k}\vert_{\mathcal{M}_{k}}$ is
called Hausdorff measure on $\mathcal{M}_{k}$.
\end{definition}

\begin{remark}
\indent
\begin{enumerate}
  \item In $\mathbb{R}^{n}$,
  $\mathcal{H}^{*}_{n} = \mathcal{L}^{*}_{n}, \mathcal{H}_{n} = \mathcal{L}_{n}$,
  where $\mathcal{L}^{*}_n$ is the
  Lebesgue outer measure and $\mathcal{L}_{n}$
  is the Lebesgue measure on
  $\mathbb{R}^{n}$.
  \item $\mathcal{H}^{*}_{0}\(E\right)$ measures the cardinality of the set
$E$:
  \begin{align}\begin{split}\nonumber
  \mathcal{H}^{*}_{0}\(E\right) = \mathcal{H}_{0}\(E\right) = \left\{
  \begin{array}{ll}
    \text{number of element in } E, &
    E \text{ is empty or finite,} \\
    + \infty, & E \text{ is infinite}.
  \end{array}\right.
  \end{split}\end{align}
\end{enumerate}
\end{remark}

In this application,
we focus on Hausdorff measure defined on
Euclidean spaces $\mathbb{R}^{k}$.
\end{comment}

\begin{definition}\label{def J}
(Jacobian) For
$k \in \{1, 2, \ldots, n\}$, the Jacobian at $x$ is defined as
\begin{align}\begin{split}\nonumber
Jf\(x\right) = \left\{\begin{array}{ll}
		   \sqrt{\det\left(\nabla f\(x\right)^{T} \nabla f\(x\right)\right)}, &
		   f: E \subset \mathbb{R}^{k} \to
		   \mathbb{R}^{n}, \\
		   \sqrt{\det\left(\nabla f\(x\right)\nabla f\(x\right)^{T}\right)}, &
		   f: E \subset \mathbb{R}^{n} \to
		   \mathbb{R}^{k}.
		 \end{array}\right.
\end{split}\end{align}
\end{definition}


The following Theorem \ref{area formula classic}
generalizes the conventional
integration change of variable formula for
invertible
mapping to allow for non-invertible mapping and an increase in dimension.

\begin{theorem}\label{area formula classic}
Area formula\\
Let $E \subset \mathbb{R}^{k}$ be an open set,
$k \in \{1, 2, \ldots, n\}$,
$\psi: E \mapsto \mathbb{R}^{n}$ be a
$C^1$ or Lipschitz function.
Then for all measurable $S \subset E$,
\begin{align}\begin{split}\label{area formula area}
\int_{S}J\psi\(x\)d\mathcal{L}_{k}x =
\int_{\psi\(S\right)}\mathcal{H}_{0}\(S \cap
\psi^{-1}\(y\right)\)d\mathcal{H}_{k}y.
\end{split}\end{align}
If a measurable function $f: S \mapsto \mathbb{R}$ is
nonnegative or if the left hand side
of \eqref{area formula integral} is finite, then the following equality holds,
\begin{align}\begin{split}\label{area formula integral}
\int_{S}f\(x\)J\psi\(x\)d\mathcal{L}_{k}x =
\int_{\psi\(S\right)}\left[\int_{S \cap \psi^{-1}\(y\right)}
f\(x\)d\mathcal{H}_{0}x\]d\mathcal{H}_{k}y.
\end{split}\end{align}
\end{theorem}

Furthermore, the next \textit{curving Fubini-Tonelli}
theorem allows for slicing among curving surfaces and mapping from a higher
dimension to a lower dimension.


\begin{theorem}\label{coarea formula classic}
{Coarea formula}\\
 Let $E \subset \mathbb{R}^{n}$ be an open set,
$k \in \{1, 2, \ldots, n\}$,
$\varphi: E \mapsto \mathbb{R}^{k}$ be a $C^1$ or Lipschitz function,
then for all measurable $S \subset E$,
\begin{align}\begin{split}\label{coarea formula area}
\int_{S}J\varphi\(x\)d\mathcal{L}_{n}x =
\int_{\mathbb{R}^{k}}\mathcal{H}_{n - k}\(S \cap
\varphi^{-1}\(y\right)\)d\mathcal{H}_{k}y.
\end{split}\end{align}
If a measurable function $f: S \mapsto \mathbb{R}$ is
nonnegative or if the left hand side
of \eqref{coarea formula integral} is finite, then the following equality holds,
\begin{align}\begin{split}\label{coarea formula integral}
\int_{S}f\(x\)J\varphi\(x\)d\mathcal{L}_{n}x =
\int_{\mathbb{R}^{k}}\left[\int_{S \cap \varphi^{-1}\(y\right)}
f\(x\)d\mathcal{H}_{n - k}x\]d\mathcal{H}_{k}y.
\end{split}\end{align}
\end{theorem}

\begin{remark}
  \eqref{area formula integral} and
  \eqref{coarea formula integral}
  are direct corollaries of
  \eqref{area formula area} and
  \eqref{coarea formula area}, respectively,
  by a standard simple function
  approximation of measurable functions.
  \eqref{area formula integral},
  \eqref{coarea formula integral}
  and the classical $\mathbb{R}^{n} \mapsto
  \mathbb{R}^{n}$ change of variables
  formula,
  can be collectively written in a unified formula:
  \begin{align}\begin{split}\nonumber
  \int_{S}f\(x\)J\varphi\(x\)d\mathcal{L}_{n}x =
  \int_{\mathbb{R}^{k}}\left[\int_{S \cap
  \varphi^{-1}\(y\right)}f\(x\right)
  d\mathcal{H}_{\max\{n - k, 0\}}x\]d\mathcal{H}_{\min\{n, k\}}y,
  \end{split}\end{align}
  where $\varphi: \mathbb{R}^{n} \supset E \to \mathbb{R}^{k}$, $n, k \in \{1, 2, \ldots\}$.
\end{remark}


\subsection{Hadamard derivatives of the sorting operator}
\label{sorting operator}



In this section,
we present
our mathematics results on the
Hadamard differentiability of the sorting
operator in \eqref{sorting operator in general form}, where we intend to compute
\begin{align}\begin{split}\nonumber
\lim_{t\to 0}
\frac{1}{t}
\left[
\int \(g_j\(x\right) + t G_j\(x\right)\right) f\(x\right)
1\(h\(x\right) + t H\(x\right) > c\right) d \mathcal{L}_{n} x
-
\int g_j\(x\right) f\(x\right)
1\(h\(x\right) > c\right) d \mathcal{L}_{n} x
\right].
\end{split}\end{align}
Because the main term that needs to be accounted for is
\begin{align}\begin{split}\nonumber
\lim_{t\to 0}
\frac{1}{t}
\left[
\int g_j\(x\right) f\(x\right)
1\(h\(x\right) + t H\(x\right) > c\right) d \mathcal{L}_{n} x
-
\int g_j\(x\right) f\(x\right)
1\(h\(x\right) > c\right) d \mathcal{L}_{n} x
\right],
\end{split}\end{align}
we focus on the following simplified form of the sorting operator with
vector-valued $c$ and $h\(x\right)$:
\begin{align}\begin{split}\label{simplied form of sorting operator}
F\(h,c\right) \coloneqq \int_{h\(x\right) > c}f\(x\)d\mathcal{L}_{n}x =
\int 1
\(h\(x\right) > c\)f\(x\)d\mathcal{L}_{n}x.
\end{split}\end{align}
Without loss of generality, the
$g_j\(x\right) f\(x\right)$ term in \eqref{sorting operator in general form} is replaced by $f\(x\right)$ in
\eqref{simplied form of sorting operator} above.

First, in the special case of the sorting operator where
$h\(x\right)$ is a scalar function, consider the derivative with respect to $c$.
\begin{comment}
We begin with an equality.

\begin{lemma}\label{Morrey inequality}
({A Morrey type inequality})
Let $\Omega \subset \mathbb{R}^{n}$
be an open set,
$B\(x, r\right)$ be a closed ball in $\Omega$.
Then for all $y \in B\(x, r\right)$,
\begin{align}\begin{split}\nonumber
\vert u\(x\right) - u\(y\right)\vert \leq
Cr^{1 - \frac{n}{p}}
\left(\int_{B\(x, r\right)}
\vert u'\(z\right)\vert^{p}d\mathcal{L}_{n}z\right)
^{\frac{1}{p}}
\end{split}\end{align}
for all $u: \Omega \mapsto \mathbb{R}^{1},
u \in C^{1}$,
with $n < p < \infty$ and the constant $C$
depending only on $n$ and $p$.
\end{lemma}


\begin{remark}
This applicable form of the
classical Morrey's inequality is
inspired by Lemma 3 in
\cite{figalli2008simple}.
For a proof, see
\cite{evans2018measure}
Theorem 4.5.3.3.
\end{remark}
\end{comment}
Let $h$ be  $C^{1}$ or Lipschitz, with
$\Vert \nabla h\(x\right)\Vert > 0,
\mathcal{L}_{n} \; a.e. \; x.$
Suppose also that $f$ is an integrable function.
By the coarea formula Theorem \ref{coarea formula classic},
\begin{align}\begin{split}\nonumber
\int_{h\(x\right) > c}
f\(x\)d\mathcal{L}_{n}x =
\int^{\infty}_{c}\left[
\int_{h^{-1}\(c'\right)}\frac{f\(x\right)}
{\left\Vert\nabla h\(x\right)\right\Vert}
d\mathcal{H}_{n - 1}x\]d\mathcal{L}_{1} c'.
\end{split}\end{align}
Further,
by the Lebesgue differentiation theorem, the conventional derivative is given by
\begin{align}\begin{split}\nonumber
\frac{d}{dc}\left(\int_{h\(x\right) > c}
f\(x\)d\mathcal{L}_{n}x\right) =
- \int_{h^{-1}\(c\right)}\frac{f\(x\right)}
{\left\Vert\nabla h\(x\right)\right\Vert}d\mathcal{H}_{n - 1}x
\end{split}\end{align}
for $\mathcal{L}_{1} \; a.e. \; c$.
The right hand side above is also a derivative in the distributional (weak) sense.
\begin{definition}\label{first Hadamard}
(Hadamard differentiability)
Let $\mathcal X$ and $\mathcal Y$ be normed spaces
with norms
$\Vert\cdot\Vert_{\mathcal X}$ and $\Vert\cdot\Vert_{\mathcal Y}$.
Consider the map $\phi: \mathcal X_{\phi} \subset \mathcal X \to \mathcal Y$.
Then $\phi$ is called Hadamard differentiable
  at $\theta \in \mathcal X_{\phi}$ tangentially to
  $\mathcal X_{0} \subset \mathcal X$
  if there is a continuous linear map
  $\phi'_{\theta}: \mathcal X_{0} \to \mathcal Y$ such that
  \begin{align}\begin{split}\nonumber
  \left\Vert
  \frac{\phi\left(\theta+ t_{n} \mathcal \mathcal{x}_{n}\right) -
  \phi\left(\theta\right)
  }{t_{n}} - \phi'_{\theta}\(x\right)
  \right\Vert_{\mathcal Y} \to 0, \; \text{for all} \;
  t_{n} \to 0 \; \text{and} \; \mathcal \mathcal{x}_{n} \to x \in \mathcal{X}_{0},
  \end{split}\end{align}
  as $n \to \infty$,
  where
  $\theta + t_{n} \mathcal \mathcal{x}_{n} \in \mathcal X_{\phi}$ for all $n$.
\end{definition}

\begin{remark}
When $\mathcal X_0$ is a linear subspace,
by  \cite{fang2019inference} Proposition 2.1,
  $t_{n} \to 0$ can be replaced by $t_{n} \downarrow 0$
  in Definition \ref{first Hadamard} without
loss of generality.
\end{remark}

To verify
Hadamard differentiability,
we calculate the derivative
of the sorting operator,
\begin{align}\begin{split}\nonumber
\lim_{n \to \infty}
\frac{1}{t_{n}}\int
\left[1\(h\(x\right) +
t_{n} \mathcal \mathcal{H}_{n}\(x\right) > c\right) -
1\(h\(x\right) > c\right)\right]
f\(x\)d\mathcal{L}_{n}x,
\end{split}\end{align}
where $\mathcal \mathcal{H}_{n} \to H$ and $t_{n} \downarrow 0$
as $n \to \infty$.
This limit is closely related to
the
Hausdorff measure $\mathcal{H}_{k}$ along the {\it surface}
$\{x: h\(x\right) = c\}$, which
defines the
{\it volume} of $k$-dimensional surfaces
in $n$-dimensional spaces, $k \in  \{1, \ldots, n\}$.
The important insight of
\cite{chernozhukov2018sorted}
is a representation by an integral under
$\mathcal{H}_{n - 1}$.

Let $E \subset \mathbb{R}^{n}$ be an open set,
$k \in \{1, 2, \ldots, n\}$,
$h: E \mapsto \mathbb{R}^{k}$ be a $C^{1}$
function, $f: E \mapsto \mathbb{R}$
be a continuous function.
Let $C\(E, \mathbb{R}^{k}\right)$ denote the space
of $\mathbb{R}^{k}$-valued continuous functions on $E$ equipped with
the sup-norm.
We need the following technical
assumptions about $h$ and $f$.

\begin{assump}\label{compact assumption}
The function $f$ is continuous with compact support $K_{f} \subset E$.
\end{assump}

\begin{definition}\label{critical regular}
(Critical and regular point and value)
Let $n, k \in \{1, 2, \ldots\}$,
  $E \subset \mathbb{R}^{n}$ be an
  open set, $f: E \mapsto \mathbb{R}^{k}$
  be a $C^{1}$ function.
  A point $x \in E$ is
  a critical point if
  $J f\(x\right) = 0$;
  otherwise $x$ is a regular point.
  If
  $c = f\(x\right)$
  for some critical point $x$, then
  $c$ is a critical value;
  otherwise $c$
  is a regular value.
  For tractability, we only consider regular values $c \in f\(E\right)$.
\end{definition}

\begin{assump}\label{regular value assumption}
The support $K_f$ consists of only regular points of the function $h$.
\end{assump}


\begin{comment}
\begin{remark}\label{R0 exist conditions}
One may ask
when will $\mathcal{R}$
be ``big'' enough to contain a open subset?
By a $C^{1}$ diffeomorphism $\psi$
constructed as in the proof of Corollary
\ref{another Sard lemma}, $\Delta\(E\right)$
contains a open set $U_{0}$ if $\Delta$ has
at least one regular point on $E$.
If one assumes that $\Delta$ is a
$C^{n - k + 1}$ function, then by
Lemma \ref{Morse Sard theorem} (the
Morse Sard theorem),
the set of critical values $\mathcal{T}$
will be of $0$ measure.
In this case,
$\mathcal{R}$ will have positive measure.
If one further assumes that
$\Delta$ has compact supports
for all component functions and the union
of the supports is contained in $E$,
then $\Delta\(E\right)$ will be compact.
Consequently, $\mathcal{T}$ will also be compact,
and thus
$\mathring{\mathcal{R}} \supset U_{0}
\backslash \mathcal{T}$
will be of positive measure.
\end{remark}
\end{comment}


Assumption
\ref{compact assumption}
guarantees that $K_f \cap \{x: h\(x\right) = c'\}$
is close to
$K_f \cap \{x: h\(x\right) = c\}$ when $c'$
is close to $c$.
The purpose of the
Assumption \ref{regular value assumption} is to
guarantee that $h$ is regular in a neighborhood of $K_f$ in order to apply the implicit function theorem.

\begin{comment}
\begin{manualproposition}{\ref{compact assumption}$'$}\label{compact assumption '}
For all $c \in h\(E\right)$, there exists a neighborhood $\mathcal N_c$ of $c$ such that
\begin{align}\begin{split}\nonumber
\overline{
\bigcup_{c' \in \mathcal N_c}
h^{-1}\(c'\right)} \subset E
\end{split}\end{align}
and is bounded. The function $f$ is continuous.
\end{manualproposition}
\end{comment}

\begin{theorem}\label{general k derivative}
Let $h: E \to \mathbb{R}^k$ be a $C^1$ function, $k\in \{1,2,\ldots,n\}$, $E$ an open subset of $\mathbb{R}^n$.
Let Assumption
 \ref{compact assumption}
and Assumption \ref{regular value assumption} hold.
Let $\mathcal D \subset h\(E\right)$ be a compact subset.
Then,
the map $F\(h,	c\right):
C\(E, \mathbb{R}^{k}\right) \to \mathbb{R}$
in \eqref{simplied form of sorting operator}
is Hadamard differentiable uniformly with respect to $c \in \mathcal{D}$ at $h$ tangentially
to $C\(E, \mathbb{R}^{k}\right)$.
When $k > 1$, the derivative is given by
\begin{align}\begin{split}\label{derivative k>1}
F'_{h, c}\(H, 0\right) =  \sum_i \int_{y > \tau_{\neg i}\(c\right)} \left[
\int_{h^{-1}\(c'\(y, c_{i}; i\right)\right)}
\frac{
H_i\(x\right) f\(x\right)
}{
J h\(x\right)
}
d \mathcal{H}_{n-k} x
\right] d \mathcal{L}_{k-1} y,
\end{split}\end{align}
where $\tau_{\neg i}\(c\right)$ denotes the $\mathbb{R}^k \to \mathbb{R}^{k-1}$ coordinate projection except
the $i$-th coordinate, $c'\(y, c_{i}; i\right)$ is the $k$-dimensional vector such that $\tau_{\neg i}\(c'\(y, c_{i}; i\right)\right) = y$ and $\tau_{i}\(c'\(y, c_{i}; i\right)\right) = c_i$.
When
$k=1$,	i.e. when
$h$ is a scalar-valued function,
the derivative
is given by
\begin{align}\begin{split}\label{theorem 2.1 derivative k=1}
F'_{h, c}\(H, 0\right) =
\int_{h^{-1}\(c\right)}
\frac{H\(x\)f\(x\right)}{\left\Vert
\nabla h\(x\right)\right\Vert}
d\mathcal{H}_{n - 1}x.
\end{split}\end{align}
\end{theorem}

\begin{comment}
\begin{theorem}\label{derivative k=1}
Consider when $k = 1$, i.e. when
$h$ is a scalar-valued function.
Let Assumption
\ref{compact assumption} (or \ref{compact assumption '})
and Assumption
\ref{regular value assumption '} hold.
Then, there exists a bounded closed interval
$\mathcal D \subset h\(E\right)$, $c \in \mathcal{D}$,
such that
the map $F\(h,	c\right):
C\(E\right) \to \mathbb{R}$ is Hadamard differentiable uniformly
on $\mathcal{D}$ at $h$ tangentially
to $C\(E\right)$ ($C_{b}\(E\right)$ under Assumption \ref{compact assumption '}).
The derivative
is given by
\begin{align}\begin{split}\label{theorem 2.1 derivative k=1}
F'_{h, c}\(H, 0\right) =
\int_{h^{-1}\(c\right)}
\frac{H\(x\)f\(x\right)}{\left\Vert
\nabla h\(x\right)\right\Vert}
d\mathcal{H}_{n - 1}x.
\end{split}\end{align}
\end{theorem}
\end{comment}

\begin{comment}
The empirical ROC curves require estimating both
$\Delta\left(\cdot\right)$ and the threshold $c$.
This prompts the derivative
with respect to both $\Delta$ and $c$.

\begin{cor}\label{derivative k=1 D and c}
Under the conditions in Theorem
\ref{derivative k=1}, the
map $F_{\Delta, f, c}:
C\(E\right) \times \mathbb{R} \mapsto \mathbb{R}$
is Hadamard differentiable uniformly
on $\mathcal{D}$ at $\left(\Delta, c\right)$
tangentially to $C_{0}\(E\right) \times \mathbb{R}$.
The derivative in the direction of $\(G\left(\cdot\right), K\right)$ is given by
\begin{align}\begin{split}\nonumber
F'_{\Delta, f, c}\(G, K\right) =
\int_{\Delta^{-1}\(c\right)}
\frac{\(G\(x\right) - K\)f\(x\right)}{\left\Vert
\partial\Delta\(x\right)\right\Vert}
d\mathcal{H}_{n - 1}x.
\end{split}\end{align}
\end{cor}
\end{comment}

Embedded in the proof of Theorem \ref{general k derivative}
is the continuity of the derivative of the sorting operator,
expressed in the following Proposition \ref{surface integral continuity}, which
handles the general case when $k \geq 1$.


\begin{proposition}\label{surface integral continuity}
Under the conditions in Theorem \ref{general k derivative},
the density of the random variable $h\(X\right)$
is continuous on $h\(E\right)$ and is given by
\begin{align}\begin{split}\nonumber
\int_{h^{-1}\(c\right)}\frac{f\(x\right)}
{J h\(x\right)}d\mathcal{H}_{n - k}x.
\end{split}\end{align}
\end{proposition}

We next consider a variant of Theorem \ref{general k derivative} under Lipschitz conditions.

\begin{assump}\label{borel compact assumption}
The function $f$ is Borel and bounded with compact support $K_{f} \subset E$.
\end{assump}

\begin{assump}\label{Clarke regular point assumption}
The Clarke Jacobian $J_{c}h\(x\right)$ (defined in \ref{Clarke Jacobian} in the Technical addendum) is of full rank for all $x \in E$.
For all $x$, there exists a neighborhood $B_{x}$ and a constant $\epsilon\(x\right) > 0$, such that for all $x' \in B_{x}$, if $\nabla h\(x'\right)$ exists then $Jh\(x'\right) > \epsilon\(x\right)$.
\end{assump}

\begin{assump}\label{a.e. continuous at boundary assumption}
For all $c \in h\(E\right)$, $\mathcal{H}_{n - k}\left\{x: h\(x\right) = c, h\left(\cdot\right) \text{ is not differentiable at } x\right\} = 0$ and $\mathcal{H}_{n - k}\left\{x: h\(x\right) = c, \exists x_{\(k\right)}\left(\left\vert\det\left(\frac{\partial}{\partial x_{\(k\right)}}h\left(\cdot\right)\right)\right\vert = 0\ \text{or}\ \frac{f\left(\cdot\right)}{\left\vert\det\left(\frac{\partial}{\partial x_{\(k\right)}}h\left(\cdot\right)\right)\right\vert} \text{ is discontinuous at } x\right)\right\} = 0$,
where the subscript $\(k\right)$ denotes $k$-combination of the $n$ coordinates.
\end{assump}





The next theorem \ref{nonsmooth general k derivative}  is a nonsmooth version of Theorem \ref{general k derivative}.
\begin{theorem}\label{nonsmooth general k derivative}
Let $h: E \to \mathbb{R}^{k}$ be Lipschitz, $k \in \{1, 2, \ldots, n\}$, $E$ an open subset of $\mathbb{R}^{n}$.
Let Assumption \ref{borel compact assumption} to
Assumption \ref{a.e. continuous at boundary assumption} hold. Let $\mathcal{D} \subset h\(E\right)$ be a compact subset.
Then the map $F\(h, c\right): C\(E, \mathbb{R}^{k}\right) \to \mathbb{R}$ is Hadamard differentiable uniformly on $c \in \mathcal{D}$ at $h$ tangentially to $C\(E, \mathbb{R}^{k}\right)$.
The derivatives are given by \eqref{derivative k>1} and \eqref{theorem 2.1 derivative k=1} respectively
when $k > 1$ and when $k = 1$.
\end{theorem}

The assumptions employed in Theorem \ref{nonsmooth general k derivative} play several roles.
Assumption \ref{borel compact assumption} provides compactness and
Assumption \ref{Clarke regular point assumption} guarantees $h$ is regular.
The compactness Assumption
\ref{borel compact assumption} ensures that the set $\{x: h\(x\right)=c\} \cap K_f$
in the area of integration is not too large and
changes continuously with $c$, meaning that
each open covering of $\{x: h\(x\right)=c\} \cap K_f$ covers $\{x: h\(x\right)=c'\} \cap K_f$ for all $c'$ close enough to $c$.
Assumption \ref{Clarke regular point assumption} requires each $x \in E$ to be a regular point of $h\left(\cdot\right)$.
Under this assumption,
$h\left(\cdot\right)$ can be expressed by the implicit function theorem to facilitate
the change of variables.

In addition, Assumption \ref{a.e. continuous at boundary assumption} requires each
level set $\{x: h\(x\right)=c\}$ to be sufficiently regular in the sense of
having
a small number of nonsmooth points and the integrand  to be
almost everywhere continuous. Under this assumption,
derivatives can be taken almost everywhere on the level set.
Assumption \ref{a.e. continuous at boundary assumption} can be replaced by continuous differentiability of $h$ and continuity of $f$.

Alternative assumptions are possible.
Consider Theorem \ref{general k derivative}.
Therein, Assumption \ref{compact assumption}, which requires compactness of the support of $f\left(\cdot\right)$, can be replaced by
the next Assumption \ref{compact assumption '} which requires that local union of level sets of $h\left(\cdot\right)$  is compact.
\begin{manualproposition}{\ref{compact assumption}$'$}\label{compact assumption '}
For all $c \in h\(E\right)$, there exists a neighborhood $\mathcal N_c$ of $c$ such that
\begin{align}\begin{split}\nonumber
\overline{
\bigcup_{c' \in \mathcal N_c}
h^{-1}\(c'\right)} \subset E
\end{split}\end{align}
and is bounded. The function $f$ is continuous.
\end{manualproposition}
In addition, when $k = 1$ and when we only need Hadamard differentiability on a singleton $\{c\}$,
where $c \in h\(E\right)$, Assumption \ref{regular value assumption} can be replaced by the following
\begin{manualproposition}{\ref{regular value assumption}$'$}\label{regular value assumption '}
$K_f \cap h^{-1}\(c\right)$ consists of only regular points of $h$.
\end{manualproposition}



\begin{comment}
\begin{lemma}\label{tube like lemma}
(A tube like lemma)
Let $k \in \{1, 2, \ldots, n\}$,
$E \subset \mathbb{R}^{n}$ be an
open set, $\Delta: E \mapsto \mathbb{R}^{k}$
be a $C^{1}$ function,
$c_{0} \in \mathbb{R}^{k}$.
Then for all regular point
$x \in \Delta^{-1}\(c_{0}\right)$
there exists
$n - k$ coordinates
$\mathcal{I} =
\{i_{1}, i_{2}, \ldots, i_{n - k}\}$,
open cubes $I_{\mathcal{I}} \subset
\mathbb{R}^{n - k},
I_{\mathcal{I}_{c}} \subset \mathbb{R}^{k}$, i.e.
\begin{align}\begin{split}\nonumber
I_{\mathcal{I}} =
X_{i_{1}} \times X_{i_{2}} \times \ldots
X_{i_{n - k}}, \;
I_{\mathcal{I}_{c}} =
X_{j_{1}} \times X_{j_{2}} \times \ldots
X_{j_{k}},
\end{split}\end{align}
where $\mathcal{I}_{c} =
\{j_{i}, \ldots, j_{k}\} = \{
1, 2, \ldots, n\} \backslash \mathcal{I}$,
$X_{i_{l}}, X_{j_{m}}$ are open intervals,
and $\eta > 0$,
such that $I_{\mathcal{I}}, I_{\mathcal{I}_{c}}$
are neighborhoods of
$x_{\mathcal{I}} =
\(x_{i_{1}}, x_{i_{2}}, \ldots,
x_{i_{n - k}}\right)$ and
$x_{\mathcal{I}_{c}} =
\(x_{j_{1}}, x_{j_{2}}, \ldots,
x_{j_{k}}\right)$, respectively,
and for all $c$,
if $\vert c_{0} - c\vert < \eta$,
then
\begin{align}\begin{split}\label{tube cover}
I_{\mathcal{I}} & \subset
\{x_{\mathcal{I}} =
\(x_{i_{1}}, x_{i_{2}, \ldots}, x_{i_{n - k}}\right)
\vert
x \in \Delta^{-1}\(c\right)\}, \\
\end{split}\end{align}
Further more,
\begin{align}\begin{split}\label{thin tube}
I_{\mathcal{I}_{c}}  \supsetneqq
\{x_{\mathcal{I}_{c}} =
\(x_{j_{1}}, x_{j_{2}}, \ldots, x_{j_{k}}\right)
\vert x \in \Delta^{-1}\(c\right)\}.
\end{split}\end{align}
\end{lemma}

\begin{proof}
By assumption,
let $x_{0} \in \Delta^{-1}\(c_{0}\right)$
be a regular point,
$J\Delta\(x_{0}\right) > 0$, i.e.
$\partial\Delta\(x_{0}\right)$ is row full rank.
Therefore, the implicit function theorem
can be applied in a neighborhood of $x_{0}$.
Specifically, pick $k$ linearly independent
columns of $\partial\Delta\(x_{0}\right)$.
Without loss of generality,
we may assume that the first $k$ columns
$\{\partial_{x_{1}}\Delta\(x_{0}\right),
\partial_{x_{2}}\Delta\(x_{0}\right), \ldots,
\partial_{x_{k}}\Delta\(x_{0}\right)\}$
are linearly independent.
Let $U \in \mathbb{R}^{n + k}$
be a neighborhood of
$\(x_{0, 1}, \ldots, x_{0, k},
x_{0, k + 1}, \ldots, x_{0, n}, c_{0}\right)$.
Define $\Psi: U \mapsto \mathbb{R}^{k}$ as
$\(x_{1}, \ldots, x_{k},
x_{k + 1}, \ldots, x_{n}, c\right) \mapsto
\Delta\(x_{1}, \ldots, x_{k},
x_{k + 1}, \ldots, x_{n}\right)
- c,$
then $\partial_{x_{1}, x_{2}, \ldots, x_{k}}
\Psi\(x_{0, 1}, \ldots, x_{0, k},
x_{0, k + 1}, \ldots, x_{0, n}, c_{0}\right)$
is full rank. Thus, there exists
an open
subset $B_{x_{1}, \ldots, x_{k}} \times
B_{x_{k + 1}, \ldots, x_{n}, c} \subset U$,
where
\begin{align}\begin{split}\nonumber
B_{x_{1}, \ldots, x_{k}} & =
\{\(x_{1}, \ldots, x_{k}\right) \in \mathbb{R}^{k}
\vert \vert\(x_{0, 1}, \ldots, x_{0, k}\right) -
\(x_{1}, \ldots, x_{k}\right)
\vert < \alpha\}, \\
B_{x_{k + 1}, \ldots, x_{n}, c} & =
\{\(x_{k + 1}, \ldots, x_{n}, c\right)
\in \mathbb{R}^{n}
\vert \vert\(x_{0, k + 1},
\ldots, x_{0, n}, c_{0}\right) -
\(x_{k + 1}, \ldots, x_{n}, c\right)
\vert < \beta\},
\end{split}\end{align}
and a $C^{1}$ implicit function $g\left(\cdot\right)$ defined on
$B_{x_{k + 1}, \ldots, x_{n}, c}$,
such that
\begin{align}\begin{split}\nonumber
\Psi\(x_{1}, \ldots, x_{k},
x_{k + 1}, \ldots, x_{n}, c\right) = 0
\Leftrightarrow
\(x_{1}, \ldots, x_{k}\right) =
g\(x_{k + 1}, \ldots, x_{n}, c\right)
\end{split}\end{align}
for all $\(x, c\right) \in
B_{x_{1}, \ldots, x_{k}} \times
B_{x_{k + 1}, \ldots, x_{n}, c}$.
Now consider the inscribed
square open cube $I_{s}$ of
$B_{x_{k + 1}, \ldots, x_{n}, c}$.
Denote the edge length of
$I_{s}$ as $2a$ and
\begin{align}\begin{split}\nonumber
I_{s, x_{k + 1}, \ldots, x_{n}} =
\{\(x_{k + 1}, \ldots, x_{n}\right) \vert
\(x_{k + 1}, \ldots, x_{n}, c\right)
\in I_{s}\}.
\end{split}\end{align}
Then
for all $\(x'_{k + 1}, \ldots, x'_{n}\right)
\in I_{s, x_{k + 1}, \ldots, x_{n}}$, the following holds
$\(x'_{k + 1}, \ldots, x'_{n}, c'\right)
\in I_{s}, \; \forall c',
\vert c_{0} - c'\vert < a.$
In other words,
for all $c'$, if
$\vert c_{0} - c'\vert < a$, then
$I_{s, x_{k + 1}, \ldots, x_{n}}  \subset
\{\(x_{k + 1}, \ldots, x_{n}\right)\vert
x \in \Delta^{-1}\(c'\right)\}.$
Denote
$g\(x_{k + 1}, \ldots, x_{n}, c\right) =
\(g_{1}\(x_{k + 1}, \ldots, x_{n}, c\right),
\ldots, g_{k}\(x_{k + 1}, \ldots,
x_{n}, c\right)\right)^{T},$
then
by the Morrey type inequality
Lemma \ref{Morrey inequality},
for $n < p < \infty$,
\begin{align}\begin{split}\label{tube estimate}
\vert \(x_{0, 1}, \ldots, x_{0, k}\right) & -
g\(x_{k + 1}, \ldots, x_{n}, c\right)\vert =
\vert g\(x_{0, k + 1},
\ldots, x_{0, n}, c_{0}\right) -
g\(x_{k + 1}, \ldots, x_{n}, c\right)\vert \\
& \leq \sum^{k}_{m = 1}
\vert g_{m}\(x_{0, k + 1},
\ldots, x_{0, n}, c_{0}\right) -
g_{m}\(x_{k + 1}, \ldots, x_{n}, c\right)\vert \\
& \leq \sum^{k}_{m = 1}
Cr^{1 - \frac{n}{p}}\left(
\int_{B\left(\(x_{0, k + 1}, \ldots, x_{0, n},
c_{0}\right), r\right)}\vert \partial
g_{m}\(z\right)\vert^{p}
d\mathcal{L}_{n}z\right)^{\frac{1}{p}} \\
& \leq kCr^{1 - \frac{n}{p}}\left(
\int_{B\left(\(x_{0, k + 1}, \ldots, x_{0, n},
c_{0}\right), r\right)}\vert \partial
g\(z\right)\vert^{p}
d\mathcal{L}_{n}z\right)^{\frac{1}{p}},
\end{split}\end{align}
for all $0 < r < \beta$, with constant
$C$ depending only on $n$ and $p$,
when $\(x_{k + 1}, \ldots, x_{n}, c\right) \in
B\left(\(x_{0, k + 1}, \ldots, x_{0, n},
c_{0}\right), r\right)$.
Write the right hand side of the last
inequality of \eqref{tube estimate} as
$\Lambda\(x_{0}, c_{0}, r, n, p\right)$.
For a fix $r$ and the inscribed square open
cube $I'_{s}$ of
$B\left(\(x_{0. k + 1}, \ldots, x_{0, n},
c_{0}\right), r\right)$, let
$I_{x_{k + 1}, \ldots, x_{n}} =
\{\(x_{k + 1}, \ldots, x_{n}\right) \vert
\(x_{k + 1}, \ldots, x_{n}, c\right)
\in I'_{s}$,
\begin{align}\begin{split}\nonumber
I_{x_{1}, \ldots, x_{k}} =
\prod^{k}_{m = 1}\(x_{0, m} -
\Lambda\(x_{0}, c_{0}, r, n, p\right),
x_{0, m} +
\Lambda\(x_{0}, c_{0}, r, n, p\right)\right).
\end{split}\end{align}
Denote the edge length of
$I'_{s}$ as $2\eta$,
then
for all $c, \vert c_{0} -
c\vert < \eta$, both \eqref{tube cover}
and
\eqref{thin tube} hold
for $I_{x_{k + 1}, \ldots, x_{n}}$ and
$I_{x_{1}, \ldots, x_{k}}$.
\end{proof}

\end{comment}

\begin{comment}
\begin{lemma}\label{manifold integral continuity}
{A generalization of Lemma E.3 in
\cite{chernozhukov2018sorted}} \\
Let $k \in \{1, 2, \ldots, n\}$,
$E \subset \mathbb{R}^{n}$ be an open set,
$\Delta: E \mapsto \mathbb{R}^{k}$ be a $C^{1}$
function,
$f: E \mapsto \mathbb{R}$
be a continuous function.
Suppose the set of regular values
$\mathcal{R} \subset \Delta\(E\right)$
is not
empty\footnote{If for all connected branches
$\mathcal{K} \subset E$,
$\Delta$ is constant on $\mathcal{K}$,
then $\mathcal{R} = \emptyset$.},
and $\mathcal{R}$ contains
an open subset $\mathcal{R}_{0}$.
Suppose also $\overline{
\bigcup_{c \in \mathcal{R}_{0}}
\Delta^{-1}\(c\right)} \subset E$
and is bounded.
Then,
\begin{align}\begin{split}\nonumber
\int_{\Delta^{-1}\(c\right)}\frac{f\(x\right)}
{J\Delta\(x\right)}d\mathcal{H}_{n - k}x
\end{split}\end{align}
is continuous on $\mathcal{R}_{0}$.
\end{lemma}

\begin{proof}
Let
$c_{0} \in \mathcal{R}_{0}$,
by Lemma \ref{tube like lemma},
for all $x \in \Delta^{-1}\(c_{0}\right)$,
there exists open cubes
$I_{\mathcal{I}}, I_{\mathcal{I}_{c}}$ and
open ball $B\(c_{0}, \eta\right)$ such that
for all $\delta \in B\(c_{0}, \eta\right)$,
\eqref{tube cover} holds.
Notice that
$\Delta^{-1}\(c_{0}\right)$
is relatively closed in $E$,
\begin{align}\begin{split}\nonumber
\Delta^{-1}\(c_{0}\right) =
\overline{\Delta^{-1}\(c_{0}\right)} \cap E
\subset \overline{\Delta^{-1}\(c_{0}\right)}
\subset \overline{
\bigcup_{c \in \mathcal{R}_{0}}
\Delta^{-1}\(c\right)} \subset E.
\end{split}\end{align}
Thus $\Delta^{-1}\(c_{0}\right)$ and
$\overline{\bigcup_{c \in \mathcal{R}_{0}}
\Delta^{-1}\(c\right)}$
are compact.
As such,
there exists a finite cover
$\{B_{j}\}$ of
$\Delta^{-1}\(c_{0}\right)$
such that $B_{j}$ is
constructed by the corresponding
open intervals of
$I_{j, \mathcal{I}}$ and
$I_{j, \mathcal{I}_{c}}$.

We argue that there exists an $\eta$ small enough such that
for all $c$, $\vert c_{0} -
c\vert < \eta$, $\Delta^{-1}\(c\right) \in
\bigcup_{j}B_{j}$. To see this,
consider $\overline{\bigcup_{c \in
\mathcal{R}_{0}}
\Delta^{-1}\(c\right)} \cap \Delta^{-1}\(c_{i}\right)$
for an arbitrary sequence
$\{c_{i}\}^{\infty}_{i = 1}$,
$\lim_{i \to \infty}c_{i} = c_{0}$.
The compactness of
$\overline{\bigcup_{c \in \mathcal{R}_{0}}
\Delta^{-1}\(c\right)}$ implies that
$\overline{\bigcup_{c \in \mathcal{R}_{0}}
\Delta^{-1}\(c\right)} \cap \Delta^{-1}\(c_{i}\right)
\subset \bigcup_{j}B_{j}$ for all
$i$ large enough.
To prove by contradiction, suppose the above claim is not
true. Then there exists
a sequence $\{x_{i}\}^{\infty}_{i = 1}$
such that
\begin{align}\begin{split}\nonumber
x_{i} \in \overline{\bigcup_{c \in
\mathcal{R}_{0}}
\Delta^{-1}\(c\right)} \cap
\Delta^{-1}\(c_{i}\right), \quad
x_{i} \notin \bigcup_{j}B_{j}.
\end{split}\end{align}
By the Bolzano–Weierstrass theorem,
there exists a convergent subsequence
$\{x'_{i}\}^{\infty}_{i = 1}$,
\begin{align}\begin{split}\nonumber
\lim_{i \to \infty}
\Delta\(x_{i}'\right) = c_{0}\quad \text{and thus}\quad
\lim_{i \to \infty}x'_{i}
\subset \Delta^{-1}\(c_{0}\right) \subset
\bigcup_{j}B_{j}.
\end{split}\end{align}
Therefore,
$x_{i} \in \bigcup_{j}B_{j}$
for all sufficiently large $i$,
which is a contradiction.
For all $B_{j}$,
we can use
Theorem
\ref{area formula classic} to apply
a change of variable technique
for all $\Delta^{-1}\(c\right) \cap B_{j}$
to the same cube $I_{\mathcal{I}_{j}}$,
where $\vert c_{0} - c\vert < \eta$.
The change of variable maps
\begin{align}\begin{split}\nonumber
\psi_{B_{j}}\(x_{\mathcal{I}_{j}}, c\right) =
\psi_{B_{j}}\(x_{i_{1}}, \ldots, x_{i_{n - k}}, c\right) =
\(g_{B_{j}}\(x_{i_{1}}, \ldots, x_{i_{n - k}}, c\right),
x_{i_{1}}, \ldots, x_{i_{n - k}}\right)^{T}
\end{split}\end{align}
are $C^{1}$ diffeomorphisms.
By the generalized Matrix determinant lemma,
\begin{align}\begin{split}\nonumber
J\psi & = \det\left(
\left(
\left(\partial_{x_{\mathcal{I}_{c}}}\Delta^{-1}
\partial_{x_{\mathcal{I}}}\Delta\right)^{T},
I_{n - k}\right)
\left(\begin{array}{c}
\partial_{x_{\mathcal{I}_{c}}}\Delta^{-1}
\partial_{x_{\mathcal{I}}}\Delta \\
I_{n - k}
\end{array}\right)\right)^{\frac{1}{2}} \\
& = \det\(I_{n - k} +
\left(\partial_{x_{\mathcal{I}_{c}}}\Delta^{-1}
\partial_{x_{\mathcal{I}}}\Delta\right)^{T}
\left(\partial_{x_{\mathcal{I}_{c}}}\Delta^{-1}
\partial_{x_{\mathcal{I}}}\Delta\right)\right)
^{\frac{1}{2}} \\
& = \left(\det\(I_{n - k}\right)
\det\(I_{k} + \left(\partial_{x_{\mathcal{I}_{c}}}\Delta^{-1}
\partial_{x_{\mathcal{I}}}\Delta\)I_{n - k}
\left(\partial_{x_{\mathcal{I}_{c}}}\Delta^{-1}
\partial_{x_{\mathcal{I}}}\Delta\right)^{T}\right)\right)
^{\frac{1}{2}} \\
& = \left(\det\left(\left(\partial_{\mathcal{I}_{c}}
\Delta\right)\left(\partial_{\mathcal{I}_{c}}
\Delta\right)^{T} +
\left(\partial_{\mathcal{I}}\Delta\right)
\left(\partial_{\mathcal{I}}\Delta\right)^{T}\right)\right)
^{\frac{1}{2}}
\vert\det\left(\partial_{x_{\mathcal{I}_{c}}}
\Delta^{-1}\right)\vert \\
& = \left(\det\left(\left(\partial_{x_{\mathcal{I}_{c}}}
\Delta, \partial_{x_{\mathcal{I}}}\Delta\right)
\left(\begin{array}{c}
\left(\partial_{x_{\mathcal{I}_{c}}}
\Delta\right)^{T} \\
\left(\partial_{x_{\mathcal{I}}}
\Delta\right)^{T}
\end{array}\right)
\right)\right)^{\frac{1}{2}}
\vert\det\left(\partial_{x_{\mathcal{I}_{c}}}
\Delta^{-1}\right)\vert \\
& = J\Delta \vert\det\left(\partial_{x_{\mathcal{I}_{c}}}
\Delta^{-1}\right)\vert.
\end{split}\end{align}
Then,
by partition of unity as
in \cite{chernozhukov2018sorted}
and area formula
\eqref{area formula integral},
\begin{align}\begin{split}\nonumber
\int_{\Delta^{-1}\(c\right)}
\frac{f\(x\right)}{J\Delta\(x\right)}
d\mathcal{H}_{n - k}x & =
\sum_{j}
\int_{\Delta^{-1}\(c\right) \cap B_{j}}
\frac{p_{j}\(x\)f\(x\right)}
{J\Delta\(x\right)}d\mathcal{H}_{n - k}x \\
& =
\sum_{j}
\int_{I_{\mathcal{I}_{j}}}
\frac{p_{j}\left(\psi_{B_{j}}\(x_{\mathcal{I}_{j}}, c\right)\right)
f\left(\psi_{B_{j}}\(x_{\mathcal{I}_{j}}, c\right)\right)}
{\left\vert\det\left(
\partial_{x_{{\mathcal{I}_{c}}_{j}}}
\Delta\left(\psi_{B_{j}}\(x_{\mathcal{I}_{j}}, c
\right)\right)\right)\right\vert}
d\mathcal{L}_{n - k}x_{\mathcal{I}_{j}}.
\end{split}\end{align}
The conclusion follows from the continuity
of integrands and \eqref{thin tube}.
\end{proof}

\end{comment}

\begin{comment}
\begin{remark}
\indent
\begin{enumerate}
  \item (Partition of unity) Let $E \subset
  \mathbb{R}^{n}$ be a open set.
  $\Sigma$ is a open cover of $E$.
  Then there exists a collection of $C^{\infty}$
  functions $\{p_{\alpha}\}$ on $E$ such that
  \begin{enumerate}[1)]
    \item $0 \leq p_{\alpha} \leq 1$ and
    $\supp g_{\alpha}$ is compact
    for all $\alpha$. Moreover, there exists
    an open set $W_{i} \in \Sigma$ such that
    $\supp p_{\alpha} \in W_{i}$.
    \item For all $x \in E$, there exits a
    neighborhood $U$ of $x$ such that
    $U \cap \supp g_{\alpha} \neq \emptyset$
    for only finite number of $\alpha$.
    \item $\sum_{\alpha}g_{\alpha} = 1$.
  \end{enumerate}
  \item For the form of partition of unity
  used here, see Theorem 3.2 in
  \cite{chen1999lectures}.
  \item The assumption that
  \textit{$\overline{
  \bigcup_{c \in \mathcal{R}_{0}}
  \Delta^{-1}\(c\right)} \subset E$
  and is bounded} aims at providing
  compactness to let
  $\Delta^{-1}\(c\right)$
  ``finely concentrates'' to
  $\Delta^{-1}\(c_{0}\right)$. It can be
  replaced by an alternative assumption	 such as
  \textit{$\supp f \subset E$ and is compact}.
\end{enumerate}
\end{remark}

\end{comment}


\begin{comment}
\begin{cor}\label{differential of threshold}
{A refined version of Lemma A.1 in
\cite{chernozhukov2018sorted}} \\
Let $E \subset \mathbb{R}^{n}$ be an open set,
$\Delta: E \mapsto \mathbb{R}$ be a $C^{1}$
function, $f: E \mapsto \mathbb{R}$
be a continuous function.
Suppose the set of regular values
$\mathcal{R} \subset \Delta\(E\right)$
is not empty, and $\mathcal{R}$ contains
an open subset $\mathcal{R}_{0}$.
Suppose also $\overline{
\bigcup_{c \in \mathcal{R}_{0}}
\Delta^{-1}\(c\right)} \subset E$
and is bounded.
Then, for all $\delta \in \mathcal{R}_{0}$,
\begin{align}\begin{split}\nonumber
\frac{d}{d\delta}\left(\int_{\Delta\(x\right) > \delta}
f\(x\)d\mathcal{L}_{n}x\right) =
- \int_{\Delta^{-1}\left(\delta\right)}\frac{f\(x\right)}
{\vert\partial\Delta\(x\right)\vert}d\mathcal{H}_{n - 1}x.
\end{split}\end{align}
\end{cor}

\begin{proof}
Use Lemma \ref{protype derivative} and
Lemma \ref{manifold integral continuity}.
\end{proof}


Intuitively, the above discussions and results
can be further generalized from
$C^{1}$ / Lipschitz functions to certain
collections of Sobolev functions.
See e.g. \cite{maly2003co}.
\end{comment}

\begin{comment}
{\color{purple}{
Figure \ref{volume integral intuition}
illustrates the insight of coarea formula
in this section and the inspiration in
\cite{chernozhukov2018sorted}.
Partition $x = \(y, z\right)$ where
$z\in \mathbb{R}^{1}$ and
$y \in \mathbb{R}^{n - 1}$.
The volume of the differential parallelepiped
formed by the manifold surface $OB$ and
the ray $OA$ is the same as $dy dz = d\text{Vol}dh$.
It then remains to show that
$dh = d\delta / \vert\partial\Delta\(x\right)\vert$.
It will follow from $dh = dz\cos\theta$.
The gradient
vector $\partial \Delta\(x\right)$ is perpendicular to
$OB$ and parallel to $dh$.
The angle $\theta$ is between
$\Delta\(x\right)$ and the z-axis, or $\left(0, \ldots, 0 , 1\right)$.
Therefore
\begin{align}\begin{split}\nonumber
\cos \theta = \frac{\partial\Delta\(x\right)
\left(0, \ldots, 0 , 1\right)}
{\vert\partial\Delta\(x\right)\vert} =
\frac{\partial_{z}\Delta\(x\right)}
{\vert\partial\Delta\(x\right)\vert}.
\end{split}\end{align}
Then $dh = dz\partial_{z}\Delta\(x\right) /
\vert\partial \Delta\(x\right)\vert$.
But $d\delta = dz\partial_{z}\Delta\(x\right)$
is exactly the
difference in the function value
between point $O$ and $A$.
\begin{figure}
\begin{center}
\caption{Intuition of volume integral}\label{volume integral intuition}
\includegraphics[height=\textheight]{volumeintegralintuition.pdf}
\end{center}
\end{figure}
Indeed this change of variables method holds generally
even when $\Delta\(x\right)$ is a vector instead of a scalar, in
which case, intuitively,
\begin{align}\begin{split}\nonumber
\int f\(x\)dx =
\iint\frac{f\(x\right)}
{\sqrt{\det\left(\partial\Delta\(x\right)\partial\Delta\(x\right)^{T}\right)}}
d\text{Vol}d\delta.
\end{split}\end{align}
${\sqrt{\det\left(\partial\Delta\(x\right)\partial
\Delta\(x\right)^{T}\right)}}$ is the ratio of
the infinitesimal volume of $d\delta$ to
the parallelepiped perpendicular to $d\text{Vol}$,
it needs to be compensated inversely
in the direction of $d\text{Vol}$ in order that the
infinitesimal volume of $d\text{Vol}d\delta$ the same
as the original $dx$ cube.
}}
\end{comment}



\begin{comment}
\begin{lemma}
\label{simple Hadamard differentiability}
{A refined version of Lemma E.4 in
\cite{chernozhukov2018sorted}} \\
Let $E \subset \mathbb{R}^{n}$ be an open set,
$\Delta: E \mapsto \mathbb{R}$ be a $C^{1}$
function, $f: E \mapsto \mathbb{R}$
be a continuous function with
compact support $K_{f} \subset E$.
Suppose the set of regular values
$\mathcal{R} \subset \Delta\(E\right)$
is not empty, and $\mathcal{R}$ contains
an open subset $\mathcal{R}_{0}$.
Then, the map
$F_{\Delta, f, \delta}:
C\(E\right) \mapsto \mathbb{R}$
is Hadamard differentiable for
all $\delta \in \mathcal{R}_{0}$
tangentially to $C_{0}\(E\right)$.
The derivative is given by
\begin{align}\begin{split}
\label{simple Hadamard derivative}
F'_{\Delta, f, \delta} =
\int_{\Delta^{-1}\left(\delta\right)}
\frac{G\(x\)f\(x\right)}{\vert\partial\Delta\(x\right)\vert}
d\mathcal{H}_{n - 1}x.
\end{split}\end{align}
\end{lemma}
\end{comment}

\begin{comment}
\begin{proof}[Proof of Theorem \ref{derivative k=1}]
We break the proof into several steps.
\begin{enumerate}[\textbf{Step} 1]
  \item Prove that, for
  a closed cube $Q$,
  $G_{n} = G \equiv 1_{Q}$,
\eqref{theorem 2.1 derivative k=1}
  applies, i.e.
  \begin{align}\begin{split}\nonumber
  \frac{F_{\Delta + t_{n}1_{Q}, f, \delta} -
  F_{\Delta, f, \delta}}
  {t_{n}} \to \int_{\Delta^{-1}\left(\delta\right)}
  \frac{1_{Q}\(x\)f\(x\right)}
  {\vert\partial\Delta\(x\right)\vert}
  d\mathcal{H}_{n - k}x,
  \end{split}\end{align}
  for all $t_{n} \downarrow 0$.
  Since $f$ has a compact support $K_{f}$,
  \begin{align}\begin{split}\nonumber
  F_{\Delta, f, \delta} & =
  \int_{K_{f}}1\left(\Delta\(x\right) > \delta\right)
  f\(x\)d\mathcal{L}_{n}x, \\
  F_{\Delta + t_{n}G_{n}, f, \delta} & =
  \int_{K_{f}}1\left(\Delta\(x\right) +
  t_{n}G_{n}\(x\right) > \delta\right)
  f\(x\)d\mathcal{L}_{n}x.
  \end{split}\end{align}
  Note that for arbitrary $t_{n} > 0$,
  \begin{align}\begin{split}\nonumber
  \Delta\(x\right) + t_{n}1_{Q}\(x\right) > \delta
  \quad & \text{for all} \quad
  x \in \bigcup_{c > \delta}\Delta^{-1}\(c\right), \\
  \Delta\(x\right) + t_{n}1_{Q}\(x\right) < \delta
  \quad & \text{for all} \quad
  x \in \bigcup_{c \leq \delta - t_{n}}\Delta^{-1}\(c\right),
  \end{split}\end{align}
  and thus we have
  \begin{align}\begin{split}\nonumber
  F_{\Delta + t_{n}1_{Q}, f, \delta} & -
  F_{\Delta, f, \delta} =
  \int_{K_{f}}\left[1\left(\Delta\(x\right) +
  t_{n}1_{Q}\(x\right) > \delta\right) -
  1\left(\Delta\(x\right) > \delta\right)\]f\(x\right)
  d\mathcal{L}_{n}x \\
  & =
  \int_{\bigcup_{\delta - t_{n} < c \leq \delta}
  \Delta^{-1}\(c\right) \cap K_{f}}
  \left[1\left(\Delta\(x\right) +
  t_{n}1_{Q}\(x\right) > \delta\right) -
  1\left(\Delta\(x\right) > \delta\right)\]f\(x\right)
  d\mathcal{L}_{n}x.
  \end{split}\end{align}
  For $n$ large enough such that
  $\left(\delta - t_{n}, \delta\right] \subset \mathcal{R}_{0}$,
  by coarea formula \eqref{coarea formula integral},
  we can write
  \begin{align}\begin{split}\nonumber
  & \frac{F_{\Delta + t_{n}1_{Q}, f, \delta} -
  F_{\Delta, f, \delta}}{t_{n}} = \\
  & \frac{1}{t_{n}}
  \int^{\delta}_{\delta - t_{n}}
  \left(\int_{\Delta^{-1}\(c\right) \cap K_{f}}
  \frac{\left[1\(c + t_{n}1_{Q}\(x\right) > \delta\right) -
  1\(c > \delta\right)\]f\(x\right)}
  {\vert\partial\Delta\(x\right)\vert}
  d\mathcal{H}_{n - 1}x\)d\mathcal{L}_{1}c.
  \end{split}\end{align}
  Now, consider the part in the parentheses above,
  \begin{align}\begin{split}\nonumber
  & \int_{\Delta^{-1}\(c\right) \cap K_{f}}
  \frac{\left[1\(c + t_{n}1_{Q}\(x\right) > \delta\right) -
  1\(c > \delta\right)\]f\(x\right)}
  {\vert\partial\Delta\(x\right)\vert}
  d\mathcal{H}_{n - 1}x \\
  & =
  \int_{\Delta^{-1}\(c\right) \cap K_{f} \cap Q}
  \frac{\left[1\(c + t_{n} > \delta\right) -
  1\(c > \delta\right)\]f\(x\right)}
  {\vert\partial\Delta\(x\right)\vert}
  d\mathcal{H}_{n - 1}x,
  \end{split}\end{align}
  since for all $x \notin Q$,
  $t_{n}1_{Q}\(x\right) = 0$.
  By the compactness of $K_{f} \subset E$ and
  the relative closeness of
  $\Delta^{-1}\(c\right)$ in $E$,
  $\Delta^{-1}\(c\right) \cap K_{f} \cap Q$
  is compact for all $c \in \mathbb{R}$.
  Therefore, we can construct
  finite tube like cover by
  Lemma \ref{tube like lemma}
  as in the proof of Lemma
  \ref{manifold integral continuity}
  for $\Delta^{-1}\left(\delta\right) \cap K_{f} \cap Q$.
  Noting that $1\(c > \delta\right) = 0$ when
  $c \leq \delta$
  and $1\(c + t_{n} > \delta\right) = 1$
  when $c \in \left(\delta - t_{n}, \delta\right]$,
  we have
  \begin{align}\begin{split}\nonumber
  & \frac{F_{\Delta + t_{n}1_{Q}, f, \delta} -
  F_{\Delta, f, \delta}}{t_{n}} =
\frac{1}{t_{n}}
  \int^{\delta}_{\delta - t_{n}}
  \left(\int_{\Delta^{-1}\(c\right) \cap K_{f} \cap Q}
  \frac{f\(x\right)}
  {\vert\partial\Delta\(x\right)\vert}
  d\mathcal{H}_{n - 1}x\)d\mathcal{L}_{1}c.
  \end{split}\end{align}
  Then by
repeating the
  proof of Lemma \ref{manifold integral continuity},
where
the compactness
  of $\overline{\bigcup_{c \in \mathcal{R}_{0}}
  \Delta^{-1}\(c\right)}$ is replaced by the compactness
  of $K_{f}$, we have
  \begin{align}\begin{split}\nonumber
  \frac{F_{\Delta + t_{n}1_{Q}, f, \delta} -
  F_{\Delta, f, \delta}}{t_{n}} & \to
  \int_{\Delta^{-1}\left(\delta\right) \cap K_{f} \cap Q}
  \frac{f\(x\right)}
  {\vert\partial\Delta\(x\right)\vert}
  d\mathcal{H}_{n - 1}x
\\ &
=
  \int_{\Delta^{-1}\left(\delta\right)}
  \frac{1_{Q}\(x\)f\(x\right)}
  {\vert\partial\Delta\(x\right)\vert}
  d\mathcal{H}_{n - 1}x.
  \end{split}\end{align}
  \item It is clear from the proof
  in \textbf{step} 1 that
\eqref{theorem 2.1 derivative k=1}
  holds for all closed cube $Q$,
  $G_{n} = G \equiv C1_{Q}$, where
  $C$ is an arbitrary constant.
  Next, let
  \begin{align}\begin{split}\nonumber
  \mathcal{Q}_{m} = \left\{Q \vert Q =
  \prod^{n}_{i = 1}\(a_{i}, b_{i}\right],
  a_{i} = \frac{c_{i}}{m},
  b_{i} = \frac{c_{i} + 1}{m},
  c_{i} \in \mathbb{Z}\right\}.
  \end{split}\end{align}
  Similar to \textbf{part} 3 of
  the proof of Theorem \ref{area formula classic},
  we will prove that for
  a finite subset $\{Q_{j}\}$ of $\mathcal{Q}_{m}$
  and
  $G_{n} = G = \sum_{j}
  C_{j}1_{Q_{j}},$
\eqref{theorem 2.1 derivative k=1}
applies.
  Since the boundary of a cube in $\mathbb{R}^{n}$
  is always of $0$ $\mathcal{L}_{n}$ measure,
  in this case
  \begin{align}\begin{split}\nonumber
  F_{\Delta, f, \delta} & =
  \sum_{j}\int_{K_{f} \cap Q_{j}}
  1\left(\Delta\(x\right) > \delta\)f\(x\right)
  d\mathcal{L}_{n}x + Res, \\
  & =
  \sum_{j}\int_{K_{f} \cap
  \overline{Q_{j}}}
  1\left(\Delta\(x\right) > \delta\)f\(x\right)
  d\mathcal{L}_{n}x + Res, \\
  F_{\Delta + t_{n}G_{n}, f, \delta} & =
  \sum_{j}\int_{K_{f} \cap Q_{j}}
  1\left(\Delta\(x\right) + t_{n}C_{j}1_{Q_{j}}\(x\right) >
  \delta\)f\(x\right)
  d\mathcal{L}_{n}x + Res, \\
  & =
  \sum_{j = 1}\int_{K_{f} \cap
  \overline{Q_{j}}}
  1\left(\Delta\(x\right) + t_{n}C_{j}1_{Q_{j}}\(x\right) >
  \delta\)f\(x\right)
  d\mathcal{L}_{n}x + Res,
  \end{split}\end{align}
  where
  \begin{align}\begin{split}\nonumber
  Res = \int_{K_{f} \cap \(E \backslash \bigcup_{j}Q_{j}\right)}
  1\left(\Delta\(x\right) > \delta\)f\(x\right)
  d\mathcal{L}_{n}x =
  \int_{K_{f} \cap \(E \backslash \bigcup_{j}
  \overline{Q_{j}}\right)}
  1\left(\Delta\(x\right) > \delta\)f\(x\right)
  d\mathcal{L}_{n}x.
  \end{split}\end{align}
  Therefore, we have
  \begin{align}\begin{split}\nonumber
  \frac{F_{\Delta + t_{n}G_{n}, f, \delta} -
  F_{\Delta, f, \delta}}
  {t_{n}} & =
  \frac{1}{t_{n}}\sum_{j}
  \int_{K_{f} \cap \overline{Q_{j}}}\left[
  1\left(\Delta\(x\right) + t_{n}C_{j}1_{Q_{j}}\(x\right) > \delta\right) -
  1\left(\Delta\(x\right) > \delta\right)\]f\(x\right)
  d\mathcal{L}_{n}x \\
  & \to
  \sum_{j}\int_{\Delta^{-1}\left(\delta\right)}
  \frac{C_{j}1_{Q_{j}}\(x\)f\(x\right)}
  {\vert\partial\Delta\(x\right)\vert}
  d\mathcal{H}_{n - 1}x =
  \int_{\Delta^{-1}\left(\delta\right)}
  \frac{G\(x\)f\(x\right)}
  {\vert\partial\Delta\(x\right)\vert}
  d\mathcal{H}_{n - 1}x.
  \end{split}\end{align}
  \item As in the
  proof of Lemma E.4 in
  \cite{chernozhukov2018sorted},
  for arbitrary $\zeta > 0$,
  there exist $N$ large enough such that
  $\sup_{n \geq N}\sup_{x \in E}
  \vert G_{n} - G\vert < \zeta$.
  By assumption, $G \in C_{0}\(E\right)$,
  that is, $G$ is uniformly continuous.
  Thus, for all $\epsilon > 0$,
  there exists $M_{\mathcal{Q}}$ large
  enough such that for all
  $m > M_{\mathcal{Q}}$,
  $\sup_{Q}\sup_{x_{1}, x_{2} \in Q \cap E}
  \vert G\(x_{1}\right) - G\(x_{2}\right)\vert < \epsilon$.
  Therefore, there exists a
  simple function $S$ such that
  $\supp S \supset K_{f}$ and
  $\sup_{x \in K_{f}}
  \vert S\(x\right) - G\(x\right)\vert < \epsilon$.
  Then, for large enough $n$,
  \begin{align}\begin{split}\nonumber
  1\left(\Delta\(x\right) + t_{n}G_{n}\(x\right) > \delta\right)
  \leq
  1\left(\Delta\(x\right) + t_{n}\(S\(x\right) + \zeta + \epsilon\right)\right), \\
  1\left(\Delta\(x\right) + t_{n}G_{n}\(x\right) > \delta\right)
  \geq
  1\left(\Delta\(x\right) + t_{n}\(S\(x\right) - \zeta - \epsilon\right)\right).
  \end{split}\end{align}
  Now, we have
  \begin{align}\begin{split}\nonumber
  \limsup_{n \to \infty} &
  \frac{F_{\Delta + t_{n}G_{n}, f, \delta} -
  F_{\Delta, f, \delta}}
  {t_{n}} \\
  & \leq
  \lim_{n \to \infty}
  \frac{F_{\Delta +
  t_{n}\(S + \zeta + \epsilon\right), f, \delta} -
  F_{\Delta, f, \delta}}
  {t_{n}} =
  \int_{\Delta^{-1}\left(\delta\right)}
  \frac{\(S\(x\right) + \zeta + \epsilon\)f\(x\right)}
  {\vert\partial\Delta\(x\right)\vert}
  d\mathcal{H}_{n - 1}x, \\
  \liminf_{n \to \infty} &
  \frac{F_{\Delta + t_{n}G_{n}, f, \delta} -
  F_{\Delta, f, \delta}}
  {t_{n}} \\
  & \geq
  \lim_{n \to \infty}
  \frac{F_{\Delta +
  t_{n}\(S - \zeta - \epsilon\right), f, \delta} -
  F_{\Delta, f, \delta}}
  {t_{n}} =
  \int_{\Delta^{-1}\left(\delta\right)}
  \frac{\(S\(x\right) - \zeta - \epsilon\)f\(x\right)}
  {\vert\partial\Delta\(x\right)\vert}
  d\mathcal{H}_{n - 1}x.
  \end{split}\end{align}
  By H\"{o}lder's inequality
  \begin{align}\begin{split}\nonumber
  \int_{\Delta^{-1}\left(\delta\right)}
  \frac{\left( G\(x\right) - S\(x\right)\right) f\(x\right)}
  {\vert\partial\Delta\(x\right)\vert}
  d\mathcal{H}_{n - 1}x & \leq
  \sup_{x \in K_{f}}\vert S\(x\right) - G\(x\right)\vert
  \int_{\Delta^{-1}\left(\delta\right)}
  \left\vert\frac{f\(x\right)}
  {\vert\partial\Delta\(x\right)\vert}\right\vert
  d\mathcal{H}_{n - 1}x \\
  & \leq
  \epsilon\int_{\Delta^{-1}\left(\delta\right) \cap K_{f}}
  \left\vert\frac{f\(x\right)}
  {\vert\partial\Delta\(x\right)\vert}\right\vert
  d\mathcal{H}_{n - 1}x.
  \end{split}\end{align}
  Then by the arbitrariness of
  $\zeta$ and $\epsilon$,
  \begin{align}\begin{split}\nonumber
  \frac{F_{\Delta + t_{n}G_{n}, f, \delta} -
  F_{\Delta, f, \delta}}
  {t_{n}} \to
  \int_{\Delta^{-1}\left(\delta\right)}
  \frac{G\(x\)f\(x\right)}{\vert\partial\Delta\(x\right)\vert}
  d\mathcal{H}_{n - 1}x.
  \end{split}\end{align}
\end{enumerate}
\end{proof}
\end{comment}


\begin{comment}
\begin{lemma}
\label{multi Hadamard differentiability}
{A generalization of
Lemma
\ref{simple Hadamard differentiability}} \\
Let $E \subset \mathbb{R}^{n}$ be an open set,
$\Delta: E \mapsto \mathbb{R}^{k}$ be a $C^{1}$
function, $f: E \mapsto \mathbb{R}$
be a continuous function with
compact support $K_{f} \subset E$.
Suppose $\Delta\(E\right)$ is bounded,
the set of regular values
$\mathcal{R} \subset \Delta\(E\right)$
is not empty, and
$\mathcal{R}_{0} = \Delta\(E\right) \cap
\{\delta \vert \delta > \delta_{0}\}
\subset \mathcal{R}$ for a constant $\delta_{0}$.
Suppose also
\begin{align}\begin{split}\label{length bounded}
\sup_{\delta \in \mathcal{R}_{0}}
\int_{\Delta^{-1}\left(\delta\right)}
d\mathcal{H}_{n - k}x < \infty.
\end{split}\end{align}
Then, the map
$F_{\Delta, f, \delta}:
C\(E\right) \mapsto \mathbb{R}$
is Hadamard differentiable for
all $\delta \in \mathcal{R}_{0}$
tangentially to $C_{0}\(E\right)$.
The derivative is given by
\begin{align}\begin{split}
\label{multi Hadamard derivative}
F'_{\Delta, f, \delta}\(G\right) =
\sum_{\tau}
\int_{y > \delta_{\tau}}
\left(\int_{\Delta^{-1}\(y_{\tau}\right)}
\frac{G_{\neg\tau}\(x\)f\(x\right)}
{J\Delta\(x\right)}
d\mathcal{H}_{n - k}x\)d\mathcal{L}_{k - 1}y
,
\end{split}\end{align}
where $\delta = \left(\delta_{1}, \ldots, \delta_{k}\right)^{T}$,
$G(x) = \(G_{1}\(x\right), \ldots, G_{k}\(x\right)\right)^{T}$,
$\sum_{\tau}$ denotes the summation over the
$k$ combinations of $k - 1$ distinct elements
from $\{1, \ldots, k\}$,
$\delta_{\tau}$ denotes the
concatenation of $k - 1$
corresponding dimensions of $\delta$
according to the ascending order of
subscript,
$\neg\tau$ denotes
the only element unpicked of each combination,
and $y_{\tau}$ denotes the value obtained by
replacing the picked $k - 1$ dimensions of
$\delta$ with $y$.
\end{lemma}
\end{comment}


\begin{comment}
\begin{proof}[Proof of Theorem \ref{derivative general k}]
Let $\Delta\(x\right) =
\left(\Delta_{1}\(x\right), \ldots, \Delta_{k}\(x\right)\right)^{T}$,
$G_{n}\(x\right) =
\(G_{n, 1}\(x\right), \ldots, G_{n, k}\(x\right)\right)^{T}$.
As in the previous proof,
we
divide the arguments into $3$ steps.
\begin{enumerate}[\textbf{step} 1]
  \item Let $Q_{1}, \ldots, Q_{k}$
  be arbitrary closed cubes,
  $G_{n} = G \equiv
  \left(1_{Q_{1}}, \ldots, 1_{Q_{k}}\right)^{T}$.
  Let
  \begin{align}\begin{split}\nonumber
  D_{\tau, \Delta, G_{n}, f, \delta, t_{n}} & =
  \frac{1}{t_{n}}
  \int_{\Delta_{\tau} > \delta_{\tau}}
  \left[1\left(\Delta\(x\right) + t_{n}G_{n}\(x\right) > \delta\right) -
  1\left(\Delta\(x\right) > \delta\right)\]f\(x\)d\mathcal{L}_{n}x.
  \end{split}\end{align}
  Then we calculate the limit of
  $D_{\tau, \Delta, G_{n}, f, \delta, t_{n}}$ as
  $n \to \infty$.
  Since $f$ has a compact support $K_{f}$,
  and by coarea formula \eqref{coarea formula integral},
  we can write
  \begin{align}\begin{split}\nonumber
  D_{\tau, \Delta, G_{n}, f, \delta, t_{n}} =
  \frac{1}{t_{n}}
  \int_{c_{\tau} > \delta_{\tau}}
  \left(\int_{\Delta^{-1}\(c\right)}\frac{\left[
  1\(c + t_{n}G_{n}\(x\right) > \delta\right)
  - 1\(c > \delta\right)\]f\(x\right)}
  {J\Delta\(x\right)}d\mathcal{H}_{n - k}x\)d\mathcal{L}_{k}c
  \end{split}\end{align}
  for arbitrary $\delta \in\mathcal{R}_{0}$.
  Notice that for
  $x \in \{x \vert
  \Delta_{\tau}\(x\right) > \delta_{\tau}\}$,
  \begin{align}\begin{split}\nonumber
  1\left(\Delta\(x\right) +
  t_{n}\left(1_{Q_{1}}, \ldots, 1_{Q_{k}}\right)^{T} >
  \delta\right) & -
  1\left(\Delta\(x\right) > \delta\right) \\
  & =
  1\left(\Delta_{\neg\tau}\(x\right) +
  t_{n}1_{Q_{\neg\tau}} > \delta_{\neg\tau}\right) -
  1\left(\Delta_{\neg\tau}\(x\right) > \delta_{\neg\tau}\right).
  \end{split}\end{align}
  Therefore,
  \begin{align}\begin{split}\nonumber
  & D_{\tau, \Delta, \left(1_{Q_{1}}, \ldots, 1_{Q_{k}}\right)^{T},
  f, \delta, t_{n}} \\
  & =
  \frac{1}{t_{n}}
  \int_{c_{\tau} > \delta_{\tau}}
  \left(\int_{\Delta^{-1}\(c\right)}\frac{\left[
  1\(c_{\neg\tau} +
  t_{n}1_{Q_{\neg\tau}}\(x\right) > \delta_{\neg\tau}\right)
  - 1\(c_{\neg\tau} > \delta_{\neg\tau}\right)\]f\(x\right)}
  {J\Delta\(x\right)}d\mathcal{H}_{n - k}x\)d\mathcal{L}_{k}c \\
  & =
  \int_{c_{\tau} > \delta_{\tau}}
  d\mathcal{L}_{k - 1}c_{\tau}
  \left[\frac{1}{t_{n}}
  \int^{\delta_{\neg\tau}}_{\delta_{\neg\tau} - t_{n}}
  d\mathcal{L}_{1}c_{\neg\tau}
  \left(\int_{\Delta^{-1}\(c\right)}\frac{
  1\(c_{\neg\tau} +
  t_{n}1_{Q_{\neg\tau}}\(x\right) > \delta_{\neg\tau}\)f\(x\right)}
  {J\Delta\(x\right)}d\mathcal{H}_{n - k}x\right)\right].
  \end{split}\end{align}
  The second equality above comes from
  \textbf{part} 1 of the previous proof
and
  the Fubini-Tonelli theorem.
  By \textbf{step} 1 of the previous proof
  and Lemma \ref{manifold integral continuity},
  the part in the square brackets above
  converges for all $c_{\tau} > \delta_{\tau}$,
  and the limit is
  \begin{align}\begin{split}\nonumber
  \int_{\Delta^{-1}\(c\right)}
  \frac{1_{Q_{\neg\tau}}\(x\)f\(x\right)}{J\Delta\(x\right)}
  d\mathcal{H}_{n - k}x.
  \end{split}\end{align}
  Then by the boundedness of $\Delta\(E\right)$
  and \eqref{length bounded},
  dominated convergence theorem can be
  applied here to obtain
  \begin{align}\begin{split}\nonumber
  \lim_{n \to \infty}
  D_{\tau, \Delta, \left(1_{Q_{1}}, \ldots, 1_{Q_{k}}\right)^{T},
  f, \delta, t_{n}} =
  \int_{y > \delta_{\tau}}
  \left(\int_{\Delta^{-1}\(y_{\tau}\right)}
  \frac{1_{Q_{\neg\tau}}\(x\)f\(x\right)}{J\Delta\(x\right)}
  d\mathcal{H}_{n - k}x\)d\mathcal{L}_{k - 1}y.
  \end{split}\end{align}
  Then note that
  \begin{align}\begin{split}\nonumber
  \frac{F_{\Delta +
  t_{n}\left(1_{Q_{1}}, \ldots, 1_{Q_{k}}\right)^{T}, f, \delta} -
  F_{\Delta, f, \delta}}
  {t_{n}} & = \sum_{\tau}
  D_{\tau, \Delta, \left(1_{Q_{1}}, \ldots, 1_{Q_{k}}\right)^{T},
  f, \delta, t_{n}} + o\left(1\right).
  \end{split}\end{align}
  To see this,
  there is no loss of generality to assume that
  $f\(x\right) \geq 0$.
  Since
  \begin{align}\begin{split}\nonumber
  1\left(\Delta\(x\right) +
  t_{n}\left(1_{Q_{1}}\(x\right), \ldots, 1_{Q_{k}}\(x\right)\right)
  > \delta\right) -
  1\left(\Delta\(x\right) > \delta\right) \geq 0,
  \end{split}\end{align}
  the integrand of
  $F_{\Delta +
  t_{n}\left(1_{Q_{1}}, \ldots, 1_{Q_{k}}\right)^{T}, f, \delta} -
  F_{\Delta, f, \delta}$ can be non-zero
  only if $\Delta\(x\right) +
  t_{n}\left(1_{Q_{1}}\(x\right), \ldots, 1_{Q_{k}}\(x\right)\right)
  > \delta$. Therefore,
  \begin{align}\begin{split}\nonumber
  \frac{F_{\Delta +
  t_{n}\left(1_{Q_{1}}, \ldots, 1_{Q_{k}}\right)^{T}, f, \delta} -
  F_{\Delta, f, \delta}}
  {t_{n}} & = \sum_{\tau}
  D_{\tau, \Delta, \left(1_{Q_{1}}, \ldots, 1_{Q_{k}}\right)^{T},
  f, \delta - t_{n}, t_{n}},
  \end{split}\end{align}
  and clearly,
  $D_{\tau, \Delta, \left(1_{Q_{1}}, \ldots, 1_{Q_{k}}\right)^{T},
  f, \delta - \epsilon, t_{n}} \geq
  D_{\tau, \Delta, \left(1_{Q_{1}}, \ldots, 1_{Q_{k}}\right)^{T},
  f, \delta, t_{n}}.$

  For arbitrary $\epsilon > 0$,
  there exists $N$ large enough,
  such that for all $n > N$,
  \begin{align}\begin{split}\nonumber
  D_{\tau, \Delta, \left(1_{Q_{1}}, \ldots, 1_{Q_{k}}\right)^{T},
  f, \delta - \epsilon, t_{n}}
  \geq
  D_{\tau, \Delta, \left(1_{Q_{1}}, \ldots, 1_{Q_{k}}\right)^{T},
  f, \delta - t_{n}, t_{n}}.
  \end{split}\end{align}
  By the boundedness of $\Delta\(E\right)$,
  \begin{align}\begin{split}\nonumber
  \lim_{n \to \infty}
  D_{\tau, \Delta, \left(1_{Q_{1}}, \ldots, 1_{Q_{k}}\right)^{T},
  f, \delta - \epsilon, t_{n}} \downarrow
  \lim_{n \to \infty}
  D_{\tau, \Delta, \left(1_{Q_{1}}, \ldots, 1_{Q_{k}}\right)^{T},
  f, \delta, t_{n}},
  \end{split}\end{align}
  as $\epsilon \downarrow 0$, and thus
  \begin{align}\begin{split}\nonumber
  D_{\tau, \Delta, \left(1_{Q_{1}}, \ldots, 1_{Q_{k}}\right)^{T},
  f, \delta - t_{n}, t_{n}} -
  D_{\tau, \Delta, \left(1_{Q_{1}}, \ldots, 1_{Q_{k}}\right)^{T},
  f, \delta, t_{n}} = o\left(1\right).
  \end{split}\end{align}
  \item Following \textbf{step} 2
  of the previous proof
\eqref{theorem 2.2 derivative general k}
  is true for $G_{n} = G \equiv S$,
  where $S$ is a vector-valued
  simple function defined on cubes.
  \item Following
  \textbf{step} 3
  of the previous proof
\eqref{theorem 2.2 derivative general k}
  is true by an approximation of $G$.
\end{enumerate}
\end{proof}
\end{comment}

\begin{comment}
\begin{remark}
Lemma \ref{surface integral continuity}
verifies the continuity requirement
of $\Upsilon$ in subsection
\ref{sub consistency}.
In fact,
\begin{align}\begin{split}\nonumber
F_{\Delta, f, c_1} -
F_{\Delta, f, c_2} =
\int^{c_{1}}_{c_{2}}
\int_{\Delta^{-1}\left(\delta\right)}
\frac{f\(x\right)}{J\Delta\(x\right)}
d\mathcal{H}_{n - 1}xd\mathcal{L}_{1}\delta,
\end{split}\end{align}
where $c_{1}, c_{2} \in \mathcal{R}_{0}$,
see Lemma \ref{protype derivative}.
The continuity of the integrand actually
implies differentiability of
$F_{\Delta, f,c}$ (with respect to $c$).
\end{remark}
\end{comment}


\begin{comment}
Theorem \ref{derivative general k},  a multivariate
extension of Theorem
\ref{derivative k=1},
requires further assumptions to handle the additional
complexity of the derivative in higher dimensions.

\begin{theorem}\label{derivative general k}
Under Assumption
\ref{regular value assumption} and
\ref{compact assumption}.
Suppose $\Delta\(E\right)$ is bounded and
$\mathcal{R}_{0} = \Delta\(E\right) \cap
\{c \vert c > c_{0}\}
\subset \mathcal{R}$ for a constant $c_{0}$.
Suppose also
\begin{align}\begin{split}\label{length bounded}
\sup_{c \in \mathcal{R}_{0}}
\int_{\Delta^{-1}\(c\right)}
d\mathcal{H}_{n - k}x < \infty.
\end{split}\end{align}
Then, for
all $c \in \mathcal{R}_{0}$,
the map
$F_{\Delta, f, c}:
C\(E\right) \mapsto \mathbb{R}$
is Hadamard differentiable at $\Delta$
tangentially to $C_{0}\(E\right)$.
The derivative is given by
\begin{align}\begin{split}\label{theorem 2.2 derivative general k}
F'_{\Delta, f, c}\(G\right) =
\sum_{\tau}
\int_{y > c_{\tau}}
\left(\int_{\Delta^{-1}\(y_{\tau}\right)}
\frac{G_{\neg\tau}\(x\)f\(x\right)}
{J\Delta\(x\right)}
d\mathcal{H}_{n - k}x\)d\mathcal{L}_{k - 1}y
,
\end{split}\end{align}
where $c = \(c_{1}, \ldots, c_{k}\right)^{T}$,
$G(x) = \(G_{1}\(x\right), \ldots, G_{k}\(x\right)\right)^{T}$,
$\sum_{\tau}$ denotes the summation over the
$k$ combinations of $k - 1$ distinct elements
from $\{1, \ldots, k\}$;
$c_{\tau}$ denotes the
concatenation of $k - 1$
corresponding dimensions of $c$
according to the ascending order of
subscript;
$\neg\tau$ denotes
the only element unpicked of each combination,
and $y_{\tau}$ denotes the value obtained by
replacing the picked $k - 1$ dimensions of
$c$ with $y$.
\end{theorem}
\end{comment}











\section{Binary treatment allocation and the ROC curve}\label{constrained and roc}

In this section, we derive uniform asymptotic distributions
for a binary treatment allocation value function under a resource constraint.
Using Hadamard differentiability results
from section \ref{sorting operator}, we directly and concisely analyze a plug-in
two-step estimator of the value function.
The functional delta methods alone (see for example
\cite{van2023weak} and \cite{fang2019inference}) are sufficient to obtain asymptotic results.
An important result of this section
is showing
consistency
of a computationally feasible bootstrap procedure for the plug-in
two-step ROC estimator commonly used in computer science and other fields.

\subsection{Binary constrained allocation}\label{binary constrained case}

In population, binary optimal constrained treatment allocation takes the form of
\begin{align}\begin{split}\label{constrained optimized value function}
\max_{\phi \in \Phi} \left\{\mathbb{E} \[Y_{1}\phi\(X\right) + Y_{0}\left(1 - \phi\(X\right)\right)\right]:
\mathbb{E} \[Z_{1}\phi\(X\right) + Z_{0}\left(1 - \phi\(X\right)\right)\right] \leq \alpha\right\}
\end{split}\end{align}
where $\Phi$ is the set of critical functions with $J = 1$.
By the
Neyman-Pearson lemma (see for example Theorem 3.2.1 in \cite{lehmann2022testing}),
solutions of \eqref{constrained optimized value function} satisfy
\begin{align}\begin{split}\nonumber
\phi\(x\right) = \left\{
\begin{array}{cc}
1, & \text{when}\ \(g_{1}\(x\right) - g_{0}\(x\right)\right) > k\(c_{1}\(x\right) - c_{0}\(x\right)\right), \\
0, & \text{when}\ \(g_{1}\(x\right) - g_{0}\(x\right)\right) < k\(c_{1}\(x\right) - c_{0}\(x\right)\right),
\end{array}
\right.
\end{split}\end{align}
where $k$ is a constant to be determined, $g_{j}\(x\right) = \mathbb{E}\(Y_{j}\vert X = x\right)$,
$c_{j}\(x\right) = \mathbb{E}\(Z_{j}\vert X = x\right)$, $j=0,1$.

The constrained population programming in \eqref{constrained optimized value function} uses observed covariates to allocate a binary treatment among a population when a budget constraint limits the size of the treated group.
In this context, $Y_{1}$ and $Y_{0}$ are the welfare measures with and without treatment; $Z_{1}$ and $Z_{0}$ are the respective costs of treatment and control.
For a typical economic problem, the cost of treatment is usually no less than the cost of control.
For example, \cite{bhattacharya2012inferring} study the efficient provision of anti-malaria bed net subsidies using a set
of features $X$ that include the presence of a child under 10, the wealth
per capita and the ownership of a bank account.
Here, $Y_{1}$ and $Y_{0}$ are indexed by whether a household is covered by an insecticide-treated net; $Z_{1}$ is the cost of a redeemed net coupon and $Z_{0}$ is taken to be zero.
In \cite{elliott2013predicting}, $\(Y_1, Y_0, Z_1, Z_0\right)$ represent the utility of a true positive, true negative, false positive and false negative, respectively, in a classification problem.
In \cite{luedtke2016optimal},  the resource constraint takes the form of an upper bound on the fraction of the treated population. Their setting is equivalent to $Z_1=1$ and $Z_0=0$.
In this paper, we therefore assume $Z_{1} \geq Z_{0}$.
This section first considers full observability of $\(Y_1, Y_0, Z_1, Z_0, X\right)$.

\begin{assump}\label{variable assumption}
The random variables $\(Y_{1}, Y_{0}, Z_{1}, Z_{0}, X\right) \sim Q$, $X \sim \mu$, where $\mu$ is absolutely continuous with respect to the $\dim\(X\right)$-dimensional Lebesgue measure.
The density $\mu'$ is continuous
on an open set $E \subset \mathbb{R}^{\dim\(X\right)}$ and $\supp{\mu'} \coloneqq K_{\mu'} \subset E$.
$\(Y_{1}, Y_{0}, Z_{1}, Z_{0}\right)$ is bounded and $Z_{1} \geq Z_{0}$.
\end{assump}

Following empirical process notation conventions, let $Qf = \int fdQ$. We then
express the value and constrain functions of
\eqref{constrained optimized value function}  as
\begin{align}\begin{split}\label{functional form binary constrained allocation}
\beta\(k\right) & = Q\left[\(y_{1} - y_{0}\right)1\left(\(g_{1}\(x\right) - g_{0}\(x\right)\right) > k\(c_{1}\(x\right) - c_{0}\(x\right)\right)\right) + y_{0}\right],\\
s.t. \;
\alpha\(k\right) & = Q\left[\(z_{1} - z_{0}\right)1\left(\(g_{1}\(x\right) - g_{0}\(x\right)\right) > k\(c_{1}\(x\right) - c_{0}\(x\right)\right)\right) + z_{0}\right] \leq \alpha,
\end{split}\end{align}
where level set $\left\{x: \(g_{1}\(x\right) - g_{0}\(x\right)\right) = k\(c_{1}\(x\right) - c_{0}\(x\right)\right)\right\}$ is of zero probability mass.
The Hadamard differentiability results
in section \ref{sorting operator} allow us to derive the
asymptotic distribution of an estimator for
\eqref{functional form binary constrained allocation} by the functional delta method under the following assumptions.
We largely adopt the styles in \cite{chernozhukov2018sorted}.


\begin{assump}\label{regular Delta assumption}
The
limits
of $\left(\hat{g}_{j}\left(\cdot\right), \hat{c}_{j}\left(\cdot\right), j=0,1\right)$
satisfy
$\(g^{\star}_{j}\left(\cdot\right), c^{\star}_{j}\left(\cdot\right), j=0,1\right) \in {C^{1}\(E,\mathbb{R}^4\right)}$, $c^{\star}_{1}\left(\cdot\right) > c^{\star}_{0}\left(\cdot\right)$, and $0$ is a regular value of
$\Delta^{\star}\left(\cdot; k\right) \coloneqq g^{\star}_{1}\left(\cdot\right) - g^{\star}_{0}\left(\cdot\right) - k\(c^{\star}_{1}\left(\cdot\right) - c^{\star}_{0}\left(\cdot\right)\right)$
for all $k \in \mathcal{D}$ where $\mathcal{D} \subset \mathbb{R}$ is a bounded closed interval.
\end{assump}


\begin{assump}\label{joint convergence assumption}
Let $\(g^{\star}_{j}\left(\cdot\right), c^{\star}_{j}\left(\cdot\right), j=0,1\right) \in \mathcal{G}$
where $\mathcal{G} \subset {C\(E, \mathbb{R}^{4}\right)}$ is uniformly bounded.
With outer probability tends to $1$, $\left(\hat{g}_{j}\left(\cdot\right), \hat{c}_{j}\left(\cdot\right), j=0,1\right) \in \mathcal{G}$.
The function class
\begin{align}\begin{split}\nonumber
\mathcal{F} \coloneqq \left\{1\left(\(g'_{1}\left(\cdot\right) - g'_{0}\left(\cdot\right)\right) > k\(c'_{1}\left(\cdot\right) - c'_{0}\left(\cdot\right)\right)\right):
\(g'_{0}\left(\cdot\right), g'_{1}\left(\cdot\right), c'_{0}\left(\cdot\right), c'_{1}\left(\cdot\right)\right) \in \mathcal{G},
k \in \mathcal{D} \right\}
\end{split}\end{align}
is Donsker. The estimators and the empirical process converge jointly, i.e.
\begin{align}\begin{split}\nonumber
r_{n}\left[
\left(\begin{array}{c}
\hat{g}_{j}\left(\cdot\right)_{j=0,1} \\
\hat{c}_{j}\left(\cdot\right)_{j=0,1}
\end{array}
\right)^T -
\left(\begin{array}{c}
{g}_{j}^\star\left(\cdot\right)_{j=0,1} \\
{c}_{j}^\star\left(\cdot\right)_{j=0,1}
\end{array}
\right)^T, \mathbb{Q}_{n} - Q\right] \rightsquigarrow
\left(\mathbb{H}^T, t_{Q}\mathbb{Q}\right) \text{ in } {C\(E, \mathbb{R}^{4}\right)} \times \ell^{\infty}\left(\mathcal{F}\right),
\end{split}\end{align}
\begin{comment}
\begin{align}\begin{split}\nonumber
r_{n}\left[
\left(\begin{array}{c}
\hat{g}_{0}\(x\right) \\
\hat{g}_{1}\(x\right) \\
\hat{c}_{0}\(x\right) \\
\hat{c}_{1}\(x\right)
\end{array}
\right) -
\left(\begin{array}{c}
{g}_{0}^\star\(x\right) \\
{g}_{1}^\star\(x\right) \\
{c}_{0}^\star\(x\right) \\
{c}_{1}^\star\(x\right)
\end{array}
\right), \mathbb{Q}_{n} - Q\right] \rightsquigarrow
\left(\mathbb{H}, t_{Q}\mathbb{Q}\right) \text{ in } {\ell\(E\right)}^{4} \times \ell^{\infty}\left(\mathcal{F}\right),
\end{split}\end{align}
\end{comment}
where $r_{n} \to \infty$ as $n \to \infty$,
$\lim_{n \to \infty}\frac{r_{n}}{\sqrt{n}} = t_{Q} \in \left[0, 1\right]$ and
$\mathbb{H} = \left(\mathbb{H}_{g_0}, \mathbb{H}_{g_1}, \mathbb{H}_{c_0}, \mathbb{H}_{c_1}\right)^{T}$
is separable.
\end{assump}

\begin{theorem}\label{binary allocation asymptotic result}
Let Assumptions \ref{variable assumption} to \ref{joint convergence assumption} hold.
There exists a constant $\epsilon > 0$ such that
\begin{align}\begin{split}\label{nonsingular manifold jacobian denominator in binary classification roc curve}
\inf_{k \in \mathcal{D}}
\int_{\Delta^{\star}\(x;k\right)=0} \frac{\(c_{1}\(x\right) -
c_{0}\(x\right)\right)\(c^{\star}_{1}\(x\right) -
c^{\star}_{0}\(x\right)\right)\mu'\(x\right)}{\Vert
\nabla\Delta^{\star}\(x;k\right)\Vert}d\mathcal{H}_{n - 1}x >
\epsilon.
\end{split}\end{align}
Let $\Delta$ ($\hat{\Delta}, \Delta^{\star}$) to denote
$\(g_{j}\left(\cdot\right), c_{j}\left(\cdot\right),j=0,1\right)$
(their estimators and the estimators' limits).	Also
let $k\(Q, \Delta, \alpha\right)  = \inf\left\{k \in \mathbb{R}:
\alpha\(Q, \Delta, k\right) \leq \alpha \right\}$ where
\begin{align}\begin{split}\label{double robust representing of constrained optimized value function under full observability}
&\beta\(Q, \Delta, \alpha\right) \coloneqq \beta\(Q, \Delta, k\(Q, \Delta, \alpha\right)\right);\
\beta\(Q, \Delta, k\right) \coloneqq Q\left[\(y_{1} - y_{0}\right)1\left(\Delta\(x; k\right) > 0\right)+y_0\right],\\
&\alpha\(Q, \Delta, k\right) \coloneqq
Q\left[\(z_{1} - z_{0}\right)1\left(\Delta\(x; k\right) > 0\right)+z_0\right].
\end{split}\end{align}
If we also define
\begin{align}\begin{split}\nonumber
f_{\beta}\(Q, \Delta^{\star}, k\right) & \coloneqq
\frac{d\beta\(Q, \Delta^{\star}, k\right)}{dk} = -\int_{\Delta^{\star}\(x; k\right) = 0}
\frac{\(g_{1}\(x\right) - g_{0}\(x\right)\right)\(c^{\star}_{1}\(x\right) - c^{\star}_{0}\(x\right)\right)\mu'\(x\right)}{\Vert\nabla\Delta^{\star}\(x; k\right)\Vert}d\mathcal{H}_{n - 1}x, \\
f_{\alpha}\(Q, \Delta^{\star}, k\right) & \coloneqq
\frac{d\alpha\(Q, \Delta^{\star}, k\right)}{dk} = -\int_{\Delta^{\star}\(x; k\right) = 0}
\frac{\(c_{1}\(x\right) - c_{0}\(x\right)\right)\(c^{\star}_{1}\(x\right) -
c^{\star}_{0}\(x\right)\right)\mu'\(x\right)}{\Vert\nabla\Delta^{\star}\(x;
k\right)\Vert}d\mathcal{H}_{n - 1}x,
\end{split}\end{align}
then as a process on compact $\Lambda'
\subsetneq \overset{\circ}{\Lambda}$ where
$\Lambda \coloneqq \biggl\{Q\left[\(z_{1} -
z_{0}\right)1\left(\Delta^{\star}\(x; k\right) > 0\right) + z_0\right], k\in \mathcal D
\biggr\},$
\begin{align}
& r_{n}\left(\beta\left(\mathbb{Q}_{n}, \hat{\Delta}, \alpha\right) - \beta\(Q, \Delta^{\star}, \alpha\right)\right) \label{main asymptotic formula}\\
&\rightsquigarrow
t_{Q}\mathbb{Q}\biggl[\(y_{1} - y_{0} -\lambda\(Q, \Delta^\star, \alpha\right) \(z_1 - z_0\right)\right)1\left(\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right) > 0\right)
+ y_{0} - \lambda\(Q, \Delta^\star, \alpha\right) z_0 \biggr] \notag \\
&
+
\int_{\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right) = 0}
\frac{\Delta\(x; \lambda\(Q, \Delta^{\star}, \alpha\right)\right)\left(\mathbb{H}_{g1} - \mathbb{H}_{g0}
-
k\(Q, \Delta^{\star}, \alpha\right)
\left(
\mathbb{H}_{c1} - \mathbb{H}_{c0}
\right)
\right)\mu'\(x\right)}{\Vert\nabla\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right)\Vert}
\notag
\end{align}
In the above, $\lambda\(Q, \Delta^\star, \alpha\right) = \frac{f_{\beta}\(Q, \Delta^{\star}, k\(Q, \Delta^{\star}, \alpha\right)\right)}{f_{\alpha}\(Q, \Delta^{\star}, k\(Q, \Delta^{\star}, \alpha\right)\right)}$ and $\Delta\(x; \lambda\right) = g_{1}\(x\right) - g_{0}\(x\right) - \lambda\(c_1\(x\right) - c_0\(x\right)\right)$.
\begin{comment}
\begin{align}\begin{split}\label{main asymptotic formula}
& r_{n}\left(\beta\left(\mathbb{Q}_{n}, \hat{\Delta}, \alpha\right) - \beta\(Q, \Delta^{\star}, \alpha\right)\right) \rightsquigarrow \Biggl\{
t_{Q}\mathbb{Q}\left[\(y_{1} - y_{0}\right)1\left(\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right) > 0\right) + y_{0}\right] \\
& +  \Biggl[
\int_{\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right) = 0}\frac{\(g_{1}\(x\right) - g_{0}\(x\right)\right)\left(\mathbb{H}_{1} - \mathbb{H}_{0}\right)\mu'\(x\right)}{\Vert\nabla\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right)\Vert}d\mathcal{H}_{n - 1}x \\
& \quad\quad\quad\quad
-\int_{\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right) = 0}\frac{
k\(Q, \Delta^{\star}, \alpha\right)
\(g_{1}\(x\right) - g_{0}\(x\right)\right)\left(\mathbb{H}_{3} - \mathbb{H}_{2}\right)\mu'\(x\right)}{\Vert\nabla\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right)\Vert}d\mathcal{H}_{n - 1}x
\Biggl]
\Biggl\} \\
& - \frac{f_{\beta}\(Q, \Delta^{\star}, k\(Q, \Delta^{\star}, \alpha\right)\right)}{f_{\alpha}\(Q, \Delta^{\star}, k\(Q, \Delta^{\star}, \alpha\right)\right)}
\Biggl\{
t_{Q}\mathbb{Q}\left[
\(z_{1} - z_{0}\right)1\left(\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right) > 0\right)\right) + z_{0}\right] \\
& +  \Biggl[
\int_{\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right) = 0}\frac{\(c_{1}\(x\right) - c_{0}\(x\right)\right)\left(\mathbb{H}_{1} - \mathbb{H}_{0}\right)\mu'\(x\right)}{\Vert\nabla\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right)\Vert}d\mathcal{H}_{n - 1}x \\
& \quad\quad\quad\quad
- \int_{\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right) = 0}\frac{
k\(Q, \Delta^{\star}, \alpha\right)
\(c_{1}\(x\right) - c_{0}\(x\right)\right)\left(\mathbb{H}_{3} - \mathbb{H}_{2}\right)\mu'\(x\right)}{\Vert\nabla\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right)\Vert}d\mathcal{H}_{n - 1}x
\Biggl]
\Biggl\}.
\end{split}\end{align}
\end{comment}
\begin{comment}
\begin{align}\begin{split}\nonumber
\Lambda \coloneqq \biggl\{Q\left[\(z_{1} - z_{0}\right)1\left(\Delta\(x; k\right) > 0\right)\right], k\in \mathcal D \biggr\},
\end{split}\end{align}
\begin{align}\begin{split}\label{main asymptotic formula}
& t_{n}\left(\beta\left(\mathbb{Q}_{n}, \hat{\Delta}, \alpha\right) - \beta\(Q, \Delta^{\star}, \alpha\right)\right) \rightsquigarrow \\
&
\Biggl\{
t_{Q}\mathbb{Q}\biggl[\(y_{1} - y_{0}\right)1\left(\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right) > 0\right) + y_{0}\\
&\hspace{1in}-\lambda\(Q, \Delta^*, k\(Q, \Delta^{\star}, \alpha\right)\right)\left(
\(z_{1} - z_{0}\right)1\left(\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right) > 0\right)\right) + z_{0}\right)
\biggr] \\
& + t_{\Delta} \Biggl[
\int_{\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right) = 0}\frac{\(g_{1}\(x\right) - g_{0}\(x\right)\right)\left(\mathbb{H}_{1} - \mathbb{H}_{0}\right)\mu'\(x\right)}{\Vert\nabla\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right)\Vert}d\mathcal{H}_{n - 1}x \\
& \quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad - \int_{\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right) = 0}\frac{\(g_{1}\(x\right) - g_{0}\(x\right)\right)\left(\mathbb{H}_{3} - \mathbb{H}_{2}\right)\mu'\(x\right)}{\Vert\nabla\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right)\Vert}d\mathcal{H}_{n - 1}x
\Biggl]
\Biggl\} \\
& + t_{\Delta} \Biggl[
\int_{\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right) = 0}\frac{\(c_{1}\(x\right) - c_{0}\(x\right)\right)\left(\mathbb{H}_{1} - \mathbb{H}_{0}\right)\mu'\(x\right)}{\Vert\nabla\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right)\Vert}d\mathcal{H}_{n - 1}x \\
& \quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad - \int_{\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right) = 0}\frac{\(c_{1}\(x\right) - c_{0}\(x\right)\right)\left(\mathbb{H}_{3} - \mathbb{H}_{2}\right)\mu'\(x\right)}{\Vert\nabla\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right)\Vert}d\mathcal{H}_{n - 1}x
\Biggl]
\Biggl\},
\end{split}\end{align}
\end{comment}
\end{theorem}
Theorem \ref{binary allocation asymptotic result}
simplifies under correct specification when $\Delta^{\star}\left(\cdot\right) = \Delta\left(\cdot\right)$.
In this case,
$k = \frac{f_{\beta}\(Q, \Delta^{\star}, k\right)}{f_{\alpha}\(Q,
\Delta^{\star},k\right)}$,
since on $\Delta^{\star}\(x; k\right) = 0$,
$\Delta\(x; k\right) =
\(g_{1}\(x\right) - g_{0}\(x\right)
-
k
\(c_{1}\(x\right) - c_{0}\(x\right)\right)
\right)
=0$,
\begin{align}\begin{split}\nonumber
0 & = f_{\beta}\(Q, \Delta^{\star}, k\right)
-k f_{\alpha}\(Q, \Delta^{\star}, k\right) \\
& =
-\int_{\Delta^{\star}\(x; k\right) = 0}
\frac{\(g_{1}\(x\right) - g_{0}\(x\right)
-
k
\(c_{1}\(x\right) - c_{0}\(x\right)\right)
\right)\(c^{\star}_{1}\(x\right) - c^{\star}_{0}\(x\right)\right)\mu'\(x\right)}{\Vert\nabla\Delta^{\star}\(x; k\right)\Vert}d\mathcal{H}_{n - 1}x.
\end{split}\end{align}
Then the asymptotic distribution in
\eqref{main asymptotic formula}
simplifies to a Gaussian process in $\alpha$:
\begin{align}\begin{split}\label{main asymptotic formula under correct specification}
r_{n}\left(\beta\left(\mathbb{Q}_{n}, \hat{\Delta}, \alpha\right) - \beta\(Q, \Delta^{\star}, \alpha\right)\right) \rightsquigarrow
t_{Q}\mathbb{Q} & \biggl[\(y_{1} - y_{0}\right)1\left(\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right) > 0\right) + y_{0} \\
-k\(Q, \Delta^{\star}, \alpha\right) & \left(
\(z_{1} - z_{0}\right)1\left(\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right) > 0\right)\right) + z_{0}\right)\biggr].
\end{split}\end{align}
In parametric models where $\Delta\(x\right) = \Delta\(x; \theta\right)$, $r_n = \sqrt{n}$ and $t_Q = 1$.
The covariance kernel for
\eqref{main asymptotic formula under correct specification}
over $\left(\alpha, \alpha'\right)$
can be consistently estimated by its sample analog.
Appendix \ref{supplementary discussion on misspecified parametric models} analyzes the implications
of parametric model misspecification.

\begin{comment}
Theorem \ref{binary allocation asymptotic result}  covers three situations: a correctly specified parametric model, a nonparametric model which by definition is correctly specified, and a misspecified parametric model.
A model of $\Delta^{\star}\left(\cdot\right)$ is correctly specified when $\Delta^{\star}\left(\cdot\right) = \Delta\left(\cdot\right)$, and is misspecified when
$\Delta^{\star}\left(\cdot\right) \neq \Delta\left(\cdot\right)$. Correct specification applies to when $\hat \Delta$ is estimated nonparametrically or
using machine learning methods. It also results from a parametric model of $\hat\Delta\left(\cdot; \hat\theta\right)$ such that
$\hat\theta \overset{p}{\rightarrow} \theta^*$ and $\Delta^{\star}\left(\cdot; \theta^*\right) = \Delta\left(\cdot\right)$. Misspecified models
are typically parametric, such that $\Delta^{\star}\left(\cdot\right) =
 \Delta^{\star}\left(\cdot; \theta^*\right) \neq \Delta\left(\cdot\right)$.

\paragraph{Under correct specification}
$\Delta^{\star}\left(\cdot\right) = \Delta\left(\cdot\right)$,
$k=\frac{f_{\beta}\(Q, \Delta^{\star}, k\right)}{f_{\alpha}\(Q,
\Delta^{\star},k\right)}$,
since on $\Delta^{\star}\(x; k\right) = 0$,
\begin{align}\begin{split}\nonumber
&g_{1}\(x\right) - g_{0}\(x\right)
-
k
\(c_{1}\(x\right) - c_{0}\(x\right)\right)
=
g_{1}^*\(x\right) - g_{0}^*\(x\right)
-
k
\(c_{1}^*\(x\right) - c_{0}^*\(x\right)\right) = 0,\\
0=&f_{\beta}\(Q, \Delta^{\star}, k\right)
-k f_{\alpha}\(Q, \Delta^{\star}, k\right)\\
=&
-\int_{\Delta^{\star}\(x; k\right) = 0}
\frac{\(g_{1}\(x\right) - g_{0}\(x\right)
-
k
\(c_{1}\(x\right) - c_{0}\(x\right)\right)
\right)\(c^{\star}_{1}\(x\right) - c^{\star}_{0}\(x\right)\right)\mu'\(x\right)}{\Vert\nabla\Delta^{\star}\(x; k\right)\Vert}d\mathcal{H}_{n - 1}x.
\end{split}\end{align}
The asymptotic distribution in Theorem \ref{binary allocation asymptotic result} simplifies under correct specification to
\begin{align}\begin{split}\label{main asymptotic formula under correct specification}
& r_{n}\left(\beta\left(\mathbb{Q}_{n}, \hat{\Delta}, \alpha\right) - \beta\(Q, \Delta^{\star}, \alpha\right)\right) \rightsquigarrow
t_{Q}\mathbb{Q}\biggl[\(y_{1} - y_{0}\right)1\left(\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right) > 0\right) + y_{0} \\
&\hspace{1.5in}-k\(Q, \Delta^{\star}, \alpha\right) \left(
\(z_{1} - z_{0}\right)1\left(\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right) > 0\right)\right) + z_{0}\right)\biggr].
\end{split}\end{align}
Parametric models of $\Delta^{\star}\left(\cdot\right)$ correspond to when $r_n = \sqrt{n}$ and $t_Q = 1$. Then
\eqref{main asymptotic formula under correct specification} represents a Gaussian process over $\alpha$ with its covariance kernel given by
\end{comment}
\begin{comment}
\begin{align}\begin{split}\nonumber
&r_{n}\left(\beta\left(\mathbb{Q}_{n}, \hat{\Delta}, \alpha\right) - \beta\(Q, \Delta^{\star}, \alpha\right)\right) \rightsquigarrow N\left(0, \sigma^2\right)
\quad\text{where}\quad\\
&\sigma^2 = Var\biggl(
\(y_{1} - y_{0}\right)1\left(\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right) > 0\right) + y_{0} \\
&\hspace{1in} - k\(Q, \Delta^{\star}, \alpha\right)
\(z_{1} - z_{0}\right)1\left(\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right) > 0\right)\right) + z_{0}
\biggr),
\end{split}\end{align}
\end{comment}
\begin{comment}
\begin{align}\begin{split}\nonumber
\left(\alpha, \alpha'\right) \mapsto
&Cov\biggl(
\(y_{1} - y_{0}\right)1\left(\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right) > 0\right) + y_{0} \\
&\hspace{.6in} - k\(Q, \Delta^{\star}, \alpha\right)
\left(\(z_{1} - z_{0}\right)1\left(\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right) > 0\right)\right) + z_{0}\right),\\
&\hspace{0.6in}\(y_{1} - y_{0}\right)1\left(\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha'\right)\right) > 0\right) + y_{0} \\
&\hspace{1.5in} - k\(Q, \Delta^{\star}, \alpha'\right)
\left(\(z_{1} - z_{0}\right)1\left(\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha'\right) > 0\right)\right) + z_{0}\right)
\biggr),
\end{split}\end{align}
which can be consistently estimated by its sample analog.
Consistent inference
can also be done by applying multiplier bootstrap to the estimated score function
\begin{align}\begin{split}\nonumber
\(y_{1i} - y_{0i}\right)1\left(\hat\Delta\(x_i; \hat k\left(\alpha\right)\right) > 0\right) + y_{0i}  -
\hat k\left(\alpha\right) \(z_{1i} - z_{0i}\right)1\left(\hat\Delta\(x_i; \hat k\left(\alpha\right) > 0\right)\right) + z_{0i},
\end{split}\end{align}
where $\hat k\left(\alpha\right) = \inf\left\{k \in \mathbb{R}: \alpha\left(\mathbb Q_n, \hat\Delta, k\right) \leq \alpha \right\}$.
\end{comment}

An alternative model-based representation of
\eqref{constrained optimized value function}
is
\begin{align}\begin{split}\label{model based constrained optimized value function}
\max_{\phi \in \Phi} \left\{\mathbb{E} \[g_{1}\(X\right) \phi\(X\right) + g_{0}\(X\right)\left(1 - \phi\(X\right)\right)\right]: \mathbb{E} \[c_{1}\(X\right)\phi\(X\right) + c_{0}\(X\right)\left(1 - \phi\(X\right)\right)\right] \leq \alpha\right\}.
\end{split}\end{align}
Corresponding to \eqref{model based constrained optimized value function},
the measure $Q$ for $\(Y_{1}, Y_{0}, Z_{1}, Z_{0}, X\right)$ in the functional expression of \eqref{double robust representing of constrained optimized value function under full observability} can be changed to the measure $\mu$ for $X$:
\begin{align}\begin{split}\nonumber
\alpha\left(\mu, \Delta, k\right) & \coloneqq
\mu\left[\(c_{1}\(x\right) - c_{0}\(x\right)\right)1\left(\Delta\(x; k\right) >
0\right)+c_0\(x\right)\right], \\
k\left(\mu, \Delta, \alpha\right) & \coloneqq   \inf\left\{k \in
\mathbb{R}: \alpha\left(\mu, \Delta, k\right) \leq \alpha \right\},
\\
\beta\left(\mu, \Delta, \alpha\right) & \coloneqq \beta\left(\mu, \Delta, k\left(\mu, \Delta, \alpha\right)\right),
\beta\left(\mu, \Delta, k\right)  \coloneqq
\mu\left[\(g_{1}\(x\right) - g_{0}\(x\right)\right)1\left(\Delta\(x; k\right) > 0\right)+g_0\(x\right)\right].
\end{split}\end{align}
When the conditional data generating process of $y_1, y_0, z_1, z_0$ given $x$ is a
parametric model
$f\(y_1, y_0, z_1, z_0 \vert x; \theta\right)$, we denote $\hat \Delta\(x\right) = \Delta\(x; \hat\theta\right)$.
A plug-in estimator of
\eqref{model based constrained optimized value function} is then
\begin{align}\begin{split}\label{estimate of model based representation of constrained optimized value function under full observability}
\alpha\left(\hat{\mu}, \hat{\theta}, k\right) & \coloneqq \alpha\left(\hat{\mu}, \hat{\Delta}, k\right),
\beta\left(\hat{\mu}, \hat{\theta}, k\right)  \coloneqq \beta\left(\hat{\mu}, \hat{\Delta}, k\right),
\\
\beta\left(\hat\mu, \hat\theta, \alpha\right) & \coloneqq \beta\left(\hat\mu, \hat\Delta, k\left(\hat\mu, \hat\theta, \alpha\right)\right), \;
k\left(\hat\mu, \hat\theta, \alpha\right)  \coloneqq \inf\left\{k \in \mathbb{R}: \alpha\left(\hat\mu, \hat\theta, k\right) \leq \alpha \right\}.
\end{split}\end{align}
We will call $\beta\left(\mathbb{Q}_{n}, \hat{\Delta}, \alpha\right)$ in \eqref{main asymptotic formula}
an \textit{orthogonal} estimator since its first order derivative cancels out under correct specification,
and $\beta\left(\hat\mu, \hat\theta, \alpha\right)$ in \eqref{estimate of model based representation of constrained optimized value function under full observability} a parametric model-based estimator.

Typically, $\hat\theta$ is the maximum likelihood estimator:
$\hat\theta = \mathop{\arg\max}\limits_{\theta\in\Theta} \sum_{i=1}^n \log f\(y_{i1}, y_{i0}, z_{i1}, z_{i0} \vert x; \theta\right).$
Let $\omega = \{y_1, y_0, z_1, z_0\}$ and denote the score function $s_{\theta}\left(\omega \vert x; \theta\right) =
\frac{\partial}{\partial \theta} \log f\left(\omega \vert x, \theta\right)$.
It is a standard result from maximum likelihood estimation that
\begin{align}\begin{split}\label{mle influence function}
\sqrt{n}\left(\hat\theta - \theta^{\star}\right) = - H\left(\theta^{\star}\right)^{-1} \frac{1}{\sqrt{n}} \sum_{i=1}^n s_{\theta}\left(\omega_i \vert x_i; \theta^{\star}\right)
+ o_{\mathbb{P}}\left(1\right),\ H\left(\theta\right) = Q\left[\frac{\partial^2}{\partial \theta^2} \log f\left(\omega \vert x, \theta\right)\right].
\end{split}\end{align}
See for example \cite{newey_mcfadden} section 3 or \cite{van2000asymptotic} chapter 5 for sufficient regularity conditions
for the asymptotic linear representation.

Under correct specification, the parametric model-based estimator has an efficiency advantage over the orthogonal estimator.
We have the following efficiency comparison proposition.

\begin{proposition} \label{parametric model based estimator more efficient than orthogonal estimator}
Assume that \eqref{mle influence function} holds.  Under the conditions in Theorem \ref{binary allocation asymptotic result},
when the parametric model is correctly specified,
the model based estimator $\beta\left(\hat\mu, \hat{\theta}, \alpha\right)$ in
\eqref{estimate of model based representation of constrained optimized value function under full observability}
is asymptotically more efficient than
the orthogonal estimator in the parametric case of
\eqref{main asymptotic formula under correct specification}.
\end{proposition}



The population value function $\beta\left(\mu, \Delta, \alpha\right)$ of the constrained optimization problem in \eqref{constrained optimized value function}
satisfies concavity
and monotonicity shape restrictions in the arguments as in the general framework developed by \cite{chen2021shape}.
The orthogonal estimator $\beta\left(\mathbb{Q}_{n}, \hat{\Delta}, \alpha\right)$ is only monotonically increasing but not necessarily concave.
When shape restrictions like concavity and monotonicity are applicable,
shape-enforcing operators proposed by
\cite{chen2021shape}
can be used to improve estimation and inference \textit{ex post}. In particular,
\cite{chen2021shape} demonstrate that an estimator transformed by the shape-enforcing operator is more accurate, and
confidence intervals transformed by the shape-enforcing operator have greater coverage and shorter length.

The double Legendre-Fenchel transform and the ``pooled adjacent violators algorithm'' as described in \cite{chen2021shape}
can be used to enforce concavity of the orthogonal estimator $\beta\left(\mathbb{Q}_{n}, \hat{\Delta}, \alpha\right)$.
An alternative approach of concavification is via the Lagrangian dual.
In particular, the model based estimator $\beta\left(\hat\mu, \hat{\theta}, \alpha\right)$ is equivalent to the Lagrangian dual
\begin{align}\begin{split}\nonumber
\mathcal{L}\left(\hat{\mu}, \hat{\theta}, \alpha\right) = \min_{k \geq 0}\hat{\mu}\left[\left(\hat{g}_{1}\(x\right) - \hat{g}_{0}\(x\right)\right)1\left(\hat{\Delta}\(x; k\right) > 0\right) - k\left(\left(\hat{c}_{1}\(x\right) - \hat{c}_{0}\(x\right)\right)1\left(\hat{\Delta}\(x; k\right) > 0\right) - \alpha\right)\right],
\end{split}\end{align}
where $\hat{g}_{j}\left(\cdot\right) = g_{j}\left(\cdot, \hat{\theta}\right)$, $\hat{c}_{j}\left(\cdot\right) = c_{j}\left(\cdot, \hat{\theta}\right)$ and $j = 0, 1$.
Since the Lagrangian dual minimization problem is affine with respect to $\alpha$, $\beta\left(\hat\mu, \hat{\theta}, \alpha\right)$
can be considered an implicit concavification of $\beta\left(\mathbb{Q}_{n}, \hat{\Delta}, \alpha\right)$.
Both the double Legendre-Fenchel transform and the Lagrangian dual $\mathcal{L}\left(\hat{\mu}, \hat{\theta}, \alpha\right)$ can be generalized to the problem of multi-constraints.

Under misspecification, the asymptotic distribution of the parametric orthogonal estimator
$\beta\left(\mathbb{Q}_{n}, \hat{\theta}, \alpha\right)$
is described in Appendix \ref{supplementary discussion on misspecified parametric models},
where the integrals with respect to Hausdorff measure in \eqref{main asymptotic formula} do not vanish in general.

In causal inference,
$Y_j, Z_j, j \in \{0,1\}$ are not fully observed.
Only $Y=D Y_1 + \left(1-D\)Y_0$ and $Z=D Z_1 + \left(1-D\)Z_0$ are observed,
where $D \in \{0,1\}$ and unconfoundedness holds:
\begin{align}\begin{split}\label{binary unconfoundedness assumption}
\(Y_1, Y_0, Z_{1}, Z_{0}\right) \perp\!\!\!\perp D \vert X.
\end{split}\end{align}
The unconfoundedness assumption \eqref{binary unconfoundedness assumption} allows the model based constrained value function
\eqref{model based constrained optimized value function}
to be identified using
$g_1\(x\right) = \mathbb{E}\(Y \vert D=1, x\right)$,
$g_0\(x\right) = \mathbb{E}\(Y \vert D=0, x\right)$,
$c_1\(x\right) = \mathbb{E}\(Z \vert D=1, x\right)$,
$c_0\(x\right) = \mathbb{E}\(Z \vert D=0, x\right)$.
When the conditional data generating processes are parametric models of $f\(y, z \vert x, d; \theta\right)$, denoted as
\begin{align}\begin{split}\label{joint conditional outcome and cost distribution 1}
f_1\(y, z \vert x; \theta\right) \coloneqq
f\(y, z \vert d=1, x; \theta\right),\quad  f_0\(y, z \vert x; \theta\right) \coloneqq f\(y, z \vert d=0, x; \theta\right),
\end{split}\end{align}
a parametric model based estimator is given in
\eqref{estimate of model based representation of constrained optimized value function under full observability}
such that $f\(y_1 \vert x; \theta\right)$, $f\(z_1 \vert x; \theta\right)$, $f\(y_0 \vert x; \theta\right)$
and $f\(z_0 \vert x; \theta\right)$ are the marginal densities implied by
\eqref{joint conditional outcome and cost distribution 1},
and such that $\hat\theta$ is the conditional MLE:
\begin{align}\begin{split}\nonumber
\hat\theta = \mathop{\arg\max}\limits_{\theta\in\Theta} \sum_{i=1}^n
\log f\(y_{i}, z_{i} \vert d_i, x_i; \theta\right)
=\sum_{i=1}^n \[d_i \log f_1\(y_{i}, z_{i} \vert x_i; \theta\right)
+ \left(1-d_i\right) \log f_0\(y_{i}, z_{i} \vert x_i; \theta\right)\right].
\end{split}\end{align}
Under standard regularities, the asymptotic linear representation as in \eqref{mle influence function} continues to hold with
\begin{align}\begin{split}\nonumber
s_\theta\(y_i, z_i \vert d_i, x_i; \theta\right) &
 = d_i \frac{\partial}{\partial\theta}	\log f_1\(y_{i}, z_{i} \vert x_i; \theta\right) + \left(1-d_i\right) \frac{\partial}{\partial\theta}	\log f_0\(y_{i}, z_{i} \vert x_i; \theta\right), \\
H\left(\theta\right) &
= Q\left(
d \frac{\partial^{2}}{\partial\theta^{2}}  \log f_1\(y, z \vert x; \theta\right) + \left(1-d\right) \frac{\partial^{2}}{\partial\theta^{2}}  \log f_0\(y, z \vert x; \theta\right)\right).
\end{split}\end{align}

In addition to the model based estimator \eqref{estimate of model based representation of constrained optimized value function under full observability},
an orthogonal doubly robust value function estimator under
unconfoundedness can be formed by re-expressing
\eqref{double robust representing of constrained optimized value function under full observability}
as
\begin{align}\begin{split}\label{double robust representing of constrained optimized value function under partial observability}
\beta\(Q, \Delta, k\right) & \coloneqq   Q\biggl[
\Gamma_{1}\(x, y, d\right)
1\left(\Delta\(x; k\right) > 0\right)
+
\Gamma_{0}\(x, y, 1 - d\right)
1\left(\Delta\(x; k\right) \leq 0\right)
\biggr], \\
\alpha\(Q, \Delta, k\right) & \coloneqq
Q\biggl[
\Xi_{1}\(x, z, d\right)
1\left(\Delta\(x; k\right) > 0\right)
+
\Xi_{0}\(x, z, 1 - d\right)
1\left(\Delta\(x; k\right) \leq 0\right)
\biggr],\\
\beta\(Q, \Delta, \alpha\right) & \coloneqq	\beta\(Q, \Delta, k\(Q, \Delta, \alpha\right)\right), \ k\(Q, \Delta, \alpha\right)  = \inf\left\{k \in \mathbb{R}: \alpha\(Q, \Delta, k\right) \leq \alpha \right\}.
\end{split}\end{align}
In the above,
for $j \in \{0, 1\}$,  $p_{1}\left(\cdot\right) \coloneqq p\left(\cdot\right)$ and $p_{0}\left(\cdot\right) \coloneqq 1 - p\left(\cdot\right)$, we denote
$\Gamma_{j}\(x, y, d\right) \coloneqq g_{j}\(x\right) + \frac{d}{p_{j}\(x\right)}\(y - g_{j}\(x\right)\right)$ and $\Xi_{j}\(x, z, d\right) = c_{j}\(x\right) + \frac{d}{p_{j}\(x\right)}\(z - c_{j}\(x\right)\right)$.
Under correct specification, and assume that $\epsilon < p\left(\cdot\right) < 1-\epsilon$, for some constant $\epsilon > 0$,
an adaption to Theorem \ref{binary allocation asymptotic result} shows that
\begin{align}\begin{split}\nonumber
\sqrt{n}\left(\beta\left(\mathbb{Q}_{n}, \hat{\Delta}, \alpha\right) - \beta\(Q, \Delta^{\star}, \alpha\right)\right) \rightsquigarrow
\mathbb{Q}\left(\zeta\(y, z, d, x; \alpha\right)\right)
\end{split}\end{align}
where for $\phi^{\star}\(x; \alpha\right) \coloneqq 1\left(\Delta^{\star}\(x; k\(Q, \Delta^{\star}, \alpha\right)\right) > 0\right)$, we write
\begin{align}\begin{split}\nonumber
\zeta\(y,z,d,x; \alpha\right) & \coloneqq
\Gamma_{1}\(x, y, d\right)
\phi^{\star}\(x; \alpha\right)
+
\Gamma_{0}\(x, y, 1 - d\right)
\left(1 - \phi^{\star}\(x; \alpha\right)\right) \\
& - k\(Q, \Delta^{\star}, \alpha\right)
\biggl(
\Xi_{1}\(x, z, d\right)
\phi^{\star}\(x; \alpha\right)
+
\Xi_{0}\(x, z, 1 - d\right)
\left(1 - \phi^{\star}\(x; \alpha\right)\right)
\biggr).
\end{split}\end{align}

\begin{proposition}\label{treatment parametric model based estimator more efficient than orthogonal estimator}
Assume that $\epsilon < p\left(\cdot\right) < 1-\epsilon$ for $\epsilon > 0$ and that
both \eqref{mle influence function} and \eqref{binary unconfoundedness assumption}
hold.
Under the conditions in Theorem \ref{binary allocation asymptotic result},
when the parametric models $f_{1}\(y, z\vert x; \theta\right)$ and $f_{0}\(y, z\vert x; \theta\right)$ are correctly specified,
the model based estimator
\eqref{estimate of model based representation of constrained optimized value function under full observability}
is asymptotically more efficient than
the parametric variant of the orthogonal estimator \eqref{double robust representing of constrained optimized value function under partial observability}
where $\hat \Delta\(x\right) = \Delta\(x; \hat\theta\right)$.
\end{proposition}

\subsection{The ROC curve}\label{theory for plug-in ROC curve}
A special case of \eqref{constrained optimized value function} is
when $Y_{1}=Y \in \{0, 1\}$, $Y_{0} = Z_{0} = 0$, $Z_{1} = 1 - Y_{1}$,
$g_{0}\(x\right) = c_{0}\(x\right) \equiv 0$, $g_{1}\(x\right) = p\(x\right)$, $c_{1}\(x\right) = 1 - p\(x\right)$ where $p\(x\right) = \mathbb{E}\(Y \vert\ X = x\right)$.
This special case, i.e.
\begin{align}\begin{split}\nonumber
\max_{\phi \in \Phi}\left\{\mathbb{E} \[Y \phi\(X\right)\right]: \mathbb{E} \left[\left(1 - Y\right)\phi\(X\right)\right] \leq \alpha\right\},
\end{split}\end{align}
is closely related to the population receiver operating characteristic (ROC) curve defined by
\begin{align}\begin{split}\label{roc curve raw definition}
\beta\left(\alpha\right) \coloneqq \max_{\phi \in \Phi} \frac{\mathbb{E} \[Y\phi\(X\right)\right]}{\mathbb{E}Y} \quad \text{such that} \quad
\frac{\mathbb{E} \left[\left(1 - Y\right)\phi\(X\right)\right]}{\mathbb{E}\left(1 - Y\right)} \leq \alpha.
\end{split}\end{align}
The objective in \eqref{roc curve raw definition} is to maximize the power $\beta$
while controlling the size of the test $\alpha$.
This optimization problem is exactly the same as that of the \textit{Uniformly Most Powerful Tests} under simple null and alternative hypotheses.
See for example section 3 of \cite{lehmann2022testing} for rigorous definitions and implications of these statistical testing concepts.
In this paper, we adopt the more commonly employed terminology in the computer science, biostatistics and economics literatures to refer to $\beta$ and $\alpha$ as \textit{true positive rate} (TPR) and \textit{false positive rate} (FPR).
In \cite{berge2011evaluating}, $Y_1 = Y$ is interpreted as the state of the economy ($Y = 0$ for expansion and $Y = 1$ for recession).
The ROC curve is also used in the finance loan literature
\citep{vallee2019marketplace,sadhwani2021deep} to evaluate the tradeoff between earned interest and lost principal, where
$Y=0$ indicates loan fulfillment and $Y=1$ indicates loan default.
The abnormal birth detection application of \cite{feng2025statisticalMS} uses $Y$ as the indicator of abnormal birth ($Y = 0$ for normal birth
and $Y = 1$ for abnormal birth).

The analog of \eqref{functional form binary constrained allocation} for the
constraint and value functions of \eqref{roc curve raw definition} is
\begin{align}\begin{split}\label{functional form roc curve}
\alpha\(k\right)  = Q\left[\left(1-y\right)1\left(
p\(x\right) > k
\right) \right] / Q\left(1-y\right), \
\beta\(k\right)  = Q\[y1\left(
p\(x\right) > k
\right) \right] / Q\(y\right).
\end{split}\end{align}
In order to adapt Theorem \ref{binary allocation asymptotic result} to the ROC curve in \eqref{roc curve raw definition} and \eqref{functional form roc curve},
define	similarly
to \eqref{double robust representing of constrained optimized value function under full observability},
\begin{align}\begin{split}\label{double robust representing of roc constrained optimized value function under full observability}
&\beta\(Q, p, \alpha\right) \coloneqq \beta\(Q, p, k\(Q, p, \alpha\right)\right),
\beta\(Q, p, k\right) = Q\[y1\(p\(x\right) > k\right)\right] / Q\(y\right),\\
&\alpha\(Q, p, k\right) \coloneqq
Q\left[\left(1-y\right)1\(p\(x\right) > k\right)\right] / Q\left(1-y\right),
\; k\(Q, p, \alpha\right) = \inf\left\{k \in \mathbb{R}:
\alpha\(Q, p, k\right) \leq \alpha \right\}.
\end{split}\end{align}
Different from \eqref{double robust representing of constrained optimized value function under full observability},
\eqref{double robust representing of roc constrained optimized value function under full observability} has
denominators
and $\Delta\(x; k\right)$ is replaced by $p\(x\right) - k$.
We also define
\begin{align}\begin{split}\label{beta alpha derivative wrt k}
f_{\beta}\(Q, p^{\star}, k\right) & \coloneqq
 \frac{d\beta\(Q, p^{\star}, k\right)}{dk} = - \frac{1}{Q y}
\int_{p^{\star}\(x\right) = k}
\frac{p\(x\right)
\mu'\(x\right)}{\Vert\nabla p^{\star}\(x\right)\Vert}d\mathcal{H}_{n - 1}x, \\
f_{\alpha}\(Q, p^{\star}, k\right) & \coloneqq
\frac{d\alpha\(Q, p^{\star}, k\right)}{dk} = - \frac{1}{1- Qy}\int_{p^{\star}\(x\right) = k}
\frac{\left(1 - p\(x\right)\right)\mu'\(x\right)}{\Vert\nabla p^{\star}\(x\right)\Vert}d\mathcal{H}_{n - 1}x.
\end{split}\end{align}
The following
corollary
adapts
Theorem \ref{binary allocation asymptotic result} to the commonly used orthogonal estimator of \eqref{roc curve raw definition}.

\begin{cor}\label{roc asymptotic result}
Let the conditions in Theorem \ref{binary allocation asymptotic result} hold with $\(g_{0}\(x\right), g_{1}\(x\right), c_{0}\(x\right), c_{1}\(x\right)\right) = \left(0, p\(x\right), 0, 1 - p\(x\right)\right)$ and $\mathbb{H}_p\left(\cdot\right)=\mathbb{H}_{g1}\left(\cdot\right)$.
On compact $\Lambda' \subsetneq \overset{\circ}{\Lambda}$, $\Lambda \coloneqq \left\{\frac{Q\left[\left(1 - y\right)1\(p^{\star}\(x\right) > k\right)\right]}{Q\left(1 - y\right)}; k \in \mathcal D\right\}$,
\begin{align}
& r_{n}\left(\beta\left(\mathbb{Q}_{n}, \hat{p}, \alpha\right) - \beta\(Q, p^{\star}, \alpha\right)\right) \rightsquigarrow \frac{1}{Q y}\Biggl\{
t_{Q}\mathbb{Q}\[y\left(1\(p^{\star}\(x\right) > k\(Q, p^{\star}, \alpha\right)\right) - \beta\(Q, p^{\star}, \alpha\right)\right)\right] \notag \\
& +
\int_{p^{\star}\(x\right)= k\(Q, p^{\star}, \alpha\right)} \frac{p\(x\right)\mathbb{H}_p\(x\right)\mu'\(x\right)}{\Vert\nabla p^{\star}\(x\right)\Vert}d\mathcal{H}_{n - 1}x
\Biggl\} - \frac{1}{1-Q y} \frac{f_{\beta}\(Q, p^{\star}, k\(Q, p^{\star}, \alpha\right)\right)}{f_{\alpha}\(Q, p^{\star}, k\(Q, p^{\star}, \alpha\right)\right)} \Biggl\{ \label{roc main asymptotic distribution} \\
& t_{Q}\mathbb{Q}\left[\left(1 - y\right)\left(1\(p^{\star}\(x\right) > k\(Q, p^{\star}, \alpha\right)\right) - \alpha\right)\right] +
\int_{p^{\star}\(x\right)=k\(Q, p^{\star}, \alpha\right)} \frac{\left(1-p\(x\right)\right)\mathbb{H}_p\(x\right)\mu'\(x\right)}{\Vert\nabla p^{\star}\(x\right)\Vert}d\mathcal{H}_{n - 1}x
\Biggl\}. \notag
\end{align}
\end{cor}

Under correct specification, $p^\star\left(\cdot\right) = p\left(\cdot\right)$ and
$\frac{
k
}{
1- k
} = \frac{Q y}{1-  Qy}
\frac{f_{\beta}\(Q, p^{\star}, k\right)}{f_{\alpha}\(Q, p^{\star}, k\right)}$.
This is
because, as  $\left(1-k\right) p\(x\right) - k \left(1-p\(x\right)\right) = 0$
on $p^\star\(x\right) = p\(x\right)= k$,
\begin{align}\begin{split}\nonumber
k \left(1-Qy\right) f_{\alpha}\(Q, p^{\star}, k\right)-
\left(1-k\right) \(Q y\right) f_{\beta}\(Q, p^{\star}, k\right)
= \int_{p^{\star}\(x\right)=k}
\frac{
0 \cdot \mu'\(x\right)}{\Vert\nabla p^{\star}\(x\right)\Vert}d\mathcal{H}_{n - 1}x=0.
\end{split}\end{align}
Furthermore, the two integrals on the level set in Corollary \ref{roc asymptotic result} cancel out since
\begin{align}\begin{split}\nonumber
&
\frac{1}{Q y}
\int_{p^{\star}\(x\right)= k} \left[
\frac{p\(x\right)\mathbb{H}_{p}\(x\right)\mu'\(x\right)}{\Vert\nabla p^{\star}\(x\right)\Vert}
- \frac{Qy}{1-Q y} \frac{f_{\beta}\(Q, p^{\star}, k\right)}{f_{\alpha}\(Q, p^{\star}, k\right)}
\frac{\left(1-p\(x\right)\right)\mathbb{H}_{p}\(x\right)\mu'\(x\right)}{\Vert\nabla p^{\star}\(x\right)\Vert}
\]d\mathcal{H}_{n - 1}x \\
&=\frac{1}{Q y}
\int_{p^{\star}\(x\right)= k}\underbrace{\left[
p\(x\right)
- \frac{k}{1-k}
\left(1-p\(x\right)\right)
\right]}_{= 0 \; \text{on} \; p^\star\(x\right) = p\(x\right) = k}
\frac{\mathbb{H}\(x\right)\mu'\(x\right)}
{\Vert\nabla p^{\star}\(x\right)\Vert}
d\mathcal{H}_{n - 1}x = 0.
\end{split}\end{align}
Under correct specification \eqref{roc main asymptotic distribution}
simplifies to
\begin{align}
&r_{n}\left(\beta\left(\mathbb{Q}_{n}, \hat{p}, \alpha\right) - \beta\(Q, p, \alpha\right)\right)
\label{roc curve asymptotics under correct specification}
\\
& \rightsquigarrow  \frac{t_{Q}}{Qy}\Biggl\{\mathbb{Q}\biggl[
\frac{y - k\(Q, p, \alpha\right)}{1 - k\(Q, p, \alpha\right)}
1\(p\(x\right) > k\(Q, p, \alpha\right)\right) - y\beta\(Q, p, \alpha\right)
+ \frac{k\(Q, p, \alpha\right)}{1 - k\(Q, p, \alpha\right)} \left(1 - y\right)\alpha
\biggr]
\Biggl\}.
\notag
\end{align}
In parametric models of $p\(x\right)$, where $r_n = \sqrt{n}$ and $t_Q = 1$,
\eqref{roc curve asymptotics under correct specification}
has a Gaussian
covariance kernel
\begin{align}\begin{split}\label{theoretical distribution of correctly specified ROC model}
\left(\alpha, \alpha'\right) \mapsto
\frac{1}{\(Q y\right)^2} & \biggl(Q y \left(\beta\(Q, p, \alpha \wedge \alpha'\right)
-
\beta\(Q, p, \alpha\right)\beta\(Q, p, \alpha'\right)\right) \\
&
+ \frac{
k\(Q, p, \alpha\)k\(Q, p, \alpha'\right)
}{
\left(
1- k\(Q, p, \alpha\right)
\right)\left(
1- k\(Q, p, \alpha'\right)
\right)
} \left(1- Qy\right) \left(\alpha \wedge \alpha' - \alpha \alpha'\right)\biggr).\\
\end{split}\end{align}
\begin{comment}
\begin{align}\begin{split}\nonumber
	\left(\alpha, \alpha'\right) \mapsto &
	\frac{1}{\(Q y\right)^2} \left(\(Q y\right) \beta\(Q, p, \alpha \wedge \alpha'\right)+
	\frac{
k\(Q, p, \alpha\)k\(Q, p, \alpha'\right)
}{
\left(
1- k\(Q, p, \alpha\right)
\right)\left(
1- k\(Q, p, \alpha'\right)
\right)
} \left(1- Qy\right) \left(\alpha \wedge \alpha'\right)\right)\\
	&- \left(
\beta\(Q, p, \alpha\right)
-
	\frac{
k\(Q, p, \alpha\right)
}
{
1- k\(Q, p, \alpha\right)
}
\frac{1-Q y}{Q y} \alpha
	\right)
      \left(
\beta\(Q, p, \alpha'\right)
-
	\frac{
k\(Q, p, \alpha'\right)
}
{
1- k\(Q, p, \alpha'\right)
}
\frac{1-Q y}{Q y} \alpha'
	\right),
\end{split}\end{align}
\end{comment}
Similar to \eqref{model based constrained optimized value function}
and \eqref{estimate of model based representation of constrained optimized value function under full observability},
a model based representation of the ROC curve \eqref{roc curve raw definition} is
\begin{align}\begin{split}\nonumber
\beta\left(\alpha\right) \coloneqq \max_{\phi \in \Phi} \frac{\mathbb{E} \[p\(X\right)\phi\(X\right)\right]}{\mathbb{E} p\(X\right)}\quad \text{such that}\quad \frac{\mathbb{E} \left[\left(1 - p\(X\right)\right)\phi\(X\right)\right]}{\mathbb{E}\left(1 - p\(X\right)\right)} \leq \alpha.
\end{split}\end{align}
The corresponding estimator is
\begin{align}
&\alpha\left(\hat{\mu}, \hat{\theta}, k\right) \coloneqq
\hat{\mu}\left[\left(1 - p\(x; \hat{\theta}\right)\right)1\(p\(x; \hat{\theta}\right) > k\right)\right] / \hat{\mu}\left(1 - p\(x; \hat{\theta}\right)\right),\nonumber \\
&k\left(\hat{\mu}, \hat{\theta}, \alpha\right)  = \inf\left\{k \in \mathbb{R}: \alpha\left(\hat{\mu}, \hat{\theta}, k\right) \leq \alpha \right\},
  \label{roc model-based functional expression}
\\
&\beta\left(\hat{\mu}, \hat{\theta}, \alpha\right) \coloneqq
\beta\left(\hat{\mu}, \hat{\theta}, k\left(\hat{\mu}, \hat{\theta}, \alpha\right)\right),
\beta\left(\hat{\mu}, \hat{\theta}, k\right)
= \hat{\mu}\[p\(x; \hat{\theta}\right)1\(p\(x; \hat{\theta}\right) > k\right)\right] / \hat{\mu}\(p\(x; \hat{\theta}\right)\right).\nonumber
\end{align}

Typically,
$\hat\theta = \arg\max_{\theta\in\Theta} \sum_{i=1}^n \[y_i \log p\(x_i; \theta\right) + \left(1-y_i\right) \log \left(1-p\(x_i; \theta\right)\right)\right]$ is the maximum likelihood estimator.
\begin{comment}
On the one hand, when the parametric model is misspecified, the orthogonal estimator $\beta\left(\mathbb Q_n, \hat\theta, \alpha\right)$
still has the correct meaning of the ROC curve,
while it will be hard to interpret the model-based estimator $\beta\left(\hat\mu, \hat\theta, \alpha\right)$ in \eqref{roc model-based functional expression}.
On the other hand,
when the parametric model is correctly specified, $\beta\left(\hat\mu, \hat\theta, \alpha\right)$ in
is more efficient than
$\beta\left(\mathbb Q_n, \hat\theta, \alpha\right)$.
See also a synthetic data example in section \ref{simulation data CI and testing}.
\end{comment}
When the parametric model is correctly specified,
the model based estimator is
more efficient than
the orthogonal estimator,
as formalized in the next proposition and illustrated in a
synthetic data example in section \ref{simulation data CI and testing}.


\begin{proposition}\label{ROC parametric model based estimator more efficient than orthogonal estimator}
Assume that \eqref{mle influence function} holds.
Under the conditions in Corollary \ref{roc asymptotic result},
when the parametric model is correctly specified,
the model based estimator $\beta\left(\hat\mu, \hat{\theta}, \alpha\right)$ in
\eqref{roc model-based functional expression}
is asymptotically more efficient than
the orthogonal estimator in
\eqref{roc curve asymptotics under correct specification}.
\end{proposition}


Misspecified parametric models estimated by the orthogonal estimator
are discussed in Appendix \ref{supplementary discussion on misspecified parametric models}.
The next Corollary \ref{roc bootstrap}
validates bootstrap consistency under general specification.
For correctly specified parametric models,
the first step model only needs to be trained once and does not need to be re-estimated for every bootstrapped sample.
\begin{comment}
In the following presentation of the uniform asymptotic distribution of the ROC curve, we differentiate between misspecified models and correctly specified models.
The limit of misspecified models is essentially a direct application of the general results in Theorem \ref{binary allocation asymptotic result}.
When the parametric propensity score model is correctly specified, or when it is estimated by
nonparametric techniques with a sufficiently fast rate of convergence,
it will be shown that the first stage estimation of the propensity score model has no impact on the asymptotic distribution of the ROC curve.
This property simplifies the inference procedure when the propensity score is nonparametrically estimated or correctly parametrically specified.
In particular, it is not necessary to reestimate the propensity score model in a bootstrap procedure.

Under correct model specification, corollary \ref{roc asymptotic result} implies a computationally feasible functional delta method for bootstrap in probability. Here, we do not need to recompute / reestimate the first step model for every bootstrapped sample.
On the contrary, the first step only need to be trained once, as formalized in the following.
\end{comment}

Let $M = \(M_{1}, \ldots, M_{n}\right)$ be the vector of bootstrap weights.
For a random element $\tilde{\mathbb{Z}}_{n} \coloneqq \tilde{\mathbb{Z}}_{n}\left(\{\(X_{i}, Y_{i}\right)\}^{n}_{i = 1}, M\right)$ on normed space $\mathcal{X}$,
we say its bootstrap law is consistent for a tight process $\mathbb{G}$
and denote it as $\tilde{\mathbb{Z}}_{n} \overset{\mathbb{P}}{\underset{\mathbb{W}}{\rightsquigarrow}} \mathbb{G}$
if
in outer probability\footnote{
Under measurability, the consistency definition can be simplified to
$\sup_{l \in BL_{1}\left(\mathcal{X}\right)} \left\vert \mathbb{E}_{M}l\left(\tilde{\mathbb{Z}}_{n}\right) - \mathbb{E}l\left(\mathbb{G}\right)\right\vert = o_{\mathbb{P}}\left(1\right).$
}
\begin{align}\begin{split}\label{conditional weak convergence in probability}
\sup_{l \in BL_{1}\left(\mathcal{X}\right)} \left\vert \mathbb{E}_{M}l\left(\tilde{\mathbb{Z}}_{n}\right) - \mathbb{E}l\left(\mathbb{G}\right)\right\vert \to 0, \; \mathbb{E}_{M}l\left(\tilde{\mathbb{Z}}_{n}\right)^{*} - \mathbb{E}_{M}l\left(\tilde{\mathbb{Z}}_{n}\right)_{*} \to 0.
\end{split}\end{align}
In the above $\mathbb{E}_{M}$ denotes the conditional expectation with respect to $M$ given the data $\{\(X_{i}, Y_{i}\right)\}^{n}_{i = 1}$, and $BL_{1}\left(\mathcal{X}\right)$ denotes the set of
real-valued functions on $\mathcal{X}$ with Lipschitz constant and supremum norm both bounded by $1$.

\begin{cor}\label{roc bootstrap}
Assume the function class $\mathcal{G}$ in Assumption \ref{joint convergence assumption} is Donsker
and
for $\tilde{\mathbb{Q}}_{n} \coloneqq \tilde{\mathbb{Q}}_{n}\left(\{\(X_{i}, Y_{i}\right)\}^{n}_{i = 1}, M\right)$ and $M \sim multinomial\(n, \left(\frac{1}{n}, \ldots, \frac{1}{n}\right)\right)$,
$\sqrt{n}\left(\tilde{p} - \hat{p}, \tilde{\mathbb{Q}}_{n} -\mathbb{Q}_{n}\right) \overset{\mathbb{P}}{\underset{\mathbb{W}}{\rightsquigarrow}} \left(\mathbb{H}_{p}, \mathbb{Q}\right)$.
Under the conditions in Corollary \ref{roc asymptotic result} and correct specification, the orthogonal estimator satisfies
\begin{align}\begin{split}\nonumber
\sup_{l \in BL_{1}\left(\ell^\infty\left(\mathcal{D}\right)\right)} & \left\vert
\mathbb{E}_{M}l\left(\sqrt{n}\left(\beta\left(\tilde{\mathbb{Q}}_{n}, \hat{p}, \alpha\right) - \beta\left(\mathbb{Q}_{n}, \hat{p}, \alpha\right)\right)\right)
- \mathbb{E}l\left(\beta'_{Q, p, \alpha}\left(\mathbb{Q}, 0, 0\right)\right) \right\vert\to 0,
 \\
& \mathbb{E}_{M}l\left(\sqrt{n}\left(\beta\left(\tilde{\mathbb{Q}}_{n}, \hat{p}, \alpha\right) - \beta\left(\mathbb{Q}_{n}, \hat{p}, \alpha\right)\right)\right)^{*}
- \mathbb{E}_{M}l\left(\sqrt{n}\left(\beta\left(\tilde{\mathbb{Q}}_{n}, \hat{p}, \alpha\right) - \beta\left(\mathbb{Q}_{n}, \hat{p}, \alpha\right)\right)\right)_{*} \to 0
\end{split}\end{align}
in outer probability.
Under general specification,
\begin{align}\begin{split}\nonumber
\sqrt{n}\left(\beta\left(\tilde{\mathbb{Q}}_{n}, \tilde{p}, \alpha\right) - \beta\left(\mathbb{Q}_{n}, \hat{p}, \alpha\right)\right)
\overset{\mathbb{P}}{\underset{\mathbb{W}}{\rightsquigarrow}} \beta'_{Q, p^{\star}, \alpha}\left(\mathbb{Q}, \mathbb{H}_{p}, 0\right).
\end{split}\end{align}
\end{cor}

Consider i.i.d. data $\{\(Y_i, W_i, X_i\right), i=1,\ldots,n\}$, where $Y_i$ is the true outcome, $W_i$ is the human decision,
and $X_i$ are observable features.
An estimator of the human TPR/FPR pair is
\begin{align}\begin{split}\nonumber
\hat{\alpha}_{H} = \sum_{i=1}^n \left[\left(1-Y_i\right) W_i\right] / \sum_{i=1}^n \left(1-Y_i\right), \quad
\hat{\beta}_{H} = \sum_{i=1}^n	Y_i W_i / \sum_{i=1}^n Y_i.
\end{split}\end{align}
Human decision making quality can be evaluated by testing
whether
the population TPR/FPR pair of a human decision maker, denoted $\left(\alpha_{H}, \beta_{H}\right)$,
lies on, above or below a population ROC curve $\beta\left(\alpha\right)$ of a machine learning model.
In the above,
\begin{align}\begin{split}\nonumber
\alpha_{H} = \mathbb{E}\left[\left(1-Y\right) W\right] / \mathbb{E}\left(1-Y\right), \quad
\beta_{H} = \mathbb{E}\[Y W\right] / \mathbb{E}Y.
\end{split}\end{align}
The null and alternative hypotheses can be formulated as
\begin{align}\begin{split}\label{testing a person against a roc curve}
H_0: \beta\left(\alpha_{H}\right) = \beta_{H} \quad\text{versus}\quad H_A: \beta\left(\alpha_{H}\right) \neq \beta_{H},
\end{split}\end{align}
where $H_A$ can be further categorized into two one-sided directional alternatives $H_{A,h}: \beta\left(\alpha_{H}\right) < \beta_{H}$
and $H_{A,m}: \beta\left(\alpha_{H}\right) > \beta_{H}$, as in \cite{vuong1989likelihood}.
It follows from the Lindeberg-L\'{e}vy central limit theorem that
for $\theta_{H} = \left(\alpha_{H}, \beta_{H}\right)'$,
$\hat{\theta}_{H} = \left(\hat{\alpha}_{H}, \hat{\beta}_{H}\right)'$ and $p = Q Y$,
\begin{align}\begin{split}\nonumber
\sqrt{n}\left(\hat{\theta}_{H} - \theta_{H}\right)
\rightsquigarrow  N\left(0, \Sigma\right) \quad\text{where}\quad
\begingroup
\Sigma = \begin{varmatrix}[delim = p, size = \normalsize, sep = 0.5\arraycolsep]
	   \frac{\alpha_H\left(1-\alpha_H\right)}{1 - p}   & 0 \\
	    0 & \frac{\beta_H\left(1-\beta_H\right)}{p}
	    \end{varmatrix}
\endgroup.
\end{split}\end{align}
When the
ROC $\beta\left(\alpha\right)$ is fully known,\footnote{
For example, \cite{feng2025statisticalMS} assume a known ROC because
the sample size of the data set used to train the machine learning
model is by orders of magnitude larger than the sample size for each
individual decision maker.
}
the hypothesis \eqref{testing a person against a roc curve} can be tested using the statistics	$\beta\left(\hat{\alpha}_{H}\right) - \hat{\beta}_{H}$.
By the delta method,
under the
null hypothesis
$\beta\left(\alpha_{H}\right) = \beta_{H}$ in \eqref{testing a person against a roc curve},
\begin{align}\begin{split}\nonumber
\sqrt{n} \left(\beta\left(\hat{\alpha}_{H}\right) - \hat{\beta}_{H}\right) \rightsquigarrow
N\left(0, \sigma^2\right) \quad\text{where}\quad
\sigma^2 = \beta'\left(\alpha_H\right)^2\frac{\alpha_H\left(1 - \alpha_H\right)}{1 - p} + \frac{\beta_H\left(1 - \beta_H\right)}{p}.
\end{split}\end{align}

\begin{comment}
The curvature $\beta'\left(\alpha\right)$ depends on the model specification. For a nonparametric orthogonal ROC or a parametric orthogonal ROC
under correct specification, $\beta\left(\alpha\right) = \beta\(Q, \Delta, k\(Q, \Delta, \alpha\right)\right)$ in
\eqref{double robust representing of roc constrained optimized value function under full observability},
so that
$\beta'\left(\alpha\right) = \frac{1-q}{q} \frac{k\left(\alpha\right)}{1-k\left(\alpha\right)}$ where
$k\left(\alpha\right) = k\(Q, p, \alpha\right)$ as defined in
\eqref{double robust representing of roc constrained optimized value function under full observability}.
For a parametric model based ROC,
$\beta\left(\alpha\right) = \beta\left(\mu, \theta^\star, \alpha\right)$
in \eqref{main asymptotic formula under general parametric specification of model based roc} and
\begin{align}\begin{split}\nonumber
\beta'\left(\alpha\right) = \frac{1-q^*}{q^*} \frac{k\left(\mu, \theta^*, \alpha\right)}{1-
k\left(\mu, \theta^*, \alpha\right)}.
\end{split}\end{align}
In the above $q^* = \mu\(p\(x; \theta^\star\right)\right)$ and $k\left(\mu, \theta^*, \alpha\right)$
is defined after \eqref{estimate of model based representation of roc curve}.
For an orthogonal parametric ROC under general misspecification,
$\beta\left(\alpha\right) = \beta\(Q, \theta^*, \alpha\right)$ in
\eqref{main asymptotic formula under mis parametric specification}, and
\begin{align}\begin{split}\nonumber
\beta'\left(\alpha\right) = \frac{
f_\beta\(Q, \theta^*, k\(Q, \theta^*, \alpha\right)\right)
}{
f_\alpha\(Q, \theta^*, k\(Q, \theta^*, \alpha\right)\right)
}
\end{split}\end{align}
as defined in
\eqref{main asymptotic formula under mis parametric specification} and
\eqref{beta alpha derivative wrt k under parametric misspecification}. In each
of these cases, the asymptotic variance can be consistently estimated by a sample analog, denoted as
$\hat\sigma^2$.	 The null hypothesis in \eqref{testing a person against a roc curve}
is then rejected with $\eta$ level size when
$\sqrt{n}\left(\beta\left(\hat\alpha\right) - \hat\beta\right)$ exceeds the $1-\eta$ percentile of the standard normal distribution.
\end{comment}

A more complex situation is when the machine ROC curve $\hat\beta\left(\alpha\right)$ and the human TPR/FPR pair $\left(\hat{\alpha}_{H}, \hat{\beta}_{H}\right)$ are estimated
from the same dataset.
In such a case,
a revised test statistic for the hypothesis
in \eqref{testing a person against a roc curve} is then
$\hat\beta\left(\hat{\alpha}_{H}\right) - \hat{\beta}_{H} =
\hat{\beta}\left(\hat{k}\left(\hat{\alpha}_{H}\right)\right) - \hat{\beta}_{H}$,
where $\hat\beta\left(\cdot\right)$ can be any one of orthogonal ROC $\beta\left(\mathbb Q_n, \hat p\left(\cdot\right), \cdot\right)$,
parametric model based ROC $\beta\left(\hat\mu, \hat\theta, \cdot\right)$ and parametric orthogonal ROC $\beta\left(\mathbb Q_n, \hat \theta, \cdot\right)$.
Similarly, $\hat{k}\left(\cdot\right)$ can be the corresponding orthogonal $k\left(\mathbb Q_n, \hat p\left(\cdot\right), \cdot\right)$,
parametric model based $k\left(\hat\mu, \hat\theta, \cdot\right)$ and parametric orthogonal $k\left(\mathbb Q_n, \hat \theta, \cdot\right)$ in
\eqref{double robust representing of roc constrained optimized value function under full observability} and
\eqref{roc model-based functional expression}.
The following Hadamard differentiation result applies to these estimators.

\begin{lemma}\label{test Hadamard derivative}
Let $\(Y, W, X\right) \sim S$.
Denote,
\begin{align}\begin{split}\nonumber
& \alpha\(S, p, k\right) \coloneqq S\left[\left(1 - y\right)1\(p\(x\right) > k\right)\right] / S\left(1 - y\right),\  k\(S, p, \alpha\right) \coloneqq \inf\left\{k \in \mathbb{R}: \alpha\(S, p, k\right) \leq \alpha \right\}, \\
&k\(S, p\right) \coloneqq \inf\left\{k \in \mathbb{R}: \alpha\(S, p, k\right) \leq \alpha_H \right\},\
\alpha_{H} \coloneqq S\left[\left(1 - y\)w\right] / S\left(1 - y\right),
\\
& \beta\(S, p, k\right) \coloneqq S\[y1\(p\(x\right) > k\right)\right] / S\(y\right), \; \beta\(S, p\right) \coloneqq S\[y1\(p\(x\right) > k\(S, p\right)\right)\right] / S\(y\right).
\end{split}\end{align}
Under the conditions in Corollary \ref{roc asymptotic result},
$\beta\(S, p\right)$ is Hadamard differentiable at $\(S, p^{\star}\right)$ tangentially to $\mathbf{S} \times C\(E\right)$,
where $\mathbf{S} \subset \ell^{\infty}\left(\mathcal{F}\right)$ consists of elements that are uniformly continuous on $\mathcal{F}$
with respect to the $L^{2}\(S\right)$ norm.
The Hadamard derivative is
\begin{align}
& \beta'_{S, p^{\star}}\left(\mathcal{S}, H\right) =
\frac{1}{S\(y\right)}\biggl\{\mathcal{S}\[y\left(1\(p^{\star}\(x\right) > k^{\star}\right) - \beta\(S, p^{\star}, k^{\star}\right)\right)\right] \notag \\
& + \int_{p^{\star}\(x\right) = k^{\star}}\frac{p\(x\)H\(x\right)\mu'\(x\right)}{\Vert\nabla p^{\star}\(x\right)\Vert}d\mathcal{H}_{n - 1}x\biggl\}
- \frac{1}{S\left(1 - y\right)}\frac{f_{\beta}\(S, p^{\star}, k^{\star}\right)}{f_{\alpha}\(S, p^{\star}, k^{\star}\right)}\biggl\{ \label{testing roc doctor Hadamard derivative} \\
& \mathcal{S}\left[\left(1 - y\right)\left(1\(p^{\star}\(x\right) > k^{\star}\right) - \(w - \alpha_{H}\right) - \alpha\(S, p^{\star}, k^{\star}\right)\right)\right] + \int_{p^{\star}\(x\right) = k^{\star}}\frac{\left(1 - p\(x\right)\)H\(x\right)\mu'\(x\right)}{\Vert\nabla p^{\star}\(x\right)\Vert}d\mathcal{H}_{n - 1}x\biggr\}, \notag
\end{align}
where $k^{\star} = k\(S, p^{\star}, \alpha_{H}\right)$, $f_{\alpha}\(S, p^{\star}, k\right)$ and $f_{\beta}\(S, p^{\star}, k\right)$ are defined in \eqref{beta alpha derivative wrt k}.
\end{lemma}

When $p\(x\right)$ is modeled parametrically as $p\(x; \theta\right)$
and $\sqrt{n}\left(\hat{\theta} - \theta^{\star}\right)$ has an asymptotic linear representation by $\kappa\(x, y\right)$,
by the delta method, $H\(x\right)$ will be determined by $\frac{\partial}{\partial\theta}p\(x; \theta^{\star}\right)$
and $\kappa\(x_i, y_i\right)$ giving rise to an asymptotic linear representation.
In each case, the asymptotic variance can be consistently estimated by sample analogs. Bootstrap also provides valid inference.
The asymptotic distribution and verification of bootstrapped confidence interval of
$\sqrt{n}\left[\left(\hat{\beta}\left(\hat{\alpha}_{H}\right) - \hat{\beta}_{H}\right) - \left(\beta\left(\alpha_{H}\right) - \beta_{H}\right)\right]$ is obtained by
combining the standard term  $\hat{\beta}_{H} - \beta_{H}$  with $\hat\beta\left(\hat{\alpha}_{H}\right) - \beta\left(\alpha_{H}\right)$ handled by
\eqref{testing roc doctor Hadamard derivative}.
Section \ref{simulation data CI and testing} showcases a synthetic data example.
\begin{comment}
\todo[caption={{\bf Kai:} Do we need to list these results? I think the most concise way is to just state Lemma \ref{test Hadamard derivative}}, size=\scriptsize, fancyline, color=yellow!20,] {\begin{spacing}{.5}{\bf Kai:} Do we need to list these results? I think the most concise way is to just state Lemma \ref{test Hadamard derivative}\end{spacing}}
{\color{blue}{
The asymptotic distribution of
\eqref{test stat for a person against an estimated curve} can be derived by
combining
\eqref{delta method for testing a person against a known curve} with the respective influence functions	 $\psi_i$
derived in \eqref{main roc asymptotic formula under correct specification and parametric rate},
\eqref{main asymptotic formula under general parametric specification of model based roc}
and \eqref{main asymptotic formula under mis parametric specification} under the null hypothesis:
\begin{align}\begin{split}\nonumber
\sqrt{n}\left(
\hat\beta\left(\hat\alpha\right) - \hat\beta
\right)
\rightsquigarrow
\mathbb Q\left(
\psi_i + \beta'\left(\alpha\right)
\frac{\left(1-Y_i\right) \(W_i - \alpha\right)}{1 - q}
- \frac{Y_i \(W_i - \beta\right)}{q}
\right).
\end{split}\end{align}
Specifically, on the one hand, using
$\psi_i$ in \eqref{main roc asymptotic formula under correct specification and parametric rate} and \eqref{delta method for testing a person against a known curve},
\begin{align}\begin{split}\nonumber
\sqrt{n}\left(
\hat\beta\left(\hat\alpha\right) - \hat\beta
\right)
\rightsquigarrow
\mathbb Q\left(
\frac1q \frac{1}{1-k\left(\alpha\right)}
\(y_i - k\left(\alpha\right)\right)
\left(\mathds{1}\(p\(x_i\right) > k\left(\alpha\right)\right) - w_i\right)
\right).
\end{split}\end{align}
On the other hand, using $\psi_i$ of the model based estimator in
\eqref{main asymptotic formula under general parametric specification of model based roc}
and \eqref{delta method for testing a person against a known curve},
\begin{align}\begin{split}\nonumber
\sqrt{n}\left(
\hat\beta\left(\hat\alpha\right) - \hat\beta
\right)
\rightsquigarrow
&\mathbb{Q}\biggl[
\frac{
\psi\(x_i; \theta^*, \theta^*, \alpha\right)	 - \mathcal J\left(\theta^*,\alpha\right) H\left(\theta^*\right)^{-1} s_{\theta_1}\(y_i \vert x_i; \theta_1\right)}{
\mu\(p\(x; \theta^*\right)\right)
}\\
& +
\frac{1-q^*}{q^*} \frac{k\left(\mu, \theta^*, \alpha\right)}{1-
k\left(\mu, \theta^*, \alpha\right)}.
\frac{\left(1-y_i\right) \(w_i - \alpha\right)}{1 - q}
- \frac{y_i \(w_i - \beta\right)}{q}
\biggr].
\end{split}\end{align}
Finally, using the orthogonal parametric estimator $\psi_i$ in
\eqref{main asymptotic formula under mis parametric specification}
and \eqref{delta method for testing a person against a known curve},  for
$k^* = k\(Q, \theta^*, \alpha\right)$,
\begin{align}\begin{split}\nonumber
\sqrt{n}\left(
\hat\beta\left(\hat\alpha\right) - \hat\beta
\right)
\rightsquigarrow
&\mathbb{Q}\biggl[
\left(\frac{
\left(1-q\right) f_\alpha^*
+ q f_\beta^*
}{
q\left(1-q\right) f_\alpha^*} y_i  - \frac{1}{1-q}
\frac{
f_\beta^*
}{
f_\alpha^*
}\right) \left(\mathds{1}\(p\(x_i; \theta^*\right) > k^*\right) - w_i\right)
+ \frac{\mathcal G\(Q,\theta^*,\alpha\right)}{q} \kappa_i,
\biggr]
\end{split}\end{align}
where $f_\alpha^* =  f_\alpha\(Q, \theta^*, k\(Q, \theta^*, \alpha\right)\right)$ and
$f_\beta^* =  f_\beta\(Q, \theta^*, k\(Q, \theta^*, \alpha\right)\right)$.
}}
\end{comment}

\begin{comment}
\section{Sub (sup) and Fr\'{e}chet differentiability}\label{frechet differentiability section}
In this section, we provide a mathematical justification of defining
the value function of the optimal treatment choice (policy / individual treatment rule) as a \textit{social welfare potential function}.
Using the seminal convexity results of \cite{bolte2021conservative},
we show that
this value function satisfies the precise definition of a potential function.
We further demonstrate that the convexity of the social welfare potential function leads to a surprising generic Fr\'{e}chet differentiability property.  See also the comprehensive discussion in \cite{chen2021shape}.


\subsection{Social welfare potential function}

Given the convexity
in $\lambda$ given $g\left(\cdot\right)$,
a subgradient (see definition \ref{sub sup differentiability}) of $\gamma\left(\lambda, g\right)$ in $\lambda$ is given by
\begin{align}\begin{split}\label{one of the subgradients}
p\left(\lambda\right)
= \left(
\mathbb{E} \phi^*_0\(X; \lambda, g\right) g_0\(X\right), \ldots, \mathbb{E} \phi^*_J\(X; \lambda, g\right) g_J\(X\right)
\right)^T.
\end{split}\end{align}
where $\phi^{*}\left(\cdot; \lambda, g\right)$ is one of the optimal allocation under $\left(\lambda, g\right)$.
To verify by direct calcuation,
\begin{align}\begin{split}\nonumber
&\gamma\left(\lambda, g\right) + \left\langle p\left(\lambda\right), \lambda' - \lambda \right\rangle \\
&= \sum_j \mathbb{E} \lambda_j \phi_j^*\(X; \lambda, g\right) g_j\(X\right)
+ \sum_j \mathbb{E} \left(\lambda_j'-\lambda_j\right) \phi_j^*\(X; \lambda, g\right) g_j\(X\right) \\
&= \sum_j \mathbb{E} \lambda_j' \phi_j^*\(X; \lambda, g\right) g_j\(X\right)
\leq
\sum_j \mathbb{E} \lambda_j' \phi_j^*\(X; \lambda', g\right) g_j\(X\right)
= \gamma\left(\lambda', g\right).
\end{split}\end{align}
The rest of this subsection rigorously justifies calling $\gamma\left(\lambda, g\right)$ a social welfare {\it potential} function, with $p\left(\lambda\right)$ belonging to the vector field generated by
$\gamma\left(\lambda, g\right)$.
Recall that in a set-value map $D: \mathbb{R}^{J+1}\rightrightarrows \mathbb{R}^{J+1}$, the image of a point under the map is a set.
A function is then understood as a singleton valued set-valued map.
The following definition is needed:

\begin{definition}{(Conservative set-valued fields, \cite{bolte2021conservative})}\label{definition 1}
	Let $D: \mathbb{R}^p \rightrightarrows \mathbb{R}^p$ be a set-valued map. $D$ is a conservative (set-valued)
	field whenever it has closed graph, nonempty compact values, and for any
	absolutely continuous loop $\ell: \left[0,1\right] \to \mathbb{R}^p$, such that $\ell\left(0\right)=\ell\left(1\right)$, the Aumann integral of $t \rightrightarrows \left\langle\dot{\ell}\(t\right), D\left(\ell\(t\right)\right)\right\rangle$ is $\{0\}$, namely,
	\begin{align}\begin{split}\label{aumann integral}
	\int^{1}_{0} & \left\langle \dot{\ell}\(t\right), D\left(\ell\(t\right)\right)\right\rangle dt \\
    & \coloneqq
    \left\{\int_{0}^{1} \omega(t)dt: \omega\(t\right): [0, 1] \to \mathbb{R}
	\text{ is a measurable selection of}
	\left\langle\dot{\ell}\(t\right), D\left(\ell\(t\right)\right)\right\rangle\right\} = \{0\}.
\end{split}\end{align}
\end{definition}
It is shown in \cite{bolte2021conservative} that \eqref{aumann integral} is equivalent to
requiring
\begin{align}\begin{split}\nonumber
\int_0^1 \max_{v \in D\left(\ell\(t\right)\right)} \left\langle\dot{\ell}\(t\right), v\right\rangle dt = 0,
\end{split}\end{align}
where the integral is understood in the Lebesgue sense. This is possible by Theorem 18.19 and Theorem 18.20 in \cite{guide2006infinite}. See also Lemma 1 in
\cite{bolte2021conservative}.
Based on Definition \ref{definition 1}, we introduce the following Definition
\ref{definition 2}:
\begin{definition}{(Potential functions of conservative fields, \cite{bolte2021conservative})}\label{definition 2}
	A function $f: \mathbb{R}^p \to \mathbb{R}$ is called a potential function of a
	conservative field whenever there exists a conservative field $D: \mathbb{R}^p \rightrightarrows  \mathbb{R}^p$
    such that
    \begin{align}\begin{split}\nonumber
    f\(x\right) = f\left(0\right) + \int^{1}_{0} \left\langle \dot{\mathtt{p}}\(t\right), D\left(\mathtt{p}\(t\right)\right)\right\rangle dt
    \end{split}\end{align}
    for all absolutely continuous path $\mathtt{p}: \left[0, 1\right] \to \mathbb{R}^{p}$ with $\mathtt{p}\left(0\right) = 0$ and $\mathtt{p}\left(1\right) = x$.
	\end{definition}
\cite{bolte2021conservative} show that a potential function must be locally Lipschitz and their Theorem 1 states that if
$f$ is a potential function of the conservative field $D$ then $D$ coincides
with the gradient of $f$ on Lebesgue almost every point where the gradient exists. Therefore,
we expect that certain
nonsmooth functions will generate conservative fields
by some generalized approach of taking derivatives. In addition, a real-valued convex (or concave) function is locally Lipschitz and thus Lebesgue almost everywhere differentiable by the
Rademacher's theorem.
See for example
Theorem \ref{Rademacher theorem} in the technical addendum \ref{technical addendum}.

\begin{theorem}\label{theorem 1}
{\cite{bolte2021conservative} Corollary 2 and Proposition 2} \\
    Let $f:\mathbb{R}^p \to \mathbb{R}$ be a convex (or concave) function,
    then $\nabla^- f\left(\nabla^+ f\right) = \partial f$ is a conservative field that admits $f$ as its potential,
    where $\partial f$ is the Clarke subgradient defined in Definition \ref{definition 3}.
    Consequently,  $\gamma\left(\lambda, g\right)$ is a potential function in the sense of Definition \ref{definition 2}.
\end{theorem}

\subsection{Envelope like theorem for social welfare potential function}
\label{Envelope like theorem for social welfare potential function}

Consider, for a given $g\left(\cdot\right)$,
\begin{align}\begin{split}\nonumber
e\left(\lambda'; \phi^{*}\left(\cdot; \lambda, g\right)\right) = \mathbb{E} \left[\sum_j \lambda_j' \phi_j^{*} g_j\right] - \gamma\left(\lambda', g\right).
\end{split}\end{align}
By construction, $e\left(\lambda'; \phi^{*}\left(\cdot; \lambda, g\right)\right)$ achieves its maximum of zero
at $\lambda'=\lambda$.
When there is no arbitrariness, we also denote $\phi^{*}\left(\cdot\right) \coloneqq \phi^{*}\left(\cdot; \lambda, g\right)$.
Note first that
$\mathbb{E}\left[\sum_j \lambda_j' \phi_j^{*} g_j\right]$ is everywhere differentiable in $\lambda_j'$ given
$\phi^{*}\left(\cdot\right)$ and $g\left(\cdot\right)$.
Because of the convexity of $\gamma\left(\lambda', g\right)$ in $\lambda'$ given $g\left(\cdot\right)$, $\gamma\left(\lambda', g\right)$  is differentiable in $\lambda'$ given $g\left(\cdot\right)$ in a set $\Lambda$ such that its complement $\Lambda^c$ is a	Lebesgue null set.
Then for each $\lambda \in \Lambda$, and for each selection $\phi^{*}$ given this choice of
$\lambda$, $e\left(\lambda'; \phi^{*}\right)$ is differentiable in $\lambda'$ at $\lambda'=\lambda$.
Since zero must be in the set of supgradients
at a point of maximum of any function, there is $0 \in \nabla^+ e\left(\lambda'; \phi^{*}\right)\big\vert_{\lambda'=\lambda}$ and thus
$\nabla e\left(\lambda'; \phi^{*}\right)\big\vert_{\lambda'=\lambda} = 0$,
implying
\begin{align}\begin{split}\nonumber
\nabla \gamma\left(\lambda', g\right)\Bigg\vert_{\lambda'=\lambda}
= \nabla \mathbb{E} \left[\sum_j \lambda_j' \phi_j^{*} g_j\right]\Bigg\vert_{\lambda'=\lambda}
= \left(\mathbb{E} \phi_j^{*} g_j, j=0,\ldots,J\right)^{T}.
\end{split}\end{align}
An additional implication of the previous result is that for each $\lambda\in\Lambda$,
$\phi^{*}\(x; \lambda, g\right)$ is unique in an almost surely sense since the expectation
$\left(\mathbb{E} \phi_j^{*} g_j, j=0,\ldots,J\right)^{T}$ is uniquely defined.

We expect such envelope theorem like statement remains true with
functional derivatives of
\begin{align}\begin{split}\label{auxiliary function}
e\left(\lambda', g'; \phi^{*}\right) = \mathbb{E}\left[\sum_j \lambda_j' \phi_j^{*} g_j'\right]-\gamma\left(\lambda', g'\right)
\end{split}\end{align}
at $\left(\lambda', g'\right) = \left(\lambda, g\right)$ (here, $\phi^{*}\left(\cdot\right) \coloneqq \phi^{*}\left(\cdot; \lambda, g\right) $).
This conjecture is verified in Theorem
\ref{theorem envelope Theorem for social welfare potential function} which
shows Fr\'{e}chet differentiability.

\begin{definition}{(Fr\'{e}chet differentiability)}
Let $\mathcal{X}$ and $\mathcal{Y}$ be normed spaces equipped with norm $\Vert\cdot\Vert_{\mathcal{X}}$ and $\Vert\cdot\Vert_{\mathcal{Y}}$, $E \subset \mathcal{X}$ be an open set. Consider the map $\xi: E \to Y$. Then $\xi$ is called Fr\'{e}chet differentiable at $\theta \in E$, if there is a continuous linear map $\xi'_{\theta}: \mathcal{X} \to \mathcal{Y}$ such that:
\begin{align}\begin{split}\nonumber
\left\Vert \xi\(x\right) - \xi\left(\theta\right) - \xi'_{\theta}\(x - \theta\right)
\right\Vert_{\mathcal{Y}} = o\left(\Vert x - \theta\Vert_{\mathcal{X}}\right).
\end{split}\end{align}
\end{definition}

Before we describe the next result we recall several topological and measure theoretic concepts. In a topological space $\mathcal X$, if a subset $S \in \mathcal X$ satisfies
$\overset{\circ}{\bar S} =\emptyset$, i.e. the closure of $S$ has empty interior, then $S$ is called
a nowhere dense subset of $\mathcal X$. If $\mathcal Z \subset \mathcal X$ is a countable union of
nowhere dense subsets of $\mathcal X$, then $\mathcal Z$ is called a meager subset of
$\mathcal X$, or
of the first category in $\mathcal X$. A subset $A \subset \mathcal X$ is called residually
many (or just residual) in $\mathcal X$ if $\mathcal X \setminus A$ is meager in $\mathcal X$.
A set $B \subset \mathcal X$ is called a $G_\delta$ set of $\mathcal X$ if $B$ is a
countable intersection of open sets; it is call a $F_\sigma$ set if it is a countable union
of closed sets.

\cite{lindenstrauss2003frechet} originate the concept of $\Gamma$-null sets to
describe the {\it magnitude} of sets.
Let $T = \left[0,1\right]^{\mathbb{N}}$, where
$\mathbb{N}$ is the set of natural numbers, be endowed with the product topology and product
Lebesgue measure $\mathcal{L}_{\infty}$.
Let $\mathcal X$ be a Banach space and
$\Gamma\left(\mathcal X\right)=\{\gamma: T \to \mathcal X\}$
be the space of continuous mappings having continuous partial derivatives $D_j\gamma$.	  Equip
$\Gamma\left(\mathcal X\right)$ with a topology generated by
$\vert\vert \gamma \vert\vert_0 = \sup_{t\in T} \vert\vert \gamma\(t\right)\vert\vert$, and all
$\vert\vert \gamma \vert\vert_k = \sup_{t\in T} \vert\vert D_k \gamma\(t\right)\vert\vert$,
$k \geq 1$. Equivalently, the same topology is generated by
all $\vert\vert\gamma\vert\vert_{\leq k} = \max_{0 \leq i \leq k} \vert\vert \gamma\vert\vert_i, k \in \mathbb{N}$.

\begin{definition}{($\Gamma$-null)}
	A Borel set $N \subset \mathcal X$ is called $\Gamma$-null if $\mathcal{L}_{\infty}\{t \in T: \gamma\(t\right)
	\in N\} = 0$ for residually many $\gamma \in \Gamma\left(\mathcal X\right)$. If a set $A$ is
	contained in such a $N$, then $A$ is also called $\Gamma$-null.
\end{definition}

Now, we can state our result. In the following theorem statement, \ref{envelope theorem 1} and
\ref{envelope theorem 2} follow from the convexity of $\gamma\left(\lambda, g\right)$ in $\lambda$ given each $g$,
and the convexity of $\gamma\left(\lambda, g\right)$ in $g$ given each $\lambda$, while
\ref{envelope theorem 3}  accounts for the non joint-convexity
of $\gamma\left(\lambda, g\right)$ in $\left(\lambda, g\right)$.

\begin{theorem}
\label{theorem envelope Theorem for social welfare potential function}
	Let $\gamma\left(\lambda, g\right)$ be a real valued social welfare potential function, where $\lambda\in \mathbb{R}^{J+1}$, $g \in \mathcal X$ where $\mathcal X$ is a Banach space. Assume that $\gamma\left(\lambda, g\right)$ is continuous at a point $\left(\bar{\lambda}, \bar{g}\right) \in \mathbb{R}^{J+1} \otimes \mathcal X$,	then we have:
	\begin{enumerate}
		\item\label{envelope theorem 1} If every separable subspace $\mathcal Y$ of $\mathcal X$ has a separable
			dual space $\mathcal Y^*$,
	then		for any given $\lambda \in
\mathbb{R}^{J+1}$, $\gamma\left(\lambda, g\right)$ is Fr\'{e}chet
			differentiable in $g$ on a dense
			$G_\delta$ subset of  $\mathcal X$.
		\item \label{envelope theorem 2} If the
			dual space $\mathcal X^*$ of $\mathcal X$ is separable, then
for any given $\lambda \in   \mathbb{R}^{J+1}$,
			$\gamma\left(\lambda, g\right)$ is $\Gamma$-almost everywhere Fr\'{e}chet differentiable
			with respect to $g \in \mathcal X$. In other words,
for any given $\lambda \in   \mathbb{R}^{J+1}$,
$\gamma\left(\lambda, g\right)$ is not Fr\'{e}chet differentiable in $g$ on a $\Gamma$-null set of $\mathcal X$.
\item\label{envelope theorem 3} In addition, If every separable subspace $\mathcal Y$ of $\mathcal X$ has a separable
			dual space $\mathcal Y^*$, and $\mathcal{X}$ can be continuously
	embedded into $L^1\left(\Omega, \mu\right)$ (in the sense that
for all $x \in \mathcal X$,
			$\Vert x\Vert_{L^1\left(\Omega, \mu\right)} \leq C \Vert x \Vert_{\mathcal X}$ for a constant $C$),
   then $\gamma\left(\lambda, g\right)$ is locally jointly Lipschitz in $\left(\lambda, g\right)$, and is also jointly Fr\'{e}chet differentiable with respect to $\left(\lambda, g\left(\cdot\right)\right)$ on a dense subset of $\mathbb{R}^{J+1} \otimes \mathcal X$.
	\end{enumerate}
When $\gamma\left(\lambda, g\right)$ is Fr\'{e}chet differentiable (totally or partially as in the theorem),
the Fr\'{e}chet derivative with respect to $\left(\lambda, g\right)$ can be calculated to be
\begin{align}\begin{split}\label{Frechet derivative}
\left(\left(\mathbb{E}\phi^{*}_{0}g_{0}, \ldots, \mathbb{E}\phi^{*}_{J}g_{J}\right),
\left(\mathbb{E}\lambda_{0}\phi^{*}_{0}, \ldots, \mathbb{E}\lambda_{J}\phi^{*}_{J}\right)\right)^{T}
\end{split}\end{align}
where
\begin{align}\begin{split}\nonumber
\phi^{*}\left(\cdot\right) \coloneqq \phi^{*}\left(\cdot; \lambda, g\right)
= \mathop{\arg\max}\limits_{\phi\left(\cdot\right) \in \Phi} \mathbb{E} \left[\sum_j \lambda_j \phi_j\(X\right) g_j\(X\right)\right]
\end{split}\end{align}
in the $\mu$ almost surely sense, where $\mu$ is the distribution of $X$.
\end{theorem}

Theorem \ref{theorem envelope Theorem for social welfare potential function} essentially shows that,
under mild conditions, a generic point\footnote{ A generic property is one that holds
on a residual set.}
	can support
	taking the Fr\'{e}chet derivative of $\gamma\left(\lambda, g\right)$.
The conditions for
Fr\'{e}chet differentiability
originate from the partial convexity of $\gamma\left(\lambda, g\right)$ in either $\lambda$ or $g$ and the joint local Lipschitz property
of $\gamma\left(\lambda, g\right)$ in $\left(\lambda, g\right)$.
In \cite{chernozhukov2018sorted},
Hadamard differentiability is verified by direct calculations with great details of the
partial effect quantile function (called sorted effect therein). We have demonstrated how to reformulate their approach
in section \ref{Weighted sorted effect}. Compared with \cite{chernozhukov2018sorted},
the assumptions in
Theorem \ref{theorem envelope Theorem for social welfare potential function}
are not more restrictive in several senses.
First, Hadamard differentiability implies continuity (See for example \cite{averbuh1968razlivcnye}) which is assumed
in Theorem \ref{theorem envelope Theorem for social welfare potential function}. Second, it is
well known that for a measure space $\left(\Omega, \mathcal F, \mu\right)$ such that $\mathcal F$ is separable or countably generated,
and $\mu$ is $\sigma$-finite, $L^p\left(\Omega\right)$ is separable for $1 < p < \infty$, which implies that
$L^p\left(\Omega\right)$ is separable and reflexive when $\Omega$ is a Polish space, $\mathcal F$ is the Borel $\sigma$-algebra,
and $\mu$ is a probability measure.
Therefore $L^p\left(\Omega\right)$ for $1 < p < \infty$ satisfies the second result of Theorem \ref{theorem envelope Theorem for social welfare potential function}.
Furthermore, when $\Omega$ is an open subset of $\mathbb{R}^n$ and $\mathcal F$ is the
$\sigma$-algebra of Lebesgue measure, and $\mu = \mathcal{L}_{n}$ the $n$-dimensional Lebesgue measure,
both $L^p\left(\Omega\right)$ and the Sobolev space $W^{k,p}\left(\Omega\right)$ are also separable and reflexive. Hence
they also satisfy the second result of	Theorem \ref{theorem envelope Theorem for social welfare potential function}.
Third, we do not directly make assumptions about the underlying distribution $\mu$, such as requiring compact support or having a continuous density function.
\end{comment}

\section{Fast convergence rates}\label{fast convergence rates}

In this section we build on the previous analysis to generalize the fast learning rates for plug-in classifiers in
\cite{devroye1996probabilistic} and \cite{audibert2007fast} to multiclassification and multi-discrete allocation problems.
The fast learning rate phenomena can also be found in recent works by \cite{semenova2023debiased} and \cite{liu2024inference}.
Following \cite{chernozhukov2018double}, \cite{chernozhukov2022locally} and \cite{foster2023orthogonal},
Neyman-orthogonalization and sampling splitting can be combined with the plug-in scheme to facilitate
statistical inference.

\begin{comment}
\subsection{Second order Hadamard differentiability}


The last section shows that generically
\begin{align}\begin{split}\nonumber
\gamma\left(\lambda, g + \theta\right) = \gamma\left(\lambda, g\right) + \left\langle \theta, \left(\mathbb{E}\lambda_{0}\phi^{*}_{0}, \ldots, \mathbb{E}\lambda_{J}\phi^{*}_{J}\right)^{T}\right\rangle + o\left(\Vert \theta\Vert\right)
\end{split}\end{align}
under very weak assumptions.
By the construction, we know the reminder term in the above is
\begin{align}\begin{split}\nonumber
\mathbb{E}\left[\sum_{j}\lambda_{j}\left(\phi^{*}\(X; \lambda, g + \theta\right) - \phi^{*}\(X; \lambda, g\right)\right)\(g_{j}\(X\right) + \theta\right)\right].
\end{split}\end{align}
To further investigate how small can this term be, we first turn back to the Hadamard differentiability framework in section \ref{Weighted sorted effect}.
Consider the social welfare potential estimation problem under full observability.
Writing in the Hadamard differentiability style, we have the decomposition:

\begin{align}\begin{split}\nonumber
\end{split}\end{align}

\begin{definition}{(Second order Hadamard differentiability)}
Let $\mathcal{X}$ and $\mathcal{Y}$ be two normed spaces equipped with norms $\Vert\cdot\Vert_{\mathcal{X}}$ and $\Vert\cdot\Vert_{\mathcal{Y}}$.
Then a map $\phi: \mathcal{X}_{\phi} \subset \mathcal{X} \to \mathcal{Y}$ is called second order Hadamard differentiable at $\theta \in \mathcal{X}_{\phi}$ tangentially to $\mathcal{X}_{0} \subset \mathcal{X}$ if
\begin{enumerate}[(i)]
  \item $\phi$ is (first order) Hadamard differentiable at $\theta$ tangentially to $\mathcal{X}_{0}$, and the derivative $\phi'_{\theta}: \mathcal{X}_{0} \to \mathcal{Y}$ is well defined on $\mathcal{X}$.
  \item There exists a continuous bilinear map $\Phi''_{\theta}: \mathcal{X}_{0} \times \mathcal{X}_{0} \to \mathcal{Y}$ such that, for $\phi''_{\theta}\(x\right) \coloneqq \Phi''_{\theta}\(x, x\right)$,
      \begin{align}\begin{split}\nonumber
      \left\Vert
      \frac{\phi\left(\theta + t_{n}x_{n}\right) - \phi\left(\theta\right) - t_{n}\phi'_{\theta}\(x_{n}\right) - t^{2}_{n}\phi''_{\theta}\(x\right)}
      {t^{2}_{n}}\right\Vert_{\mathcal{Y}} \to 0
      \end{split}\end{align}
      for all $t_{n} \to 0$ and $x_{n} \to x \in \mathcal{X}_{0}$, as $n \to \infty$, where $\theta + t_{n}x_{n} \in \mathcal{X}_{\phi}$ for all $n$.
\end{enumerate}
\end{definition}

\begin{proposition}
\end{proposition}

The original margin assumption can be extended to the following:

\begin{definition}(Marginally controlled random variable / distribution function)
Let $X$ be a random on $\[a, b\right]$, where $a < b$, $a, b \in \mathbb{R}$.
If for all $c \in \[a, b\right]$,
\begin{align}\begin{split}\nonumber
\mathbb{P}\left\{\left\vert X - c\right\vert < t\right\} \leq C\(c\)t^{\alpha},
\end{split}\end{align}
where $-\infty < C\(c\right) < +\infty$ may depends on c, we call $X$ a marginally controlled random variable of order $\alpha$.
Similarly, if $F$ is the distribution function of such a random variable, we call it a marginally controlled distribution function.
\end{definition}

\begin{lemma}\label{marginally controlled lemma}
If $X$ is a marginally controlled random variable on $\[a, b\right]$ of order $1$, where $a < b$, $a, b \in \mathbb{R}$, then $X$ is absolutely continuous with respect to Lebesgue measure. Consequently, $X$ admits a density function.
\end{lemma}

\begin{lemma}\label{density iff}
Let $f \in L^{1}\left(\mathbb{R}^{n}, \mathcal{L}_{n}\right)$ be a nonnegative function and $h_{j}: \Omega \to \mathbb{R}$ be a $C^{n}$ ($W^{n, p}$, $p > n$) function, $\Omega \subset \mathbb{R}^{n}$ be an open subset.
Assume $X$ is a random variable with $f$ as its density function.
Then $h_{j}\(X\right)$ also admit a density, i.e. the push-forward probability $h_{j\#}\(f\mathcal{L}_{n}\right) \ll \mathcal{L}_{1}$ if and only if $Jh_{j} > 0 \; a.e.$ on the support of $f$ and in this case the density of $h_{j}\(X\right)$ is
\begin{align}\begin{split}\nonumber
f_{h_{j}}\(y\right) = \int_{h^{-1}_{j}\(y\right)}\frac{f\(x\right)}{Jh_{j}\(x\right)}d\mathcal{H}_{n - 1}x.
\end{split}\end{align}
\end{lemma}
\end{comment}

\subsection{Second order Hadamard differentiability}\label{second order Hadamard differentiability}

Our generalizations are motivated by the partial convexity of
the social welfare potential function $\gamma\left(\cdot, \cdot\right)$.
Consider, for a given $g\left(\cdot\right)$,
\begin{align}\begin{split}\nonumber
e\left(\lambda'; \phi^{*}\left(\cdot; \lambda, g\right)\right) = \mathbb{E} \left[\sum_j \lambda_j' \phi_j^{*} g_j\right] - \gamma\left(\lambda', g\right).
\end{split}\end{align}
By construction, $e\left(\lambda'; \phi^{*}\left(\cdot; \lambda, g\right)\right)$ achieves its maximum of zero
at $\lambda'=\lambda$.
When there is no ambiguity, we also denote $\phi^{*}\left(\cdot\right) \coloneqq \phi^{*}\left(\cdot; \lambda, g\right)$.
Continuous convex function is locally Lipschitz and (locally) Lipschitz function is differentiable almost
everywhere by the Rademacher theorem \ref{Rademacher theorem}.
Due to the convexity of $\gamma\left(\lambda', g\right)$ in $\lambda'$ given $g\left(\cdot\right)$, $\gamma\left(\lambda', g\right)$  is differentiable in $\lambda'$ given $g\left(\cdot\right)$ in a set $\Lambda$ such that its complement $\Lambda^c$ is a	Lebesgue null set.
Then for each $\lambda \in \Lambda$ and for each selection $\phi^{*}$ given this choice of
$\lambda$, $e\left(\lambda'; \phi^{*}\right)$ is differentiable in $\lambda'$ at $\lambda'=\lambda$.
Since zero must be in the set of supgradients
at a point of maximum of any function, there is $0 \in \nabla^+ e\left(\lambda'; \phi^{*}\right)\big\vert_{\lambda'=\lambda}$ and thus
$\nabla e\left(\lambda'; \phi^{*}\right)\big\vert_{\lambda'=\lambda} = 0$,
implying that
\begin{align}\begin{split}\nonumber
\nabla \gamma\left(\lambda', g\right)\Bigg\vert_{\lambda'=\lambda}
= \nabla \mathbb{E} \left[\sum_j \lambda_j' \phi_j^{*} g_j\right]\Bigg\vert_{\lambda'=\lambda}
= \left(\mathbb{E} \phi_j^{*} g_j, j=0,\ldots,J\right)^{T}.
\end{split}\end{align}

We expect such envelope theorem like statement remains true with
functional derivatives of
\begin{align}\begin{split}\label{auxiliary function}
e\left(\lambda', g'; \phi^{*}\right) = \mathbb{E}\left[\sum_j \lambda_j' \phi_j^{*} g_j'\right]-\gamma\left(\lambda', g'\right)
\end{split}\end{align}
at $\left(\lambda', g'\right) = \left(\lambda, g\right)$ (here, $\phi^{*}\left(\cdot\right) \coloneqq \phi^{*}\left(\cdot; \lambda, g\right) $).
Theorem
\ref{theorem envelope Theorem for social welfare potential function} shows
Fr\'{e}chet differentiability and verifies \eqref{auxiliary function}.
Generically, under weak assumptions,
\begin{align}\begin{split}\nonumber
\gamma\left(\lambda, g + \theta\right) = \gamma\left(\lambda, g\right) +
\sum_{j}\mathbb{E}\lambda_{j}\phi^{*}_{j}\theta_{j}
+ o\left(\Vert \theta\Vert\right).
\end{split}\end{align}
By construction, the remainder term of $o\left(\Vert \theta\Vert\right)$ in the above expression is
\begin{align}\begin{split}\label{reminder term Frechet}
\mathbb{E}\left[\sum_{j}\lambda_{j}\left(\phi^{*}_j\(X; \lambda, g + \theta\right) - \phi^{*}_j\(X; \lambda, g\right)\right)\(g_{j}\(X\right) + \theta_{j}\right)\right].
\end{split}\end{align}
The magnitude of this term can be investigated by relating
the Hadamard differentiability framework in section \ref{Weighted sorted effect} to the margin assumption (MA) in the plug-in classifier literature.

Consider the estimation of social welfare potential  function under full observability.
Let $\mu$ denote the distribution of $X$ and  $Q$ the joint distribution of $(y,x).$
Define the following notation:
\begin{align}\begin{split}\nonumber
\gamma\left(\lambda, g(\cdot),\mu\right) & = \mu\left[\sum_{j}\lambda_{j}\left(\prod_{l \neq j}1\left(\lambda_{j}g_{j}\(x\right) > \lambda_{l}g_{l}\(x\right)\right)\)g_{j}\(x\right)\right], \\
\gamma\left(\lambda, g(\cdot),Q\right) & = Q\left[\sum_{j}\lambda_{j}\left(\prod_{l \neq j}1\left(\lambda_{j}g_{j}\(x\right) > \lambda_{l}g_{l}\(x\right)\right)\)y_{j}\right].
\end{split}\end{align}

To contrast the application of
measure $\mu$ with that of measure
$Q$, we first consider the perturbation of $\gamma\left(\lambda, g(\cdot),\mu\right)$
to
$\gamma\left(\lambda, g(\cdot)+t_n\,H_n(\cdot),\mu+t_n{\mathcal P}_n\right)$ where $t_n$
approaches zero and $H_n(\cdot)$
and ${\mathcal P}_n$ provide directions of the perturbation of the conditional expectation function $g(\cdot)$
and
of the measure $\mu\left(\cdot\right)$ over ${\mathcal X},$ respectively.
We consider the following decomposition:
\begin{align}\begin{split}\nonumber
\gamma\left(\lambda, g + t_{n}H_{n}, \mu + t_{n}\mathcal{P}_{n}\right) - \gamma\left(\lambda, g, \mu\right)
& = \underbrace{t_{n}\left(\gamma\left(\lambda, g + t_{n}H_{n}, \mathcal{P}_{n}\right) - \gamma\left(\lambda, g,
\mathcal{P}_{n}\right)\right) }_{\left(1\right)} \\
&\hspace{-.5in} + \underbrace{\gamma\left(\lambda, g + t_{n}H_{n}, \mu\right) - \gamma\left(\lambda, g, \mu\right)}_{\left(2\right)}
 + \underbrace{\gamma\left(\lambda, g, \mu + t_{n}\mathcal{P}_{n}\right) - \gamma\left(\lambda, g, \mu\right)}_{\left(3\right)}.
\end{split}\end{align}

The first term $\left(1\right)$ is the second order difference ``empirical process'' term that should vanish under suitable continuity conditions.
The third term $\left(3\right)$ is a standard ``summation of random variables'' term.
The second term $\left(2\right)$ can be further decomposed as
\begin{align}\begin{split}\nonumber
\left(2\right) & = \underbrace{\mu\left[\sum_{j}\lambda_{j}\left(\prod_{l \neq j}1\left(\lambda_{j}\(g_{j} + t_{n}H_{nj}\right) > \lambda_{l}\(g_{l} + t_{n}H_{nl}\right)\right) - \prod_{l \neq j}1\left(\lambda_{j}g_{j} > \lambda_{l}g_{l}\right)\)g_{j}\right]}_{\textcircled{\footnotesize{1}}} \\
& + \underbrace{t_{n}\mu\left[\sum_{j}\lambda_{j}\left(\prod_{l \neq j}1\left(\lambda_{j}g_{j} > \lambda_{l}g_{l}\right)\)H_{nj}\right]}_{\textcircled{\footnotesize{2}}} \\
& + \underbrace{t_{n}\mu\left[\sum_{j}\lambda_{j}\left(\prod_{l \neq j}1\left(\lambda_{j}\(g_{j} + t_{n}H_{nj}\right) > \lambda_{l}\(g_{l} + t_{n}H_{nl}\right)\right) - \prod_{l \neq j}1\left(\lambda_{j}g_{j} > \lambda_{l}g_{l}\right)\)H_{nj}\right]}_{\textcircled{\footnotesize{3}}},
\end{split}\end{align}
where for brevity we omit the argument of functions $g(\cdot)$ and $H(\cdot)$.
If we use $\gamma\left(\lambda, g, Q\right)$ instead of $\gamma\left(\lambda, g, \mu\right)$, then we will not have \textcircled{\footnotesize{2}} and \textcircled{\footnotesize{3}} terms.
In this sense, using $Y$ instead of $g$ provides a natural Neyman-orthogonalization of the estimator.
We show
that under Hadamard differentiability, the term \textcircled{\footnotesize{1}} is $o\(t_n\right)$,
leading to a simplified form of the Hadamard derivative of $\gamma\left(\lambda, g, Q\right)$.

\begin{lemma}\label{social welfare potential first order derivative lemma}
Consider the following class of functions indexed by $g \in \mathcal{G} \subset C\(E, \mathbb{R}^{J + 1}\right)$,
\begin{align}\begin{split}\nonumber
\mathcal{F} \coloneqq \left\{\(x^{T}, y^{T}\right)^{T} \mapsto \(y_{0}\left(\prod_{l \neq 0}1\left(\lambda_{0}g_{0}\(x\right) > \lambda_{l}g_{l}\(x\right)\right)\right), \ldots, y_{J}\left(\prod_{l \neq J}1\left(\lambda_{J}g_{J}\(x\right) > \lambda_{l}g_{l}\(x\right)\right)\right)\right)^{T} \right\}
\end{split}\end{align}
where $y$ is bounded. Let $\mathbf{Q} \subset \ell^{\infty}\left(\mathcal{F}\right)$ consists of elements that are linear and uniformly continuous on $\mathcal{F}$ with respect to the $L^{2}\(Q\right)$ norm.
Assume also that the conditions of Theorem \ref{general k derivative} hold for
each $\Delta_{j} = \lambda_{j}g_{j} - \left(\lambda_{1}g_{1}, \ldots, \lambda_{j - 1}g_{j - 1}, \lambda_{j + 1}g_{j + 1}, \ldots, \lambda_{J}g_{J}\right)^{T}$, $j = 0, \ldots, J$ and for $\mu$ where $X \sim \mu$.
Then the Hadamard derivative of $\gamma\left(\lambda, \cdot, \cdot\right)$ at $\(g, Q\right)$ tangentially to $  C\(E, \mathbb{R}^{J + 1}\right) \times \mathbf{Q}$ is
\begin{align}\begin{split}\label{social welfare potential derivative}
\gamma'_{\lambda, g, Q}\left(0, H, \mathcal{Q}\right) = \mathcal{Q}\left[\sum_{j}\lambda_{j}\left(\prod_{l \neq j}1\left(\lambda_{j}g_{j}\(x\right) > \lambda_{l}g_{l}\(x\right)\right)\)y_{j}\right].
\end{split}\end{align}
\end{lemma}

\begin{comment}
\begin{proof}
Note that \eqref{social welfare potential derivative} is the limit of $\frac{\left(3\right)}{t_{n}}$ as $n \to \infty$, so we only need to show that $\lim_{n \to \infty}\frac{\left(1\right)}{t_{n}} = \lim_{n \to \infty}\frac{\left(2\right)}{t_{n}} = 0$.
For the $\left(1\right)$ term, we further decompose
\begin{align}\begin{split}\nonumber
\left\vert \gamma\left(\lambda, g + t_{n}H_{n}, \mathcal{Q}_{n}\right) - \gamma\left(\lambda, g, \mathcal{Q}_{n}\right)\right\vert
& \leq \left\vert \gamma\left(\lambda, g + t_{n}H_{n}, \mathcal{Q}_{n}\right) - \gamma\left(\lambda, g + t_{n}H_{n}, \mathcal{Q}\right)\right\vert \\
& + \left\vert \gamma\left(\lambda, g, \mathcal{Q}_{n}\right) - \gamma\left(\lambda, g, \mathcal{Q}\right)\right\vert
+ \left\vert \gamma\left(\lambda, g + t_{n}H_{n}, \mathcal{Q}\right) - \gamma\left(\lambda, g, \mathcal{Q}\right)\right\vert.
\end{split}\end{align}
The first two terms converge to $0$ since $\left\Vert \mathcal{Q}_{n} - \mathcal{Q}\right\Vert_{\ell^{\infty}\left(\mathcal{F}\right)} \to 0$.
The third term also converges to $0$ by the $L^{2}\(Q\right)$ uniform continuity of $\mathcal{Q}$, since by Proposition \ref{surface integral continuity}, $g\(X\right)$ has a density function.

By Theorem \ref{general k derivative} and the chain rule of Hadamard derivative, we have
\begin{align}\begin{split}\label{social welfare potential boundary derivative}
\lim_{n \to \infty}\frac{\left(2\right)}{t_{n}} = \sum_{j}\left\{\sum_{l \neq j}\int_{y > \tau_{\neg l}\left(0\right)}\left[\int_{\Delta^{-1}_{j}\(c'\(y, 0, l\right)\right)}\frac{\left(\lambda_{j}H_{j}\(x\right) - \lambda_{l}H_{l}\(x\right)\)f\(x\right)\lambda_{j}g_{j}\(x\right)}{J\Delta_{j}\(x\right)}
d\mathcal{H}_{n - k}x\]d\mathcal{L}_{k - 1}y\right\},
\end{split}\end{align}
where $f$ is the density function of $X$ and $\Delta_{j} = \lambda_{j}g_{j} - \left(\lambda_{1}g_{1}, \ldots, \lambda_{j - 1}g_{j - 1}, \lambda_{j + 1}g_{j + 1}, \ldots, \lambda_{k}g_{k}\right)^{T}$.
Consider a Borel set $C \subset \mathbb{R}$, note that $J\Delta_{j}\(x\right) > 0 \; a.e.$ on the support of $X$, there is
\begin{align}\begin{split}\label{another expression}
& \int_{c_{l} \in C}\left[\int_{y > \tau_{\neg l}\left(0\right)}\left[\int_{\Delta^{-1}_{j}\(c'\(y, c_{l}, l\right)\right)}\frac{\left(\lambda_{j}H_{j}\(x\right) - \lambda_{l}H_{l}\(x\right)\)f\(x\right)\lambda_{j}g_{j}\(x\right)}{J\Delta_{j}\(x\right)}
d\mathcal{H}_{n - k}x\]d\mathcal{L}_{k - 1}y\]dc_{l} \\
& = \int_{\tau_{\neg l}\(y\right) > \tau_{\neg l}\left(0\right), \tau_{l}\(y\right) \in C}\left[\int_{\Delta^{-1}_{j}\(y\right)}\frac{\left(\lambda_{j}H_{j}\(x\right) - \lambda_{l}H_{l}\(x\right)\)f\(x\right)\lambda_{j}g_{j}\(x\right)}{J\Delta_{j}\(x\right)}d\mathcal{H}_{n - k}x\]d\mathcal{L}_{k}y \\
& = \int\left(\prod_{i \neq j, l}1\left(\lambda_{j}g_{j}\(x\right) > \lambda_{i}g_{i}\(x\right)\right)\right)1\left(\lambda_{j}g_{j}\(x\right) - \lambda_{l}g_{l}\(x\right) \in C\right)\left(\lambda_{j}H_{j}\(x\right) - \lambda_{l}H_{l}\(x\right)\)f\(x\right)\lambda_{j}g_{j}\(x\)d\mathcal{L}_{n}x \\
& = \int_{y \in C}\left[\int_{\Delta^{-1}_{j, l}\(y\right)}\frac{\left(\lambda_{j}H_{j}\(x\right) - \lambda_{l}H_{l}\(x\right)\)f\(x\right)\lambda_{j}g_{j}\(x\right)}{J\Delta_{j, l}\(x\right)}\left(\prod_{i \neq j, l}1\left(\lambda_{j}g_{j}\(x\right) > \lambda_{i}g_{i}\(x\right)\right)\right)\]d\mathcal{L}_{1}y,
\end{split}\end{align}
where $\Delta_{j, l} = \lambda_{j}g_{j} - \lambda_{l}g_{l}$.
Here we use the coarea formula two times.
By the arbitrariness of $C$, the right hand side of \eqref{another expression} and \eqref{derivative k>1} (without the summation) coincide.
So, switch the role of $j$ and $l$ in the summation of \eqref{social welfare potential boundary derivative}, we get $\lim_{n \to \infty}\frac{\left(2\right)}{t_{n}} = 0$ by the fact that $\lambda_{j}g_{j}\(x\right) = \lambda_{l}g_{l}\(x\right)$ on $\Delta^{-1}_{j, l}\left(0\right)$ (and $\Delta^{-1}_{l, j}\left(0\right)$).
\end{proof}
\end{comment}

Compare Lemma \ref{social welfare potential first order derivative lemma} with Theorem \ref{theorem envelope Theorem for social welfare potential function}.
By \eqref{Frechet derivative} in Theorem \ref{theorem envelope Theorem for social welfare potential function},
the first order influence of pertubing
the second variable $g$
will not come from the indicator term.
Equation \eqref{social welfare potential derivative} in Lemma \ref{social welfare potential first order derivative lemma} derives an analogous Hadamard differentiability result.
Neyman-orthogonalization changes the remainder term \eqref{reminder term Frechet} to
$\mathbb{E}\left[\sum_{j}\lambda_{j}\left(\phi^{*}_{j}\(X; \lambda, g + \theta\right) - \phi^{*}_{j}\(X; \lambda, g\right)\)g_{j}\(X\right)\right].$
Next, we   show under a strict MA in \eqref{marginal assumption MA} with $\alpha > 1$, this term is second order degenerate.

\begin{definition}{(Second order Hadamard differentiability)}
Let $\mathcal{X}$ and $\mathcal{Y}$ be two normed spaces equipped with norms $\Vert\cdot\Vert_{\mathcal{X}}$ and $\Vert\cdot\Vert_{\mathcal{Y}}$.
A map $\phi: \mathcal{X}_{\phi} \subset \mathcal{X} \to \mathcal{Y}$ is called second order Hadamard differentiable at $\theta \in \mathcal{X}_{\phi}$ tangentially to $\mathcal{X}_{0} \subset \mathcal{X}$ if
\begin{enumerate}[(i)]
  \item $\phi$ is (first order) Hadamard differentiable at $\theta$ tangentially to $\mathcal{X}_{0}$ and the derivative $\phi'_{\theta}: \mathcal{X}_{0} \to \mathcal{Y}$ is
well defined on $\mathcal{X}$;
  \item there exists a continuous bilinear map $\Phi''_{\theta}: \mathcal{X}_{0} \times \mathcal{X}_{0} \to \mathcal{Y}$ such that for $\phi''_{\theta}\(x\right) \coloneqq \Phi''_{\theta}\(x, x\right)$,
      \begin{align}\begin{split}\nonumber
      \left\Vert
      \frac{\phi\left(\theta + t_{n}x_{n}\right) - \phi\left(\theta\right) - t_{n}\phi'_{\theta}\(x_{n}\right) - t^{2}_{n}\phi''_{\theta}\(x\right)}
      {t^{2}_{n}}\right\Vert_{\mathcal{Y}} \to 0
      \end{split}\end{align}
      for all $t_{n} \to 0$ and $x_{n} \to x \in \mathcal{X}_{0}$, as $n \to \infty$, where $\theta + t_{n}x_{n} \in \mathcal{X}_{\phi}$ for all $n$.
\end{enumerate}
\end{definition}

In the following we adopt a convention for the optimal policy in \eqref{reminder term Frechet}
 such that all the weight among the maximal indexes is allocated to the smallest index member:
\begin{align}\begin{split}\label{tie breaking definition to ensure at least at 1}
\phi^{*}_j\(x; \lambda, g\right) \coloneqq \prod_{l=j+1}^J 1\left(\lambda_j g_j\(x\right) \geq \lambda_l g_l\(x\right)\right)
\prod_{l=0}^{j-1} 1\left(\lambda_j g_j\(x\right) > \lambda_l g_l\(x\right)\right).
\end{split}\end{align}
We introduce this notation in particular to handle the case $\lambda_{j}\hat{g}_{j} = \lambda_{l}\hat{g}_{l}$ when $\hat{g}$ is estimated.
For $k\neq l, k,l=0,\ldots,J$, we also write
\begin{align}\begin{split}\label{decision difference}
1_{kl}\(x; \lambda, g\right) \coloneqq 1\(k < l\right)1\left(\lambda_k g_k\(x\right) \geq \lambda_l g_l\(x\right)\right)
+
1\(k > l\right)1\left(\lambda_k g_k\(x\right) > \lambda_l
g_l\(x\right)\right).
\end{split}\end{align}

\begin{proposition}\label{first MA second Hadamard result}
Assume that $\lambda_{j}g_{j} - \lambda_{l}g_{l}$ satisfies MA in \eqref{marginal assumption MA} for all $j \neq l, j, l = 0, \ldots, J$ at $0$ with an $\alpha > 1$.
Under the conditions in Lemma \ref{social welfare potential first order derivative lemma}, where $\mathcal F$ is defined
using \eqref{tie breaking definition to ensure at least at 1} in the statement of the lemma,
the second order Hadamard derivative of $\gamma\left(\lambda, g, Q\right)$ at $g$ tangentially to $C\(E, \mathbb{R}^{J + 1}\right)$ is $0$.

Furthermore, if $\mathcal{F}$ is a Donsker class of functions, $g, \hat{g} \in \mathcal{G}$,
and $r_{n}\left(\hat{g} - g\right) \rightsquigarrow \mathbb{H},$ where $\mathbb{H}$ is a separable process supported on $C\(E, \mathbb{R}^{J + 1}\right)$, then for
$\liminf_{n \to \infty}\frac{r_{n}}{\sqrt[4]{n}} > 0$,
\begin{align}\begin{split}\nonumber
\gamma\left(\lambda, \hat{g}, \mathbb{Q}_{n}\right) = \gamma\left(\lambda, g, \mathbb{Q}_{n}\right) + o_{\mathbb{P}^{*}}\left(\frac{1}{\sqrt{n}}\right).
\end{split}\end{align}
\end{proposition}

\begin{comment}
\begin{proof}[Proof of Proposition \ref{first MA second Hadamard result}]
Note that we have the following decomposition:
\begin{align}\begin{split}\nonumber
\gamma\left(\lambda, \hat{g}, \mathbb{Q}_{n}\right) - \gamma\left(\lambda, g, \mathbb{Q}_{n}\right)
& = \underbrace{\gamma\left(\lambda, \hat{g}, \mathbb{Q}_{n} - Q\right) - \gamma\left(\lambda, g, \mathbb{Q}_{n} - Q\right)}_{\left(\romannumeral 1\right)} \\
& + \underbrace{\gamma\left(\lambda, \hat{g}, Q\right) - \gamma\left(\lambda, g, Q\right)}_{\left(\romannumeral 2\right)}.
\end{split}\end{align}
Since Donskerness implies stochastic equicontinuity (see for example Theorem 1.5.7 and section 2.1.2 of \cite{van2023weak}), the $\left(\romannumeral 1\right)$ term is of $o_{\mathbb{P}^{*}}\left(\frac{1}{\sqrt{n}}\right)$.

For the $\left(\romannumeral 2\right)$ term, consider the second order Hadamard derivative of $\gamma\left(\lambda, \cdot, Q\right)$ at $g$, i.e.
\begin{align}\begin{split}\nonumber
\lim_{n \to \infty}\frac{\gamma\left(\lambda, g + t_{n}H_{n}, Q\right) - \gamma\left(\lambda, g, Q\right)}{t^{2}_{n}}
\end{split}\end{align}
since the first order Hadamard derivative is $0$.
Here we use the observation that Lemma \ref{social welfare potential first order derivative lemma} still holds when we adopt the allocation rule in \eqref{tie breaking definition to ensure at least at 1}.
Note that we have the following bound:
\begin{align}\begin{split}\nonumber
& \gamma\left(\lambda, g, Q\right) - \gamma\left(\lambda, g', Q\right) \\
& = \mathbb{E}\left[\sum_{i\neq j} \left(\lambda_i g_i\(X\right) - \lambda_j g_j\(X\right)\right) \phi_i\(X; g\right) \phi_j\(X; g'\right)\right] \\
&\leq \mathbb{E}\left[\sum_{i\neq j} \left\vert\lambda_i g_i\(X\right) - \lambda_j g_j\(X\right) -
\left(\lambda_i g'_i\(X\right) - \lambda_j g'_j\(X\right)\right)
\right\vert \phi_i\(X; g\right) \phi_j\(X; g'\right)\right] \\
& =  \mathbb{E}\left[\sum_{i\neq j} \left\vert\lambda_i g_i\(X\right) - \lambda_j g_j\(X\right) -
\left(\lambda_i g'_i\(X\right) - \lambda_j g'_j\(X\right)\right)
\right\vert 1\left(\phi_i\(X; g\right) = \phi_j\(X; g'\right) = 1\right)\right] \\
&\leq \mathbb{E}\left[\sum_{i\neq j}
\left\vert\lambda_i g_i\(X\right) - \lambda_j g_j\(X\right) -
\left(\lambda_i g'_i\(X\right) - \lambda_j g'_j\(X\right)\right)
\right\vert
1\left(\phi\(X; g\right)\neq \phi\(X; g'\right)\right)\right] \\
&\leq \mathbb{E}\left[\sum_{i\neq j}
\left\vert\lambda_i g_i\(X\right) - \lambda_j g_j\(X\right) -
\left(\lambda_i g'_i\(X\right) - \lambda_j g'_j\(X\right)\right)\right\vert
\sum_{k \neq l} \left\vert
1_{kl}\(X; g\right) - 1_{kl}\(X; g'\right)
\right\vert\right]
\end{split}\end{align}
\end{comment}
\begin{comment}
\begin{align}\begin{split}\nonumber
& \gamma\left(\lambda, g, Q\right) - \gamma\left(\lambda, g', Q\right) \\
& = \mathbb{E}\left[\sum_{j}  \lambda_j g_j\(X\right)\left(\phi_j\(X; g\right) - \phi_j\(X; g'\right)\right)\right] \\
&\leq \mathbb{E}\left[\sum_{i\neq j} \left\vert\lambda_i g_i\(X\right) - \lambda_j g_j\(X\right) -
\left(\lambda_i g'_i\(X\right) - \lambda_j g'_j\(X\right)\right)
\right\vert \phi_i\(X; g\right) \phi_j\(X; g'\right)\right] \\
& + \mathbb{E}\left[\sum_{i\neq j} \vert \lambda_j g_j\(X\right)\vert 1\left(
\lambda_i g_i\(X\right) = \lambda_j g_j\(X\right)
\right)\right]  + \mathbb{E}\left[\sum_{i\neq j} \vert \lambda_j g_j\(X\right)\vert 1\left(
\lambda_i g_i'\(X\right) = \lambda_j g_j'\(X\right)
\right)\right] \\
&\leq \mathbb{E}\left[\sum_{i\neq j}
\left\vert\lambda_i g_i\(X\right) - \lambda_j g_j\(X\right) -
\left(\lambda_i g'_i\(X\right) - \lambda_j g'_j\(X\right)\right)
\right\vert
1\left(\phi\(X; g\right)\neq \phi\(X; g'\right)\right)\right] \\
&+ \mathbb{E}\left[\sum_{i\neq j} \vert \lambda_j g_j\(X\right)\vert 1\left(
\lambda_i g_i'\(X\right) = \lambda_j g_j'\(X\right)\right)\right]\\
&\leq \mathbb{E}\left[\sum_{i\neq j}
\left\vert\lambda_i g_i\(X\right) - \lambda_j g_j\(X\right) -
\left(\lambda_i g'_i\(X\right) - \lambda_j g'_j\(X\right)\right)\right\vert
\sum_{k \neq l} \left\vert
1_{kl}\(X; g\right) - 1_{kl}\(X; g'\right)
\right\vert\right]\\
&+ \mathbb{E}\left[\sum_{i\neq j} \vert \lambda_j g_j\(X\right)\vert 1\left(
\lambda_i g_i'\(X\right) = \lambda_j g_j'\(X\right)\right)\right].
\end{split}\end{align}
\end{comment}
\begin{comment}
For $g' = g + t_{n}H_{n}$, following \cite{chen2003estimation},
\begin{align}\begin{split}\nonumber
\left\vert 1_{kl}\(x; g\right) - 1_{kl}\(x; g + t_{n}H_{n}\right)\right\vert
\leq 1\left(-2\delta_{n} \leq \lambda_{k}g_{k}\(x\right) - \lambda_{l}g_{l}\(x\right) \leq 2\delta_{n}\right),
\end{split}\end{align}
where $\delta_{n} = \sup_{x \in K_{f}}\max _{0, \ldots, J}\vert t_{n}H_{j}\(x\right)\vert$, $K_{f}$ is the compact support of $X$.
Obviously, $\delta_{n} \leq M\vert t_{n}\vert$ for some constant $M > 0$.
So by the MA assumptions, we have
\begin{align}\begin{split}\nonumber
\gamma\left(\lambda, g + t_{n}H_{n}, Q\right) - \gamma\left(\lambda, g, Q\right) \leq C\vert t_{n}\vert^{1 + \alpha}
\end{split}\end{align}
for some constant $C > 0$.
Therefore, the second order Hadamard derivative we would like to calculate here is $0$.
Now, the delta method for second order Hadamard differentiable functional in Theorem 2.1 of \cite{chen2019inference} implies the $\left(\romannumeral 2\right)$ term is also of $o_{\mathbb{P}^{*}}\left(\frac{1}{\sqrt{n}}\right)$.
\end{proof}
\end{comment}

The main drawback of Hadamard differentiability (functional delta method) is that it does not fully utilize the convergence rate of the first step estimator.
Since Proposition \ref{surface integral continuity} guarantees $\alpha = 1$, the requirement of
$\alpha > 1$ in Proposition \ref{first MA second Hadamard result} is less satisfactory.
In section \ref{inference on social welfare potential function} we overcome this drawback using sample splitting and Neyman orthogonality.
In the following we provide two additional results to show more connection between the theory in section \ref{Weighted sorted effect} and the MA.

Recall that the push-forward measure $T_{\#}\mu$ generated by a Borel measurable map $T$ from the measure $\mu$ is defined by $T_{\#}\mu\[A\right] = \mu\[T^{-1}\(A\right)\right]$.
The original margin assumption in \eqref{marginal assumption MA} can be extended to the following:

\begin{definition}(Margin controlled random variable / distribution function)
Let $X$ be a random variable supported on $\[a, b\right]$, where $a < b$, $a, b \in \mathbb{R}$.
If for all $c \in \[a, b\right]$,
\begin{align}\begin{split}\nonumber
\mathbb{P}\left\{\left\vert X - c\right\vert < t\right\} \leq C\(c\)t^{\alpha},
\end{split}\end{align}
where $-\infty < C\(c\right) < +\infty$ may depend on c, we call $X$ a margin controlled random variable of order $\alpha$.
Similarly, if $F$ is the distribution function of such a random variable, we call it a margin controlled distribution function.
\end{definition}
A definition of a margin controlled random variable
is to impose local H\"{o}lder-continuity on its distribution
function with a universal exponent
$\alpha$.

\begin{lemma}\label{marginally controlled lemma}
If $X$ is a margin controlled random variable on $\[a, b\right]$ of order $\alpha=1$, where $a < b$, $a, b \in \mathbb{R}$, then $X$ is absolutely continuous with respect to the Lebesgue measure. Consequently, $X$ admits a density function.
\end{lemma}

\begin{lemma}\label{density iff}
Let $f \in L^{1}\left(\mathbb{R}^{n}, \mathcal{L}_{n}\right)$ be a nonnegative function and $h: \Omega \to \mathbb{R}$ be a function in $C^{m}$ (or Sobolev space $W^{m, p}$, $p > m$), where $\Omega \subset \mathbb{R}^{n}$ is an open subset.
Let $X$ be a random variable with density function $f$.
Then $h\(X\right)$ also admit a density, i.e. the push-forward probability $h_{\#}\(f\mathcal{L}_{n}\right) \ll \mathcal{L}_{1}$, if and only if $Jh > 0 \; a.e.$ on the support of $f$.	In this case the density of $h\(X\right)$ is
\begin{align}\begin{split}\nonumber
f_{h}\(y\right) = \int_{h^{-1}\(y\right)}\frac{f\(x\right)}{Jh\(x\right)}d\mathcal{H}_{n - 1}x.
\end{split}\end{align}
\end{lemma}

\begin{remark}
If one only needs the if part of Lemma \ref{density iff}, where $Jh > 0 \; a.e.$ implies a density function in the form of integration with respect to the Hausdorff measure,
then one does not need the $C^{m}$ assumption. Lipschitzness is sufficient.
Proposition \ref{surface integral continuity} tells more by confirming that with a bit more regularity conditions, $f_{h}$ is actually continuous.
\end{remark}

\begin{comment}
\subsection{Binary classification weighted accuracy}

To illustrate the insights in \cite{semenova2023debiased}, \cite{devroye1996probabilistic} and \cite{audibert2007fast},
consider a weighted
population accuracy measure of a binary classifier,
defined as
\begin{align}\begin{split}\label{weighted population accuracy}
\gamma\left(\tau, p\left(\cdot\right), R\right)
&= \mathbb{E}\left[ \tau Y 1\(X\in R\right) + \left(1-\tau\right) \left(1-Y\right) 1\(X \in R^c\right)\right]\\
&= \mathbb{E}\left[\left(1-\tau\right) \left(1-p\(X\right)\right)\right]
+ \mathbb{E}\left[\(p\(X\right) - \left(1-\tau\right)\right) 1\(X \in R\right)\right],
\end{split}\end{align}
where $p\(x\right) = \mathbb{E}\(Y \vert X=x\right)$. As in the model developed in
\cite{semenova2023debiased},
there is no restriction at all on the range of $Y$ which does not
need to be binary.
Then an optimal choice of $R$, which is
$1\(p\(x\right) > \left(1-\tau\right)\right)$,
for a given $\tau$ leads to
\begin{align}\begin{split}\nonumber
\gamma\left(\tau, p\left(\cdot\right)\right)
&= \max_R \gamma\left(\tau, p\left(\cdot\right), R\right)\\
&= \mathbb{E}\left[\left(1-\tau\right) \left(1-p\(X\right)\right)\right]
+ \mathbb{E} \left[\(p\(X\right) - \left(1-\tau\right)\right) 1\(p\(X\right) > \left(1-\tau\right)\right)\right].
\end{split}\end{align}
The welfare regret is defined as the difference between
the optimized population welfare
and the feasible welfare based on an estimate of the optimal policy
$\hat R = 1\left(\hat p\(x\right) > 1-\tau\right)$:
\begin{align}\begin{split}\nonumber
\gamma\left(\tau, p\left(\cdot\right)\right) - \gamma\left(\tau, p\left(\cdot\right), \hat R\right)
&=
\mathbb{E}\left[\(p\(X\right) - \left(1-\tau\right)\right) \left(1\(p\(X\right) > 1-\tau\right)
- 1\left(\hat p\(X\right) > 1-\tau\right)\right)\right] \\
&=
\mathbb{E}\left[\left\vert\(p\(X\right) - \left(1-\tau\right)\right)\right\vert\left\vert \left(1\(p\(X\right) > 1-\tau\right)
- 1\left(\hat p\(X\right) > 1-\tau\right)\right)\right\vert\right].
\end{split}\end{align}
The key insight for analyzing convergence speed of above regret
is that on the set where
\begin{align}\begin{split}\nonumber
1\(p\(x\right) > \left(1-\tau\right)\right)
- 1\left(\hat p\(x\right) > \left(1-\tau\right)\right)	 \neq 0,
\end{split}\end{align}
 or where either
$\left\{p\(x\right) > 1-\tau, \hat p\(x\right) \leq 1-\tau\right\}$ or
$\left\{p\(x\right) \leq 1-\tau, \hat p\(x\right) > 1-\tau\right\}$:
\begin{align}\begin{split}\nonumber
\left\vert p\(x\right) - \left(1-\tau\right)\vert \leq \vert p\(x\right) - \hat p\(x\right)\right\vert.
\end{split}\end{align}
Therefore the regret can be bounded by,
\begin{align}\begin{split}\nonumber
\left\vert
\gamma\left(\tau, p\left(\cdot\right)\right) - \gamma\left(\tau, p\left(\cdot\right),	\hat R\right)\right\vert
& \leq \mathbb{E}\left[
\left\vert p\(X\right) - \hat p\(X\right)\right\vert
\left\vert \left(1\(p\(X\right) > 1-\tau\right)
- 1\left(\hat p\(X\right) > 1-\tau\right) \right)\right\vert\right] \\
& \leq
\sup_{x\in \mathcal X}\left\vert p\(x\right) - \hat p\(x\right) \right\vert
\mathbb{E}
\left\vert\left(1\(p\(X\right) > 1-\tau\right)
- 1\left(\hat p\(X\right) > 1-\tau\right)\right)\right\vert \\
\end{split}\end{align}
As in \cite{chen2003estimation}, for $\delta_n = \sup_{x\in \mathcal X}\left\vert p\(x\right) - \hat p\(x\right)\right\vert$,
\begin{align}\begin{split}\nonumber
\left\vert\left(1\(p\(x\right) > 1-\tau\right)
- 1\left(\hat p\(x\right) > 1-\tau\right)\right)\right\vert
\leq 1\left(
1-\tau - \delta_n \leq p\(x\right) \leq
1-\tau + \delta_n
\right).
\end{split}\end{align}
As such the regret can be further bounded by
\begin{align}\begin{split}\nonumber
\left\vert
\gamma\left(\tau, p\left(\cdot\right)\right) - \gamma\left(\tau, p\left(\cdot\right), \hat R\right)\right\vert
\leq \delta_n \mathbb{P}\left(1-\tau - \delta_n \leq p\(X\right) \leq
1-\tau + \delta_n\right)
\end{split}\end{align}
Under the conditions in section \ref{Weighted sorted effect} (see for example Proposition \ref{surface integral continuity}) where $p\(x\right)$ has a bounded density at $1-\tau$, for a constant $C$,
\begin{align}\begin{split}\nonumber
\mathbb{P}\left(1-\tau - \delta_n \leq p\(X\right) \leq
1-\tau + \delta_n\right) \leq  C \delta_n.
\end{split}\end{align}
In conclusion,
$\left\vert
\gamma\left(\tau, p\left(\cdot\right)\right) - \gamma\left(\tau, p\left(\cdot\right), \hat R\right)\right\vert \leq C \delta_n^2$.
Many nonparametric estimators $\hat p\(x\right)$ achieves $\delta_n = O_{\mathbb{P}}\(n^{-\frac{\beta}{2\beta+d}} \log n\right)$ where $d$ is the dimension of $x$, and $\beta$ a smoothness parameter, implying
\begin{align}\begin{split}\nonumber
\left\vert
\gamma\left(\tau, p\left(\cdot\right)\right) - \gamma\left(\tau, p\left(\cdot\right), \hat R\right)\right\vert
= O_{\mathbb{P}}\left(
n^{-\frac{2\beta}{2\beta+d}} \log^2 n
\right)
\end{split}\end{align}
and  whenever $\beta > d/2$,
\begin{align}\begin{split}\nonumber
\left\vert
\gamma\left(\tau, p\left(\cdot\right)\right) - \gamma\left(\tau, p\left(\cdot\right), \hat R\right)\right\vert
 = O_{\mathbb{P}}\left(
n^{-\frac{1}{2}}\right).
\end{split}\end{align}
In the above, the analysis we conducted in section
\ref{Weighted sorted effect}
provides primitive conditions that offer
the case of $\alpha \geq 1$
for the \textit{margin} assumption (MA) in
\cite{mammen1999smooth}, \cite{tsybakov2004optimal}, \cite{boucheron2005theory},
\cite{audibert2007fast}, \cite{kitagawa2018should} and \cite{luedtke2020performance}.

The misclassification error rate in \cite{audibert2007fast} is the  most important
case of the weighted population accuracy \eqref{weighted population accuracy} where $\tau=\frac12$, since
\begin{align}\begin{split}\nonumber
\mathbb{P}\(Y \neq 1\(X \in R\right)\right)
&= 1 - \mathbb{P}\(Y = 1\(X \in R\right)\right) \\
& =
1 - 2 \left[
	\frac12 \mathbb{E} Y 1\(X \in R\right) +
\frac12 \mathbb{E} \left(1-Y\right) 1\(X \in R^c\right)
\right] \\
& = 1 - 2 \gamma\left(\frac12, p\left(\cdot\right), R\right).
\end{split}\end{align}

\end{comment}

\subsection{Multi discrete allocation problem}

In this section we apply the fast convergence rate to
social welfare functions with multiple treatments, generalizing the arguments
in
\cite{devroye1996probabilistic} and \cite{audibert2007fast}.
\begin{comment}
Weighted
population accuracy of this classifier is
defined as
\begin{align}\begin{split}\label{weighted population accuracy}
\gamma\left(\tau, p\left(\cdot\right), R\right)
&= \mathbb{E}\left[ \tau Y 1\(X\in R\right) + \left(1-\tau\right) \left(1-Y\right) 1\(X \in R^c\right)\right]\\
&= \mathbb{E}\left[\left(1-\tau\right) \left(1-p\(X\right)\right)\right]
+ \mathbb{E}\left[\(p\(X\right) - \left(1-\tau\right)\right) 1\(X \in R\right)\right],
\end{split}\end{align}
where $p\(x\right) = \mathbb{E}\(Y \vert X=x\right)$.
The optimal choice of $R$, which is
$1\(p\(x\right) > \left(1-\tau\right)\right)$,
leads to
\begin{align}\begin{split}\nonumber
\gamma\left(\tau, p\left(\cdot\right)\right)
& = \max_R \gamma\left(\tau, p\left(\cdot\right), R\right) \\
& = \mathbb{E}\left[\left(1-\tau\right) \left(1-p\(X\right)\right)\right]
+ \mathbb{E} \left[\(p\(X\right) - \left(1-\tau\right)\right) 1\(p\(X\right) > \left(1-\tau\right)\right)\right].
\end{split}\end{align}
The regret for feasible policy $\hat R = 1\left(\hat p\(x\right) > 1-\tau\right)$ is defined as
\begin{align}\begin{split}\nonumber
\gamma\left(\tau, p\left(\cdot\right)\right) - \gamma\left(\tau, p\left(\cdot\right), \hat R\right)
&=
\mathbb{E}\left[\(p\(X\right) - \left(1-\tau\right)\right) \left(1\(p\(X\right) > 1-\tau\right)
- 1\left(\hat p\(X\right) > 1-\tau\right)\right)\right].
\end{split}\end{align}
The key insight here is that set $\left\{1\(p\(x\right) > \left(1 - \tau\right)\right) - 1\left(\hat p\(x\right) > \left(1 - \tau\right)\right) \neq 0\right\}$ equals set
\begin{align}\begin{split}\nonumber
\left\{p\(x\right) > 1 - \tau, \hat p\(x\right) \leq 1 - \tau\right\} \cup \left\{p\(x\right) \leq 1 - \tau, \hat p\(x\right) > 1 - \tau\right\},
\end{split}\end{align}
i.e. on this set $\left\vert p\(x\right) - \left(1 - \tau\right)\vert \leq \vert p\(x\right) - \hat p\(x\right)\right\vert$.
Therefore the regret can be bounded by,
\begin{align}\begin{split}\nonumber
\left\vert
\gamma\left(\tau, p\left(\cdot\right)\right) - \gamma\left(\tau, p\left(\cdot\right),	\hat R\right)\right\vert
\leq \mathbb{E}\left[
\left\vert p\(X\right) - \hat p\(X\right)\right\vert
\left\vert \left(1\(p\(X\right) > 1-\tau\right)
- 1\left(\hat p\(X\right) > 1-\tau\right) \right)\right\vert\right].
\end{split}\end{align}
As in \cite{chen2003estimation}, for $\delta_n = \sup_{x\in \mathcal X}\left\vert p\(x\right) - \hat p\(x\right)\right\vert$,
\begin{align}\begin{split}\nonumber
\left\vert\left(1\(p\(x\right) > 1-\tau\right)
- 1\left(\hat p\(x\right) > 1-\tau\right)\right)\right\vert
\leq 1\left(
1-\tau - \delta_n \leq p\(x\right) \leq
1-\tau + \delta_n
\right).
\end{split}\end{align}
Consequently, the regret can be further bounded by
\begin{align}\begin{split}\nonumber
\left\vert
\gamma\left(\tau, p\left(\cdot\right)\right) - \gamma\left(\tau, p\left(\cdot\right), \hat R\right)\right\vert
\leq \delta_n \mathbb{P}\left(1-\tau - \delta_n \leq p\(X\right) \leq
1-\tau + \delta_n\right)
\end{split}\end{align}
Under the MA for $p\(X\right)$ at $1 - \tau$ with $\alpha \geq 1$, for a constant $C$,
\begin{align}\begin{split}\nonumber
\mathbb{P}\left(1-\tau - \delta_n \leq p\(X\right) \leq
1-\tau + \delta_n\right) \leq  C \delta_n.
\end{split}\end{align}
To sum up, the regret can be upper bounded as
$\left\vert
\gamma\left(\tau, p\left(\cdot\right)\right) - \gamma\left(\tau, p\left(\cdot\right), \hat R\right)\right\vert \leq C \delta_n^2$.
\end{comment}



Recall that in a general discrete allocation problem, we define welfare under policy $\phi\left(\cdot\right)$ as
\begin{align}\begin{split}\label{multi welfare under policy}
\gamma\left(\lambda, g, \phi\right)
= \mathbb{E} \left[\sum_{j=0}^J  \lambda_j\phi_j\(X\)Y_j\right]
= \mathbb{E} \left[\sum_{j=0}^J \lambda_j\phi_j\(X\)g_j\(X\right)\right],
\end{split}\end{align}
where
$g_j\(x\right) = \mathbb{E}\(Y_j \vert X = x\right)$ and $\phi_j\(x\right) \geq 0, \forall j$, $\sum_{j=0}^J \phi_j\(x\right) = 1$.
A special case of \eqref{multi welfare under policy} is the weighted multiclassification accuracy when
$Y_j \in \{0,1\}$, $\sum_{j=0}^J Y_j =1$, and when $\phi_j\(x\right), j=0,\ldots,J$ is a multiclassification
allocation.\footnote{The binary treatment case in \cite{audibert2007fast} is a special of \eqref{multi welfare under policy} where $g_1\(x\right)=p(x)$, the propensity score,
$g_0\(x\right)=1-p\(x\right)$, $\phi_1\(x\right) = 1\(p(x)>\tau\right)$,  $\phi_0\(x\right) = 1\(p(x) \leq \tau\right)$,  $\lambda_1=\tau$ and $\lambda_0=1-\tau$.}


In classification problems, $Y_j, j=0,\ldots,J$ are directly observed.
In contrast, in optimal treatment allocation
problem, $Y_j$ is typically observed only when $d_j=1$, where $d_j$ is the indicator  of whether the $j$th treatment is taken,
such that $d_j \in \{0,1\}$ and $\sum_{j=0}^J d_j = 1$.
We assume the standard unfoundedness condition, i.e.,
$Y_{j}\perp\!\!\!\perp D_{j} \vert X, \forall j=0,\ldots,J$.
Then $g_j\(x\right) = \mathbb{E}\(Y_j \vert D_j =1, x\right)$ can be estimated
based on the subsample where $d_j = 1$.

The envelope-like Theorem \ref{theorem envelope Theorem for social welfare potential function} essentially
states that generically there is
no first order impact of estimating $\phi^*\left(\cdot; \lambda, g\right)$
in $\gamma\left(\lambda, g\right) = \mathbb{E} \left[\sum_j \phi_j^*\(X; \lambda, g\right) g_j\(X\right)\right]$
by $\phi^{*}\left(\cdot; \lambda, \hat{g}\right)$.
The fast convergence rate of regret to zero is a consequence of the second order effect
of estimating $\phi^*\left(\cdot; \lambda, g\right)$ by $\phi^{*}\left(\cdot; \lambda, \hat{g}\right)$.
We use the same convention as
\eqref{tie breaking definition to ensure at least at 1}
in
the following expressions for the optimal and feasible policies:
\begin{align}\begin{split}\nonumber
\phi_j^{*}\(x\right) &= \prod_{l=j+1}^J 1\left(\lambda_j g_j\(x\right) \geq \lambda_l g_l\(x\right)\right)
\prod_{l=0}^{j-1} 1\left(\lambda_j g_j\(x\right) > \lambda_l g_l\(x\right)\right),\\
\hat\phi_j\(x\right) &= \prod_{l=j+1}^J 1\left(\lambda_j \hat g_j\(x\right) \geq \lambda_l \hat g_l\(x\right)\right)
\prod_{l=0}^{j-1} 1\left(\lambda_j \hat g_j\(x\right) > \lambda_l \hat g_l\(x\right)\right).
\end{split}\end{align}
The regret of using the feasible policy for welfare
is:
\begin{align}\begin{split}\nonumber
\gamma\left(\lambda, g, \phi^{*}\right)
-
\gamma\left(\lambda, g, \hat\phi\right)
= \sum_{j=0}^{J} \mathbb{E}\left[\lambda_j g_j\(X\right) \left(
\phi_j^{*}\(X\right) - \hat \phi_j\(X\right)
\right)\right].
\end{split}\end{align}
By definition, $\phi_j^{*}\(x\right)$ is either $0$ or $1$.
Furthermore, for each $x$, one and only one of $\phi_j^{*}\(x\right)$ out of $j=0,\ldots,J$ takes
the value $1$.
The same is true for $\hat\phi_j\(x\right)$ for each $x$.
Therefore we can write
\begin{align}\begin{split}\label{tsybakov 2nd order bound}
\gamma\left(\lambda, g, \phi^{*}\right)
& -\gamma\left(\lambda, g, \hat\phi\right) \\
&\leq \mathbb{E}\left[\sum_{i\neq j} \left\vert\lambda_i g_i\(X\right) - \lambda_j g_j\(X\right) -
\left(\lambda_i \hat g_i\(X\right) - \lambda_j \hat g_j\(X\right)\right)
\right\vert \phi_i^{*}\(X\right) \hat\phi_j\(X\right)\right] \\
&\leq \mathbb{E}\left[\sum_{i\neq j}
\left\vert\lambda_i g_i\(X\right) - \lambda_j g_j\(X\right) -
\left(\lambda_i \hat g_i\(X\right) - \lambda_j \hat g_j\(X\right)\right)\right\vert
\sum_{k \neq l} \left\vert
1_{kl}^{*} -\hat 1_{kl}
\right\vert\right],
\end{split}\end{align}
where the second inequality uses the simplified notation \eqref{decision difference}.
The first inequality of \eqref{tsybakov 2nd order bound} is due to the fact that when
$\phi_i^{*}\(x\right) = \hat\phi_j\(x\right) =1$,
\begin{align}\begin{split}\nonumber
0 \leq \lambda_i g_i\(x\right) - \lambda_j g_j\(x\right)
\leq \lambda_i g_i\(x\right) - \lambda_j g_j\(x\right) - \left(
\lambda_i \hat g_i\(x\right) - \lambda_j \hat g_j\(x\right)
\right).
\end{split}\end{align}
Following Example 1 in \cite{chen2003estimation}, for
$\delta_n = \sup_{x\in\mathcal X} \max_{j=0,\ldots,J} \vert
\lambda_j g_j\(x\right) - \lambda_j \hat{g}_j\(x\right)
\vert$,
\begin{align}\begin{split}\nonumber
\vert
1_{kl}^{*} -\hat 1_{kl}
\vert \leq
1\left(
-2 \delta_n \leq \lambda_k g_k\(x\right) - \lambda_l g_l\(x\right)\leq 2 \delta_n
\right).
\end{split}\end{align}
Then we can further bound the regret as
\begin{align}\begin{split}\nonumber
&\gamma\left(\lambda, g, \phi^{*}\right)
-\gamma\left(\lambda, g, \hat\phi\right)
\leq 2 \(J+1\right)^2 \delta_n
\sum_{k\neq l} \mathbb{P}\left(-2 \delta_n \leq \lambda_k g_k\(X\right) -
\lambda_l g_l\(X\right)\leq 2 \delta_n\right).
\end{split}\end{align}
The conditions in section \ref{Weighted sorted effect} imply that
for
$k\neq l$,
$\lambda_k g_k\(x\right) - \lambda_l g_l\(x\right)$
has a bounded density at zero, leading to
$\mathbb{P}\left(-2 \delta_n \leq \lambda_k g_k\(X\right) -
\lambda_l g_l\(X\right)\leq 2 \delta_n\right) \leq C \delta_n$ and
$\gamma\left(\lambda, g, \phi^{*}\right)
-\gamma\left(\lambda, g, \hat\phi\right)
\leq C \delta_n^2.$

Nonparametric estimators $\hat{g}_j\(x\right)$ typically achieve the convergence rate $\delta_n = O_{\mathbb{P}}\(n^{-\frac{\beta}{2\beta+d}} \log n\right)$ where $d$ is the dimension of $x$, and $\beta$ a smoothness parameter, implying that
\begin{align}\begin{split}\nonumber
\left\vert
\gamma\left(\lambda, g, \phi^{*}\right)
-\gamma\left(\lambda, g, \hat\phi\right)\right\vert
= O_{\mathbb{P}}\left(
n^{-\frac{2\beta}{2\beta+d}} \log^2 n
\right).
\end{split}\end{align}
Whenever $\beta > d/2$, the convergence rate of
$\gamma\left(\lambda, g, \phi^{*}\right) -\gamma\left(\lambda, g, \hat\phi\right)$
becomes $O_{\mathbb{P}}\left(
n^{-\frac{1}{2}}\right).$


\subsection{Inference on social welfare potential function}\label{inference on social welfare potential function}

The unknown social welfare potential function $\beta_0 = \gamma\left(\lambda, g\right) = \mathbb{E}\left[\sum_{j}\lambda_j \phi^{*}_{j}\(X\)g_{j}\(X\right)\right]$
can be estimated by its sample analog
$\frac{1}{n'} \sum_{i=1}^{n'} \sum_{j} \lambda_j \hat{\phi}_{j}\(X_{i}\right)\hat g_j\(X_{i}\right)$ for $n' = O\(n\right)$.
On the one hand, under full observability, the Neyman-orthogonalized estimator
$\frac{1}{n'} \sum_{i=1}^{n'} \sum_{j} \lambda_j \hat{\phi}_{j}\(X_{i}\)Y_{ij}$ can be combined with
a sample splitting scheme for estimating $\hat{\phi}_{j}\(X_{i}\right)$ to achieve  desirable
statistical properties.
On the other hand, for a typical treatment allocation problem, $Y_{ij}$ are not fully observable.
We only observe $Y_{i} = \sum_{j}D_{ij}Y_{ij}$ and need to use
the treatment status variables $D_{ij}$ to construct a Neyman-orthogonalized estimator.

\begin{theorem}\label{treatment effect allocation simple case}
Let $p, \hat{p}, g, \hat{g}: E \subset \mathbb{R}^{\dim\(X_i\right)} \to \mathbb{R}^{J + 1}$ be uniformly bounded functions, where $E$ is an open set.
$Var\(Y_{i} \vert X_i\right)$ is uniformly bounded.
Assume that there exists a constant $\epsilon > 0$ such that $p_{j}\(x\right), \hat{p}_{j}\(x\right) > \epsilon$ for all $x \in E$ and for all $j \in \{0, \ldots, J\}$.
Assume also that for some sequence $a_{n}$, $\lim_{n \to \infty}\frac{a_{n}}{\sqrt[4]{n}} = + \infty$,
there is $\operatorname*{ess\,sup} a_{n}\left\vert \hat{p} - p\right\vert = o_{\mathbb{P}}\left(1\right)$ and $\operatorname*{ess\,sup} a_{n}\left\vert \hat{g} - g\right\vert = o_{\mathbb{P}}\left(1\right)$, where the essential supremum is taken with respect to the distribution of $X$.
Further assume that
$\lambda_{j}g_{j} - \lambda_{l}g_{l}$ satisfies the MA for all $j \neq l$, $j, l = 0, \ldots, J$ at $0$ with $\alpha = 1$.
Under unconfoundedness and using sample splitting scheme for estimating $\hat p\(x\right)$ and $\hat g\(x\right)$ in a treatment allocation model,
\begin{align}\begin{split}\label{influence function multiclass policy learning}
\sqrt{n'}\left(\hat\beta - \beta_0\right)
= \frac{1}{\sqrt{n'}}
\sum_{i=1}^{n'}
\left(\sum_{j=0}^J \lambda_j \phi_j^{*}\(X_i\right)
\left[
	g_j\(X_i\right) + \frac{D_{ij}}{p_j\(X_i\right)} \(Y_i - g_j\(X_i\right)\right)
\right] - \beta_0\right)
+ o_\mathbb{P}\left(1\right)
\end{split}\end{align}
where
\begin{align}\begin{split}\label{DML social welfare potential treatment}
\hat\beta=
\frac{1}{n'} \sum_{i=1}^{n'}
\sum_{j=0}^J \lambda_j \hat \phi_j\(X_i\right)
\left[
	\hat g_j\(X_i\right) + \frac{D_{ij}}{\hat p_j\(X_i\right)} \(Y_i - \hat g_j\(X_i\right)\right)
\right].
\end{split}\end{align}
Additionally, suppose $g$ and $\mu$ satisfy the conditions of Theorem \ref{general k derivative}.
Let $\mathcal{D} \subset \mathbb{R}^{J + 1}_{++}$ be a compact set where $\mathbb{R}^{J + 1}_{++} \coloneqq \left\{\lambda: \lambda \in \mathbb{R}^{J + 1}, \lambda_{j} > 0 , \forall j = 0, \ldots, J\right\}$
and let $\lambda_{j}g_{j} - \lambda_{l}g_{l}$ satisfy the MA with a uniform constant.
Then \eqref{influence function multiclass policy learning} holds uniformly on $\mathcal{D}$.
\end{theorem}

\begin{theorem}\label{classification simple case}
Under the conditions in Theorem \ref{treatment effect allocation simple case} and using sample splitting scheme
to estimate $\hat \phi\(x\right)$ with full observability,
\begin{align}
&\sqrt{n'}\left(\hat\beta - \beta_0\right)
= \frac{1}{\sqrt{n'}} \sum_{i=1}^{n'}
\left(
\sum_{j=0}^J \lambda_j \phi_j^*\(X_i\right)	Y_{ij} - \beta_0
\right)   + o_\mathbb{P}\left(1\right),\label{influence function multiclass policy learning for classification}\\
&\text{where}\quad \hat\beta=
\frac{1}{n'} \sum_{i=1}^{n'}
\sum_{j=0}^J \lambda_j \hat \phi_j\(X_i\right)  Y_{ij}.\notag
\end{align}
Additionally, suppose $g$ and $\mu$ satisfy the conditions of Theorem \ref{general k derivative}.
Let $\mathcal{D} \subset \mathbb{R}^{J + 1}_{++}$ be a compact set where $\mathbb{R}^{J + 1}_{++} \coloneqq \left\{\lambda: \lambda \in \mathbb{R}^{J + 1}, \lambda_{j} > 0, \forall j = 0, \ldots, J\right\}$
and let $\lambda_{j}g_{j} - \lambda_{l}g_{l}$ satisfy the MA with a uniform constant.
Then \eqref{influence function multiclass policy learning for classification} holds uniformly on $\mathcal{D}$.
\end{theorem}

\begin{comment}
\begin{proof}
In the following, the notation $C$ should be understood as \textit{some constant}, and they are not necessarily equal to each other.
With notation $\mathbb{E}$ and $\mathbb{P}$, the expectation and probability are taken with respect to $\(X_{i}, Y_{i}, D_{i}\right)$ ($o_{\mathbb{P}}\left(1\right)$ means convergence in probability with respect to the sample used to estimate $\hat{g}$ and $\hat{p}$).
The result will follow if we can show that $\Delta = o_\mathbb{P}\left(1\right)$, where
\begin{align}\begin{split}\nonumber
\Delta = \frac{1}{\sqrt{n'}}
\sum_{i=1}^{n'} \sum_{j=0}^J
\Biggl(\lambda_j
\hat \phi_j\(X_i\right)
& \left[
	\hat g_j\(X_i\right) + \frac{D_{ij}}{\hat p_j\(X_i\right)} \(Y_i - \hat g_j\(X_i\right)\right)
\right] \\
& - \lambda_j
\phi_j^{*}\(X_i\right)
\left[
	g_j\(X_i\right) + \frac{D_{ij}}{p_j\(X_i\right)} \(Y_i - g_j\(X_i\right)\right)
\right]\Biggl).
\end{split}\end{align}
For this purpose we decompose $\Delta = \Delta_1 + \Delta_2$, where
\begin{align}\begin{split}\nonumber
\Delta_1 =
\frac{1}{\sqrt{n'}}
\sum_{i=1}^{n'} \sum_{j=0}^J
\lambda_j
\hat \phi_j\(X_i\right)
\left[
	\hat g_j\(X_i\right) + \frac{D_{ij}}{\hat p_j\(X_i\right)} \(Y_i - \hat g_j\(X_i\right)\right)
	- g_j\(X_i\right) - \frac{D_{ij}}{p_j\(X_i\right)} \(Y_i - g_j\(X_i\right)\right)
\right]
\end{split}\end{align}
and
\begin{align}\begin{split}\nonumber
\Delta_2 =
\frac{1}{\sqrt{n'}}
\sum_{i=1}^{n'} \sum_{j=0}^J
\lambda_j
\left(
\hat \phi_j\(X_i\right) - \phi_j^{*}\(X_i\right)\right)
\left[
	g_j\(X_i\right) + \frac{D_{ij}}{p_j\(X_i\right)} \(Y_i - g_j\(X_i\right)\right)
\right].
\end{split}\end{align}
Also define a linearized approximation of $\Delta_1$ as
\begin{align}\begin{split}\nonumber
\Delta_3 =
\frac{1}{\sqrt{n'}}
\sum_{i=1}^{n'}	 \sum_{j=0}^J
\lambda_j
\hat \phi_j\(X_i\right)
\left[
	\left(1- \frac{D_{ij}}{p_j\(X_i\right)}\right)\left(\hat g_j\(X_i\right)-  g_j\(X_i\right)\right)
	- \frac{D_{ij}}{p_j^2\(X_i\right)} \(Y_i - g_j\(X_i\right)\right)
	\left(\hat p_j\(X_i\right) - p_j\(X_i\right)\right) \right].
\end{split}\end{align}
Then we can write, when $p\(x\right)$ and $\hat{p}\(x\right)$ is bounded away from zero,
\begin{align}\begin{split}\nonumber
\left\vert \Delta_1 - \Delta_3 \right\vert \leq C
\frac{1}{\sqrt{n'}}
\sum_{i=1}^{n'} \sum_{j=0}^J
\left(\left\vert \hat g_j\(X_i\right) - g_j\(X_i\right)\right\vert^2
+ \left\vert \hat p_j\(X_i\right) - p_j\(X
_i\right)\right\vert^2\right).
\end{split}\end{align}
By the essential supremum convergence rate assumption, $\vert \Delta_1 - \Delta_3 \vert = o_{\mathbb{P}}\left(1\right)$.
To see that $\Delta_3 = o_{\mathbb{P}}\left(1\right)$, note that by the split sample scheme, conditional on the
estimate $\hat p\left(\cdot\right)$ and $\hat g\left(\cdot\right)$,
$\mathbb{E} \Delta_3 = 0$,
\begin{align}\begin{split}\nonumber
Var\left(\Delta_3\right)
& \leq 2\sum_{j=0}^J \lambda_j^2
\Biggl(\mathbb{E}\left[
	\frac{1-p_j\(X_i\right)}{p_j\(X_i\right)} \left(\hat g\(X_i\right) - g\(X_i\right)\right)^2 \hat \phi_j\(X_i\right)^2\right] \\
& + \mathbb{E}\left[\hat \phi_j\(X_i\right)^2 Var\(Y_{ij} \vert X_i, D_{ij}=1\right)
\frac{1}{p_j^3\(X_i\right)} \left(\hat p\(X_i\right) - p\(X_i\right)\right)^2\right]
\Biggr) \\
&\leq C\left( \mathbb{E} \left(\hat g\(X_i\right) - g\(X_i\right)\right)^2
+ \mathbb{E} \left(\hat p\(X_i\right) - p\(X_i\right)\right)^2\right).
\end{split}\end{align}
Next we decompose $\Delta_2 = \Delta_2^1 +  \Delta_2^2$, where
\begin{align}\begin{split}\nonumber
\Delta_2^1 & =
\frac{1}{\sqrt{n'}}
\sum_{i=1}^{n'} \sum_{j=0}^J\lambda_j
\left(
\hat \phi_j\(X_i\right) - \phi_j^{*}\(X_i\right)\right)
	\frac{D_{ij}}{p_j\(X_i\right)} \(Y_i - g_j\(X_i\right)\right), \\
\Delta_2^2 & =
\frac{1}{\sqrt{n'}}
\sum_{i=1}^{n'} \sum_{j=0}^J\lambda_j
\left(
\hat \phi_j\(X_i\right) - \phi_j^{*}\(X_i\right)\right)  g_j\(X_i\right).
\end{split}\end{align}
To see $\Delta_2^1 = o_\mathbb{P}\left(1\right)$, note that $\mathbb{E} \Delta_2^1 = 0$. It suffices to show
that $Var\left(\Delta_2^1\right)	 = o_\mathbb{P}\left(1\right)$ by bounding
\begin{align}\begin{split}\nonumber
Var\left(\Delta_2^1\right)
&\leq \sum_{j=0}^J  \lambda_j^2 \mathbb{E}
\left[\left(
\hat \phi_j\(X_i\right) - \phi_j^{*}\(X_i\right)\right)^2 \frac{1}{p_j\(X_i\right)}	 Var\(Y_j \vert X_i, D_{ij}=1\right)\right]\\
& \leq C
\sum_{j=0}^J \mathbb{E}\left(
\hat \phi_j\(X_i\right) - \phi_j^{*}\(X_i\right)\right)^2 \\%,
& \leq C \sum_{j=0}^J \sum_{p\neq q}  \mathbb{E} 1\left(\phi^{*}\(X_i\right)\neq \hat\phi\(X_i\right)\right)\\
&\leq C \sum_{j=0}^J \sum_{p\neq q} \sum_{k\neq l} \mathbb{E}\left\vert
1_{kl}^{*}\(X_i\right) -\hat 1_{kl}\(X_i\right)\right\vert\\
&\leq C
\sum_{k\neq l} \mathbb{P}\left(-2 \delta_n \leq \lambda_k g_k\(X\right)
- \lambda_l g_l\(X\right)\leq 2 \delta_n\right)
\end{split}\end{align}
where $\delta_{n} = C\max_{j = 0, \ldots, J}\operatorname*{ess\,sup} \left\vert \hat{g}_{j} - g_{j}\right\vert$.
By MA assumption, the right hand side of the last inequality is $O\left(\delta_{n}\right)$.

Since $\Delta^{2}_{2} < 0$, we have
\begin{align}\begin{split}\nonumber
-\Delta^{2}_{2} &=
\frac{1}{\sqrt{n'}}
\sum_{i=1}^{n'}	 \sum_{j=0}^J \lambda_j
\left(
\phi_j^{*}\(X_i\right) - \hat\phi_j\(X_i\right)\right)	 g_j\(X_i\right)
\\
&
=
\frac{1}{\sqrt{n'}}
\sum_{i=1}^{n'}
\sum_{p\neq q} \lambda_p g_p\(X_i\right) - \lambda_q g_q\(X_i\right)
\phi_p^{*}\(X_i\right) \hat\phi_q\(X_i\right)\\
&\leq
\frac{1}{\sqrt{n'}}
\sum_{i=1}^{n'}
\sum_{p\neq q} \left\vert\lambda_p g_p\(X_i\right) - \lambda_q g_q\(X_i\right) -
\left(\lambda_p \hat g_p\(X_i\right) - \lambda_q \hat g_q\(X_i\right)\right)
\right\vert \phi_p^{*}\(X_i\right) \hat\phi_q\(X_i\right)\\
&\leq
\frac{1}{\sqrt{n'}}
\sum_{i=1}^{n'}
\sum_{p\neq q}\left[
\left\vert\lambda_p g_p\(X_i\right) - \lambda_q g_q\(X_i\right) -
\left(\lambda_p \hat g_p\(X_i\right) - \lambda_q \hat g_q\(X_i\right)\right)
\right\vert
 1\left(\phi^{*}\(X_i\right)\neq \hat\phi\(X_i\right)\right)\right] \\
&\leq
\frac{1}{\sqrt{n'}}
\sum_{i=1}^{n'}
\sum_{p\neq q}\left[
\left\vert\lambda_p g_p\(X_i\right) - \lambda_q g_q\(X_i\right) -
\left(\lambda_p \hat g_p\(X_i\right) - \lambda_q \hat g_q\(X_i\right)\right)\right\vert
\sum_{k \neq l}\left\vert
1_{kl}^{*}\(X_i\right) -\hat 1_{kl}\(X_i\right)\right\vert\right] \\
&\leq 2 \(J+1\right)^2 \delta_n
\frac{1}{\sqrt{n'}}
\sum_{i=1}^{n'}
\sum_{k\neq l}
\vert
1_{kl}^{*}\(X_i\right) -\hat 1_{kl}\(X_i\right)
\vert.
\end{split}\end{align}
To show that $\Delta_2^2 =o_{\mathbb{P}}\left(1\right)$, it suffices to check that
\begin{align}\begin{split}\nonumber
- \mathbb{E} \Delta_2^2 &\leq \mathbb{E} C \delta_n \sqrt{n'} \sum_{k\neq l} \left\vert
1_{kl}^{*}\(X_i\right) -\hat 1_{kl}\(X_i\right)
\right\vert \\
&\leq C \sqrt{n'} \delta_n
\sum_{k\neq l} \mathbb{P}\left(-2 \delta_n \leq \lambda_k g_k^{*}\(X\right)
- \lambda_l g_l^{*}\(X\right)\leq 2 \delta_n\right)\\
&= C \sqrt{n'} \delta_n^2 = o_{\mathbb{P}}\left(1\right).
\end{split}\end{align}

If the conditions of Theorem \ref{general k derivative} hold, by Proposition \ref{surface integral continuity}, the vector random variable $g\(X_{i}\right)$ will have a continuous and bounded density.
By coarea formula, for $\left(\lambda_{k}, \lambda_{l}\right) \neq \left(0, 0\right)$, we have
\begin{align}\begin{split}\label{density of summation}
f_{\lambda_{k}g_{k} - \lambda_{l}g_{l}}\(y\right)
= \int_{\lambda_{k}g_{k} - \lambda_{l}g_{l} = y}\frac{f_{g}\(x\right)}{\sqrt{\lambda^{2}_{k} + \lambda^{2}_{l}}}d\mathcal{H}_{J}x,
\end{split}\end{align}
where $f_{g}$ and $f_{\lambda_{k}g_{k} - \lambda_{l}g_{l}}$ are the density functions of $g\(X\right)$ and $\lambda_{k}g_{k}\(X\right) - \lambda_{l}g_{l}\(X\right)$, respectively.
Since $\mathcal{D}$ is a compact subset of $\mathbb{R}^{J + 1}_{++}$ and $g$ is bounded, the right hand side of \eqref{density of summation} is uniformly bounded on $\mathcal{D}$.
It is direct to check that all probability bounds in the above are uniform on $\mathcal{D}$.
\end{proof}
\end{comment}

\begin{comment}
\begin{remark}
It is possible to get some uniform convergence version of Theorem \ref{treatment effect allocation simple case} through careful settings of $\frac{g_{j}}{g_{l}}, l \neq j$, instead of $\lambda_{j}g_{j} - \lambda_{l}g_{l}$.
We focus on the pointwise case in this work since in most of applications, $\lambda = \left(1, \ldots, 1\right)$ is fixed.
We also note that the proof of \ref{treatment effect allocation simple case} already provides a justification of uniform convergence for the binary classification case.
\end{remark}
\end{comment}


We note that \cite{luedtke2016statistical} and \cite{luedtke2020performance} utilize a formula similar to \eqref{DML social welfare potential treatment} for the binary case under empirical process settings.
To apply broadly to machine learning methods,
Theorem \ref{treatment effect allocation simple case} and Theorem \ref{classification simple case}
combine the fast convergence rate of a plug-in classifier with the insighful techniques
of \cite{chernozhukov2018double}, \cite{chernozhukov2022locally} and \cite{foster2023orthogonal} based on Neyman orthogonalization and sample splitting.
Inspired by \cite{semenova2023debiased}, the asymptotic distributions of \eqref{influence function multiclass policy learning} and \eqref{influence function multiclass policy learning for classification} can be estimated through bootstrap.


\begin{cor}\label{multiplier bootstrap}
Let the multiplier $W_{n'i}$ be i.i.d. $Exponential(1)$ random variables,
which are independent of $\(Y, X, D\right)$ or $\(Y, X\right)$,
or follow the $n'$ times $n'$-dimensional Dirichlet distribution with parameter vector $\left(1, \ldots, 1\right)$ independent of $\(Y, X, D\right)$ or $\(Y, X\right)$.
Under the conditions in Theorem \ref{treatment effect allocation simple case} and Theorem \ref{classification simple case},
$\sqrt{n'}\left(\tilde{\beta} - \hat{\beta}\right) \overset{\mathbb{P}}{\underset{\mathbb{W}}{\rightsquigarrow}} \mathbb{G}$,
where the $\overset{\mathbb{P}}{\underset{\mathbb{W}}{\rightsquigarrow}}$ symbol is defined in \eqref{conditional weak convergence in probability}.
On the one hand, in treatment allocation problems with partial
observability,
\begin{align}\begin{split}\nonumber
\tilde{\beta} & =
\frac{1}{n'} \sum_{i=1}^{n'}
W_{n'i}\left(\sum_{j=0}^J \lambda_j \hat \phi_j\(X_i\right)
\left[
	\hat g_j\(X_i\right) + \frac{D_{ij}}{\hat p_j\(X_i\right)} \(Y_i - \hat g_j\(X_i\right)\right)
\right]\right)\ \text{and}\\
\mathbb{G} & = N\left(0, Var\left(\sum_{j=0}^J \lambda_j \phi_j^{*}\(X_i\right)
\left[
	g_j\(X_i\right) + \frac{D_{ij}}{p_j\(X_i\right)} \(Y_i - g_j\(X_i\right)\right)
\right]\right)\right). \;
\end{split}\end{align}
On the other hand, in classification problems with full observability,
\begin{align}\begin{split}\nonumber
\tilde{\beta}  =
\frac{1}{n'} \sum_{i=1}^{n'} W_{n'i}
\left(\sum_{j=0}^J \lambda_j \hat \phi_j\(X_i\right)  Y_{ij}\right) \ \ \text{and}\ \
\mathbb{G}  = N\left(0, Var\left(\sum_{j=0}^J \lambda_j \phi_j^*\(X_i\right)	 Y_{ij}\right)\right).
\end{split}\end{align}
\end{cor}

\begin{comment}
\begin{remark}
    In the last, we should note that we can also change the sup-norm first step convergence assumption to $L^{2}$-norm first step convergence assumption, but increase the $\lim_{n \to \infty}\frac{a_{n}}{\sqrt[4]{n}} = + \infty$ requirement to $\lim_{n \to \infty}\frac{a_{n}}{\sqrt[3]{n}} = + \infty$.
    The reason here is quite simple.
    If $\Vert \hat{g} - g\Vert_{2} = o_{\mathbb{P}}\(a^{-1}_{n}\right)$, then on a set $S$ of probability $1 - a^{-1}_{n}$, $\operatorname*{ess\,sup}_{x \in S}\vert \hat{g} - g\vert = O_{\mathbb{P}}\(a^{-\frac{1}{2}}_{n}\right)$.
    Therefore, the second step convergence rate can at least speed up to $a_n^{-1-\frac{1}{2}}$.
    For general $L^{p}$-norm convergence, we then speed up to $a_n^{-1-\frac{p-1}{p}}$.
    Ours main results in Theorem \ref{treatment effect allocation simple case}, Theorem \ref{classification simple case} and Corollary \ref{multiplier bootstrap} correspond to the case $p = \infty$.
\end{remark}
\end{comment}

\section{Simulation and empirical application}

\subsection{Confidence intervals and hypothesis testing of ROC curves}\label{simulation data CI and testing}



To evaluate the finite sample performance of our asymptotic results on the ROC curves,
 we first study the simplified asymptotic distribution of the ROC curves under correct specification.
 Next, we consider a synthetic data example and illustrate an important application of comparing a machine generated ROC curve with a human decision maker.

In our first simulation, we generate data according to a logit model:
\begin{align}\begin{split}\nonumber
Y_{i} = 1\left(\frac{e^{X^{T}_{i}\theta_{0}}}{1 + e^{X^{T}_{i}\theta_{0}}} > U_{i}\right), \theta^{T}_{0} = \left(1, 0.8, 0.5\right), U_{i} \sim Uniform\left(0, 1\right), X_{i} \sim Epanechnikov\left(1\right)^{\bigotimes 3}.
\end{split}\end{align}
The Epanechnikov distribution with the parameter $c = 1$ has the density function
\begin{align}\begin{split}\nonumber
f\(x; c = 1\right) = \frac{3}{4c}\max\left\{0, 1 - \left(\frac{x}{c}\right)^{2}\right\}\bigg\vert_{c = 1}.
\end{split}\end{align}

We examine the empirical coverage frequencies and average
lengths of three types of confidence intervals (hereafter abbreviated as CI) for the ROC curve of the
correctly specified logit model.
The first $CI_{bootstrap}$ is formed via the orthogonal estimator using the maximum likelihood estimator (MLE) of the logit model and the bootstrapping method in Corollary \ref{roc bootstrap}.
The second $CI_{theoretical}$ is formed via the MLE orthogonal estimator and the analytic asymptotic distribution in \eqref{theoretical distribution of correctly specified ROC model}.
The third $CI_{model \; based}$ is formed by bootstrapping the model based ROC curve described in Proposition \ref{ROC parametric model based estimator more efficient than orthogonal estimator}.
To calculate the $CI_{bootstrap}$ under
correctly specification we do not need to reestimate the logit model for each bootstrap sample.
The constructions of the ROC estimators and three types of CIs are summarized in Algorithm \ref{algorithm three CI}.

\begin{algorithm}
\caption{ROC estimators and CIs}\label{algorithm three CI}
\KwData{$\left\{ \(X_{i}, Y_{i}\right)\right\}_{i = 1}^{n}$,
$type \in \left\{bootstrap, theoretical, model \; based\right\}$, FPR level grid $G$, number of bootstrap repetitions $R$.}
\KwResult{ROC curve $\left[\left(\alpha, \hat{\beta}\left(\alpha\right)\right): \alpha \in G\right]$, pointwise CI $\left[\left(\beta_{l}\left(\alpha\right), \beta_{u}\left(\alpha\right)\right): \alpha \in G\right]$.}
Estimate propensity score model parameter $\hat{\theta}$; \\
\eIf{$type \neq model \;based$}{
\lForEach{$\alpha \in G$}{Estimate $\hat{\beta}\left(\alpha\right) = \beta\left(\mathbb{Q}_{n}, \hat{\theta}, \alpha\right)$ via \eqref{double robust representing of roc constrained optimized value function under full observability}
}
\eIf{$type = bootstrap$}{
\For{$r \in \{1, 2, \ldots, R\}$}{
\lForEach{$\alpha \in G$}{Estimate $\tilde{\beta}_{r}\left(\alpha\right) = \beta\left(\tilde{\mathbb{Q}}_{n, r}, \hat{\theta}, \alpha\right)$
}}
}
{
Calculate CI via \eqref{theoretical distribution of correctly specified ROC model} \;
}
}{
\lForEach{$\alpha \in G$}{
Estimate $\hat{\beta}\left(\alpha\right) = \beta\left(\hat{\mu}, \hat{\theta}, \alpha\right)$ via \eqref{roc model-based functional expression}
}
\For{$r \in \left\{1, 2, \ldots, R\right\}$}{
Re-estimate propensity score parameter $\tilde{\theta}_{r}$ using the bootstrapped sample \;
\lForEach{$\alpha \in G$}{
Estimate $\tilde{\beta}_{r}\left(\alpha\right) = \beta\left(\tilde{\mu}_{r}, \tilde{\theta}_{r}, \alpha\right)$
}}
}
\If{$type \neq theoretical$}{
\lForEach{$\alpha \in G$}{
Calculate the reverse percentile CI via
\begin{align}\begin{split}\nonumber
\left(\beta_{l}\left(\alpha\right), \beta_{u}\left(\alpha\right)\right) \coloneqq
\left(2\hat{\beta}\left(\alpha\right) - \tilde{\beta}\left(\alpha\right)_{\left(1 - \tau / 2\right)},
2\hat{\beta}\left(\alpha\right) - \tilde{\beta}\left(\alpha\right)_{\left(\tau / 2\right)}\right),
\end{split}\end{align}
where $\tilde{\beta}\left(\alpha\right)_{\left(\tau\right)}$ is the $\tau$-quantile of $\left\{\tilde{\beta}_{r}\left(\alpha\right)\right\}_{r = 1}^{R}$
}
}
\end{algorithm}

We use $R = 1000$ bootstrap repetitions each in 1000 Monte Carlo simulations for sample sizes $n \in \{1000, 2000, 5000, 10000\}$.
We evaluate the pointwise CI at FPR level between $0.3$ and $0.7$ with stepsize $0.05$.
Since the analytic expression for the population ROC curve is
difficult to derive, we
calculate an approximation by simulating a large sample size of $5 \cdot 10^{6}$ from the data generating model.
As shown in Table \ref{coverage and average length table}, the coverage is very close to the nominal level $95\%$.
The average lengths of $CI_{bootstrap}$ and $CI_{theoretical}$ are nearly the same, while $CI_{model \; based}$
is saliently the shortest.

\begin{table}[htbp]
  \centering
  \tabcolsep = 3pt
    \caption{Coverage and average lengths of the confidence intervals}\label{coverage and average length table}
  \begin{tabular}{lccccccccc}
    \toprule
    \multicolumn{10}{l}{\textbf{Panel A}: $n = 1000$} \\
    \midrule
    FPR level & $0.30$ & $0.35$ & $0.40$ & $0.45$ & $0.50$ & $0.55$ & $0.60$ & $0.65$ & $0.70$ \\
    \hline
    \multirow{2}{*}{$CI_{bootstrap}$} & 0.938 & 0.929 & 0.924 & 0.931 & 0.929 & 0.932 & 0.931 & 0.928 & 0.924 \\
    & (0.128) & (0.124) & (0.119) & (0.114) & (0.108) & (0.101) & (0.0932) & (0.0857) & (0.0772) \\
    \multirow{2}{*}{$CI_{theoretical}$} & 0.954 & 0.947 & 0.95 & 0.949 & 0.952 & 0.952 & 0.939 & 0.946 & 0.946 \\
    & (0.127) & (0.123) & (0.118) & (0.113 ) & (0.106) & (0.0997) & (0.0925) & (0.0847) & (0.0764) \\
    \multirow{2}{*}{$CI_{model \; based}$} & 0.931 & 0.933 & 0.934 & 0.936 & 0.939 & 0.937 & 0.936 & 0.935 & 0.936 \\
    & (0.0987) & (0.0965) & (0.0929) & (0.088) & (0.0821) & (0.0754) & (0.0681) & (0.0603) & (0.0522) \\
    \midrule
    \multicolumn{10}{l}{\textbf{Panel B}: $n = 2000$} \\
    \midrule
    FPR level & $0.30$ & $0.35$ & $0.40$ & $0.45$ & $0.50$ & $0.55$ & $0.60$ & $0.65$ & $0.70$ \\
    \hline
    \multirow{2}{*}{$CI_{bootstrap}$} & 0.954 & 0.952 & 0.938 & 0.955 & 0.931 & 0.956 & 0.939 & 0.95 & 0.929 \\
    & (0.0906) & (0.0872) & (0.0841) & (0.0801) & (0.0759) & (0.0714) & (0.0661) & (0.0601) & 0.0545 \\
    \multirow{2}{*}{$CI_{theoretical}$} & 0.965 & 0.967 & 0.956 & 0.961 & 0.954 & 0.959 & 0.951 & 0.959 & 0.956 \\
    & (0.0897) & (0.0871) & (0.0837) & (0.0798) & (0.0754) & (0.0706) & (0.0654) & (0.0599) & (0.054) \\
    \multirow{2}{*}{$CI_{model \; based}$} & 0.947 & 0.95 & 0.952 & 0.953 & 0.955 & 0.955 & 0.956 & 0.958 & 0.958 \\
    & (0.0708) & (0.0692) & (0.0666) & (0.0631) & (0.0589) & (0.0541) & (0.0489) & (0.0433) & (0.0374) \\
    \midrule
    \multicolumn{10}{l}{\textbf{Panel C}: $n = 5000$} \\
    \midrule
    FPR level & $0.30$ & $0.35$ & $0.40$ & $0.45$ & $0.50$ & $0.55$ & $0.60$ & $0.65$ & $0.70$ \\
    \hline
    \multirow{2}{*}{$CI_{bootstrap}$} & 0.936 & 0.931 & 0.936 & 0.942 & 0.942 & 0.953 & 0.952 & 0.946 & 0.933 \\
    & (0.0571) & (0.0555) & (0.0532) & (0.0506) & (0.0478) & (0.045) & (0.0416) & (0.0381) & (0.0345) \\
    \multirow{2}{*}{$CI_{theoretical}$} & 0.944 & 0.938 & 0.943 & 0.943 & 0.953 & 0.96 & 0.956 & 0.951 & 0.95 \\
    & (0.0568) & (0.0551) & (0.053) & (0.0505) & (0.0477) & (0.0447) & (0.0414) & (0.0379) & (0.0342) \\
    \multirow{2}{*}{$CI_{model \; based}$} & 0.944 & 0.938 & 0.943 & 0.943 & 0.953 & 0.96 & 0.956 & 0.951 & 0.95 \\
    & (0.0453) & (0.0443) & (0.0427) & (0.0404) & (0.0377) & (0.0347) & (0.0314) & (0.0277) & (0.024) \\
    \midrule
    \multicolumn{10}{l}{\textbf{Panel D}: $n = 10000$} \\
    \midrule
    FPR level & $0.30$ & $0.35$ & $0.40$ & $0.45$ & $0.50$ & $0.55$ & $0.60$ & $0.65$ & $0.70$ \\
    \hline
    \multirow{2}{*}{$CI_{bootstrap}$} & 0.945 & 0.947 & 0.943 & 0.96 & 0.937 & 0.943 & 0.943 & 0.961 & 0.949 \\
    & (0.0404) & (0.039) & (0.0376) & (0.0359) & (0.0337) & (0.0317) & (0.0293) & (0.0269) & (0.0243) \\
    \multirow{2}{*}{$CI_{theoretical}$} & 0.946 & 0.953 & 0.952 & 0.964 & 0.95 & 0.956 & 0.951 & 0.96 & 0.949 \\
    & (0.0402) & (0.039) & (0.0375) & (0.0357) & (0.0338) & (0.0316) & (0.0293) & (0.0268) & (0.0242) \\
    \multirow{2}{*}{$CI_{model \; based}$} & 0.946 & 0.951 & 0.948 & 0.951 & 0.947 & 0.946 & 0.945 & 0.944 & 0.943 \\
    & (0.0321) & (0.0314) & (0.0303) & (0.0287) & (0.0268) & (0.0246) & (0.0222) & (0.0197) & (0.017) \\
    \bottomrule
  \end{tabular}
\end{table}

Next, we consider a problem of comparing algorithms with decision making of ``doctors'' using synthetic data.
Consider i.i.d. observations $\{\(Y_i, W_i, X_i\right), i = 1, \ldots, n\}$, such that $Y_i$ is the true outcome, $W_i$ is the doctor's diagnosis,
and $X_i$ are observable features.
To compare an algorithm with a doctor,
we test the null hypothesis that the human TPR/FPR pair lies on the machine ROC curve against the alternative hypotheses that the human pair lies above or below the ROC curve.
We generate synthetic data according to the following model
\begin{align}\begin{split}\nonumber
& Y_{i} = 1\left(\frac{e^{X^{T}_{i}\theta_{0}}}{1 + e^{X^{T}_{i}\theta_{0}}} > U_{i}\right), \; \theta^{T}_{0} = \left(1, 0.8, 0.5, 1\right), \\
& W^{E}_{i} = 1\left(\frac{e^{X^{T}_{i}\theta_{E}}}{1 + e^{X^{T}_{i}\theta_{E}}} > 0.5\right), \; \theta_{E} = \theta_{0}, \;
W^{I}_{i} = 1\left(\frac{e^{X^{T}_{i}\theta_{I}}}{1 + e^{X^{T}_{i}\theta_{I}}} > 0.5\right), \; \theta_{I} = \left(1, 0, 0.5, 0\right), \\
& X_{i} \sim N\left(0, I_{4}\right), U_{i} \sim Uniform\left(0, 1\right),
\end{split}\end{align}
where $I_{4}$ is the $4$-dimensional identity matrix.
The $W^{E}_{i}$ decisions are made by an ideal ``experienced doctor'' who knows the data generating process, while the $W^{I}_{i}$ decisions are made by an ``inexperienced doctor'' who only utilizes $X_{1}$ and $X_{3}$.
Theorem \ref{general k derivative} is applicable to this model.

A human decision maker (e.g. doctor, judge and admission officer) is unlikely to make a large number of decisions.
We set the simulation sample size $n$ to	 $\{100, 250, 500, 1000, 2000, 5000, 10000\}$.
We first use Algorithm \ref{algorithm comparison CI} with $R = 1000$ to form a CI for the difference $\beta\left(\alpha_{doc}\right) - \beta_{doc}$.
We then determine whether $0$ is within, above or below the interval.
In the second or third cases,
we reject the hypothesis of equal performance between the doctor and the machine model in favor of either the doctor
or the model.

\begin{algorithm}
\caption{ROC and TPR/FPR comparison CI}\label{algorithm comparison CI}
\KwData{$\left\{ \(X_{i}, W_{i}, Y_{i}\right)\right\}_{i = 1}^{n}$, the number of bootstrap repetitions $R$.}
\KwResult{CI $\left(\beta_{l}, \beta_{u}\right)$.}
Estimate propensity score model parameter $\hat{\theta}$ \;
Estimate $\hat{\beta}\left(\hat{\alpha}_{doc}\right) = \beta\left(\mathbb{Q}_{n}, \hat{\theta}, \hat{\alpha}_{doc}\right)$ via \eqref{double robust representing of roc constrained optimized value function under full observability}; \\%, where $\hat{\alpha}_{doc}$ is the doctor's FPR on $\left\{ \(X_{i}, W_{i}, Y_{i}\right)\right\}_{i = 1}^{n}$; \\
\For{$r \in \left\{1, 2, \ldots, R\right\}$}{
Re-estimate propensity score parameter $\tilde{\theta}_{r}$ using the bootstrapped sample \;
Estimate $\tilde{\beta}_{r}\left(\tilde{\alpha}_{doc}\right) = \beta\left(\tilde{\mathbb{Q}}_{n}, \tilde{\theta}_{r}, \tilde{\alpha}_{doc}\right)$ and calculate $\left(\tilde{\alpha
}_{doc}, \tilde{\beta}_{doc}\right)$ \;
}
Calculate the reverse percentile CI via
\begin{align}\begin{split}\nonumber
\left(2\left(\hat{\beta}\left(\hat{\alpha}_{doc}\right) - \hat{\beta}_{doc}\right) - \left(\tilde{\beta}\left(\tilde{\alpha}_{doc}\right) - \tilde{\beta}_{doc}\right)_{\left(1 - \tau / 2\right)},
2\left(\hat{\beta}\left(\hat{\alpha}_{doc}\right) - \hat{\beta}_{doc}\right) - \left(\tilde{\beta}\left(\tilde{\alpha}_{doc}\right) - \tilde{\beta}_{doc}\right)_{\left(\tau / 2\right)}\right).
\end{split}\end{align}
\end{algorithm}

\begin{table}[!htp]
  \centering
  \tabcolsep = 3pt
    \caption{ROC curve and doctor test rejection frequencies}
  \label{roc doctor test table}
  \begin{tabular}{llccccccc}
    \toprule
    \multicolumn{9}{l}{\textbf{Panel A}: nominal level 5\%} \\
    \midrule
    sample size $n$ & & $100$ & $250$ & $500$ & $1000$ & $2000$ & $5000$ & $10000$ \\
    \hline
    \multirow{3}{*}{experienced doctor} & reject in favor of model & 0.006 & 0.001 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
    & reject in favor of doctor & 0.166 & 0.398 & 0.712 & 0.936 & 1.0 & 1.0 & 1.0 \\
    & fail to reject & 0.827 & 0.601 & 0.288 & 0.064 & 0.0 & 0.0 & 0.0 \\
    \hline
    \multirow{3}{*}{inexperienced doctor} & reject in favor of model & 0.212 & 0.344 & 0.525 & 0.79 & 0.981 & 1.0 & 1.0 \\
    & reject in favor of doctor & 0.004 & 0.001 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
    & fail to reject & 0.783 & 0.654 & 0.475 & 0.21 & 0.019 & 0.0 & 0.0 \\
    \hline
    \multirow{3}{*}{model-like case} & reject in favor of model & 0.027 & 0.034 & 0.026 & 0.027 & 0.02 & 0.014 & 0.017 \\
    & reject in favor of doctor & 0.026 & 0.019 & 0.023 & 0.016 & 0.02 & 0.015 & 0.021 \\
    & fail to reject & 0.946 & 0.947 & 0.951 & 0.957 & 0.96 & 0.971 & 0.962 \\
    \midrule
    \multicolumn{9}{l}{\textbf{Panel B}: nominal level 10\%} \\
    \midrule
    sample size $n$ & & $100$ & $250$ & $500$ & $1000$ & $2000$ & $5000$ & $10000$ \\
    \hline
    \multirow{3}{*}{experienced doctor} & reject in favor of model & 0.008 & 0.004 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
    & reject in favor of doctor & 0.262 & 0.524 & 0.806 & 0.97 & 1.0 & 1.0 & 1.0 \\
    & fail to reject & 0.728 & 0.471 & 0.194 & 0.03 & 0.0 & 0.0 & 0.0 \\
    \hline
    \multirow{3}{*}{inexperienced doctor} & reject in favor of model & 0.295 & 0.454 & 0.627 & 0.873 & 0.995 & 1.0 & 1.0 \\
    & reject in favor of doctor & 0.008 & 0.002 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
    & fail to reject & 0.696 & 0.544 & 0.373 & 0.127 & 0.005 & 0.0 & 0.0 \\
    \hline
    \multirow{3}{*}{model-like case} & reject in favor of model & 0.073 & 0.067 & 0.055 & 0.053 & 0.052 & 0.041 & 0.032 \\
    & reject in favor of doctor & 0.049 & 0.04 & 0.037 & 0.036 & 0.044 & 0.035 & 0.053 \\
    & fail to reject & 0.874 & 0.891 & 0.908 & 0.911 & 0.904 & 0.924 & 0.915 \\
    \bottomrule
  \end{tabular}
\end{table}


Table \ref{roc doctor test table} shows the empirical rejection frequencies for nominal 5\% and 10\% tests.
In the first two entries of both panels,
the ``experienced doctor'' and the ``inexperienced doctor'' are compared with a misspecified logit model using only $\(X_{1}, X_{2}, X_{3}\right)$.
Additionally, in the last entry of both panels labelled ``model-like case'',
we compare the same ``experienced doctor'' with a correctly specified and estimated logit model using all of $\(X_{1}, X_{2}, X_{3}, X_{4}\right)$.
When the sample size is only $100$, the test has low power but rarely rejects in the wrong direction.
A high power level is achieved when the sample size reaches $500$ or $1000$.
The ``model-like case'' represents the null hypothesis of equal performance between
 the ``experienced doctor'' and the model.
As shown in Table \ref{roc doctor test table}, the empirical size of the test is close to the nominal level.
The simulation results validate the finite sample quality of our asymptotic approximation theory.


\subsection{Real data application and simulation}\label{empirical analysis}

In this section, we apply the plug-in multi-class classification method in section \ref{fast convergence rates} to a voting dataset of the August 2006 primary
election in Michigan.
This dataset originates from the field experiment reported in \cite{gerber2008social}.\footnote{We download the dataset from \url{https://github.com/gsbDBI/ExperimentData/tree/master/Social}.}
The optimal voting incentive policy has been studied in the preprint version of \cite{zhou2023offline} using their decision-tree search method.\footnote{See the
preprint version of \cite{zhou2023offline} at \url{https://arxiv.org/pdf/1810.04778}.}


The voting dataset contains 180002 electorates from households spanning Michigan.
Following \cite{zhou2023offline}, we use $10$ features in the dataset: year of birth, sex, household size, city, turnout for 2000, 2002 and 2004 primary and general elections.
We experiment with applying both the multilayer perceptron (MLP) algorithm and the causal forest (CF) algorithm (developed in
\cite{wager2018estimation} and \cite{athey2019generalized}) to the voting data set.

There are four treatment groups and a control group in the voting field experiment.
Hereafter, the control group will be called \textbf{Control}, and the four treatment groups will be called \textbf{Civic Duty}, \textbf{Hawthorne}, \textbf{Self} and \textbf{Neighbors}.
See \cite{gerber2008social} for the specific meaning of those treatments.
Electorates are assigned to the five groups at random with probabilities $\left(\frac{5}{9}, \frac{1}{9}, \frac{1}{9}, \frac{1}{9}, \frac{1}{9}\right)$.

First, we generate a synthetic dataset similar to the voting experiment
in order to validate the multiplier bootstrapping method described in Corollary \ref{multiplier bootstrap}.
The data generating process is a MLP model with $1$ hidden layer of size $8$, ReLU activation and a softmax function (i.e. standard logistic function).
The outcome is then generated according to
\begin{align}\begin{split}\nonumber
& \mathbb{P}\(Y_{ij} = 1 \vert X_{i}\right) = \text{softmax} \circ \(W_{2} \circ \text{ReLU} \circ \(W_{1}\(X^{T}_{i}, j\right)^{T} + b_{1}\right) + b_{2}\right), \quad j \in \{0, 1, 2, 3, 4\}, \\
& Y_{i} = \sum^{4}_{0}Y_{ij}D_{ij}, \quad D_{i} \sim multinomial\left(1, \left(\frac{5}{9}, \frac{1}{9}, \frac{1}{9}, \frac{1}{9}, \frac{1}{9}\right)\right),
\end{split}\end{align}
where
the treatment assignment probabilities are independent of the characteristics $X_i$.  We let $X_{i} \sim N\left(\mu, \Sigma\right)$, where $\mu$ and $\Sigma$ are specified in Appendix \ref{parameters}.
The weights $W_{1}$, $W_{2}$, $b_{1}$ and $b_{2}$ are randomly drawn.
The values of the weights matrices are also reported in Appendix \ref{parameters}.

Using this data generating process, we first draw a large sample of size $10^{7}$ to compute a close approximation of the social welfare potential $\beta_{0}$
around $0.311$ .
An overparameterized MLP with $2$ hidden layers of size (48, 24) is used in estimation.
We then use
the multiplier bootstrap method ($1000$ repetitions) in section \ref{inference on social welfare potential function} to construct CI
based on a sample splitting scheme of $20000$ training and $5000$ testing observations.
To the best of our knowledge, there is no current guarantee of the almost sure uniform convergence rate for neural networks.
However, inspired by the theory in \cite{bos2022convergence}, it is reasonable to expect a fast uniform convergence rate of the MLP classifier on
 a large probability set of $X_{i}$.
In $1000$ simulations, our multiplier bootstrap CI has an empirical coverage rate of $0.939$, close to the nominal $0.95$ level.
The average length of the CI is around $0.0423$.

Next, two machine learning methods are applied to this data.
The first method is a MLP with three hidden layers of size $\left(64, 64, 64\right)$ and ReLU activation (i.e. $\max\{0, x\}$). The second method is a CF with $500$ trees where
the minimum number of samples for each group in each leaf is equal to $600$.
We use the Python package scikit-learn \citep{scikit-learn} for the MLP and the Python implement from the EconML package \citep{econml} for the CF.
The treatment group of each electorate is concatenated to the MLP input features as a $1$-dimensional variable taking value $0$ to $4$ (in the order listed above).

We follow \cite{zhou2023offline} and use $5$-fold cross validation.
We report the estimated difference and CI of the
difference between the plug-in policy and the control group based on
\eqref{DML social welfare potential treatment}.
 Note that \eqref{influence function multiclass policy learning} is Neyman orthogonal and amenable to a sample splitting scheme
described in Algorithm \ref{cross validation multi} with $R = 1000$ and $K = 5$.

\begin{algorithm}
\caption{Policy gain estimation based on cross validation}\label{cross validation multi}
\KwData{$\left\{\(X_{i}, D_{i}, Y_{i}\right)\right\}_{i = 1}^{n}$, the number of bootstrap iterations $R$, cross validation folds $K$, policy for comparison $\hat{\phi}_{c}\left(\cdot\right)$.}
\KwResult{gain $\hat{\beta}$, CI $\left(\beta_{l}, \beta_{u}\right)$.}
Split the sample $\left\{\(X_{i}, Y_{i}\right)\right\}_{i = 1}^{n}$ to $K$ folds \;
\For{$k \in \{1, 2, \ldots, K\}$}{
Estimate propensity score model $\hat{p}_{k}$ and conditional welfare $\hat{g}_{k}$ using the data from folds $\{1, 2, \ldots, k - 1, k + 1, \ldots, K\}$ \;
Estimate $\hat{\beta}_{k}$ via \eqref{DML social welfare potential treatment} on the $k$-th fold of the data \;
}
Estimate $\hat{\beta} = \frac{1}{K}\sum_{k = 1}^{K}\hat{\beta}_{k}$ \;
\For{$r \in \left\{1, 2, \ldots, R\right\}$}{
Estimate
\begin{align}\begin{split}\nonumber
\tilde{\beta}_{r} & =
\frac{1}{n} \sum_{k = 1}^{K}\sum_{i=1}^{n_{k}}
W_{n_{k}i}^r\left(\sum_{j=0}^J \lambda_j \left(\hat \phi_{kj}\(X_{ki}\right) - \hat{\phi}_{cj}\(X_{ki}\right)\right)
\left[
	\hat g_{kj}\(X_{ki}\right) + \frac{D_{kij}}{\hat p_{kj}\(X_{ki}\right)} \(Y_{ki} - \hat g_{kj}\(X_{ki}\right)\right)
\right]\right),
\end{split}\end{align}
where $\left\{\(X_{ki}, D_{ki}, Y_{ki}\right)\right\}_{i = 1}^{n_{k}}$ is the $k$-th fold of the data \;
}
Calculate $\left(\beta_{l}, \beta_{u}\right)$ via the reverse percentile method.
\end{algorithm}

Both the MLP and the CF assign nearly all electorates to the \textbf{Neighbors} group.
Meanwhile, the two models give value differences of $0.0865$ and $0.0837$ compared to \textbf{control}.
The CIs for the true $\beta_{0}$ are reported in Table \ref{voting table}.
These findings are similar to the value difference around $0.0864$ using the ERM policy of \cite{zhou2023offline}.
Their preprint version
reported a value difference of $0.082$ for the same ERM policy.
The difference is due to the randomness from sample splitting and model training.
The policy gain results need to be interpreted with caution.
As shown in Table \ref{voting table},
the CIs for the policy gain between the MLP / CF and the ERM policy include $0$.
Given that the performance of our proposed method is statistically on par with ERM,
the method that we propose provides a viable computationally practical alternative approach to ERM policy learning.

\begin{table}[!htp]
  \centering
  \tabcolsep = 3pt
    \caption{Policy gain estimation comparison}
  \label{voting table}
  \begin{tabular}{lccc}
    \toprule
     & ERM & MLP & CF \\
    \midrule
    v.s. \textbf{control} & 0.0864 $\left(0.0769, 0.0954\right)$ & 0.0865 $\left(0.0753, 0.0965\right)$ & 0.0837 $\left(0.0748, 0.0937\right)$ \\
    \hline
    v.s. ERM & - & 0.000271 $\left(-0.00232, 0.00302\right)$ & -0.00261 $\left(-0.00669, 0.00173\right)$ \\
    \bottomrule
  \end{tabular}
  \captionsetup{font=small}\caption*{\textit{Notes}: In this table, we first compare the ERM policy from \cite{zhou2023offline} and our method based on the MLP and the CF with \textbf{control}.
  When comparing the ERM policy and \textbf{control}, we apply \eqref{DML social welfare potential treatment} but let $\hat{\phi}$ be the fixed ERM policy.
  We use the CF algorithm to construct $\hat{g}$ to evaluate the comparison between the ERM policy and \textbf{control}.
  CIs are then constructed for the policy gain difference between the MLP / CF and the ERM.
  99\% confidence intervals are reported in the parentheses.
  }
\end{table}


\begin{comment}
\begin{python}
import numpy as np
from matplotlib import rcParams
from matplotlib import pyplot as plt
from sklearn.model_selection import StratifiedKFold
import joblib
from tqdm.notebook import tqdm, trange
import pandas as pd
from pathlib import Path
from sklearn.neural_network import MLPClassifier

WORKSPACE_PATH = Path().resolve()
DATA_PATH = WORKSPACE_PATH / 'social_data'
csv_files = [DATA_PATH / f"socialpress{app}.csv" for app in ['nofact'] + \
	     [f"wgeooneperhh{app2}" for app2 in ('', '_CIVIC', '_HAWTHORNE', '_NEIGH', '_SELF')]]

USE_COLUMNS = ['sex', 'yob', 'g2000', 'g2002', \
	       'g2004', 'p2000', 'p2002', 'p2004', 'hh_size', 'treatment']

data = pd.read_csv(csv_files[1])
mapper = {'yes': 1, 'no': 0}
data['p2004'] = data['p2004'].apply(lambda x: mapper[x])
data['treatment'] = data['treat_civic'] + data['treat_hawthorne'] * 2 + \
data['treat_self'] * 3 + data['treat_neighbors'] * 4
features = data[USE_COLUMNS].to_numpy()
truth = data['outcome_voted'].to_numpy()

SKF = StratifiedKFold(n_splits = 5)
result_list = []
for fold_index, (train_index, test_index) in enumerate(SKF.split(features, truth)):
    model = MLPClassifier(hidden_layer_sizes=(128, 128), verbose=True)
    model.fit(features[train_index], truth[train_index])
    results = np.stack([model.predict_proba(
		np.concatenate(
		    (features[test_index][:, :-1], np.full((len(test_index), 1), treatment)),
		    axis=1)
		)[:, 1] for treatment in range(5)])
    j = results.argmax(axis=0)
    p = np.array([10/18, 2/18, 2/18, 2/18, 2/18])
    for j in (results.argmax(axis=0), np.zeros_like(j)):
	beta_hat = np.mean(results[j, np.arange(results.shape[1])] + \
			   (features[test_index][:, -1] == j).astype(np.int32) * \
			   (truth[test_index] - results[j, np.arange(results.shape[1])]) / p[j])
	result_list.append(beta_hat)
\end{python}
\end{comment}



\begin{comment}
\begin{python}
import numpy as np
from matplotlib import rcParams
from matplotlib import pyplot as plt
from sklearn.model_selection import StratifiedKFold
import joblib
from tqdm.notebook import tqdm, trange
import pandas as pd
from pathlib import Path
from sklearn.neural_network import MLPClassifier
from econml.grf import CausalForest, CausalIVForest

WORKSPACE_PATH = Path().resolve()
DATA_PATH = WORKSPACE_PATH / 'social_data'
csv_files = [DATA_PATH / f"socialpress{app}.csv" for app in ['nofact'] + \
	     [f"wgeooneperhh{app2}" for app2 in ('', '_CIVIC', '_HAWTHORNE', '_NEIGH', '_SELF')]]

USE_COLUMNS = ['sex', 'yob', 'g2000', 'g2002', 'g2004', 'p2000', 'p2002', \
	       'p2004', 'hh_size', 'treatment']

data = pd.read_csv(csv_files[1])
mapper = {'yes': 1, 'no': 0}
data['p2004'] = data['p2004'].apply(lambda x: mapper[x])
data['treatment'] = data['treat_civic'] + data['treat_hawthorne'] * 2 + \
data['treat_self'] * 3 + data['treat_neighbors'] * 4
features = data[USE_COLUMNS].to_numpy()
truth = data['outcome_voted'].to_numpy()

SKF = StratifiedKFold(n_splits=5)
result_list = []
for fold_index, (train_index, test_index) in enumerate(SKF.split(features, truth)):
    model = CausalForest(n_estimators=500, min_samples_split=600, min_samples_leaf=600)
    treatment = features[train_index][:, -1]
    sta_treatment = np.stack([treatment == x for x in range(1, 5)], axis=-1).astype(np.int32)

    model.fit(features[train_index][:, :-1], sta_treatment, truth[train_index])
    theta_hat = model.predict_full(features[test_index][:, :-1])
    results = np.concatenate((
	theta_hat[:, -1:],
	theta_hat[:, :-1] + theta_hat[:, -1:]
    ), axis=-1).T
    p = np.array([10/18, 2/18, 2/18, 2/18, 2/18])
    for j in (results.argmax(axis=0).astype(np.int32), np.zeros(results.shape[1], dtype=np.int32)):
	beta_hat = np.mean(results[j, np.arange(results.shape[1])] + \
			   (features[test_index][:, -1] == j).astype(np.int32) * \
			   (truth[test_index] - results[j, np.arange(results.shape[1])]) / p[j])
	result_list.append(beta_hat)
\end{python}
\end{comment}

\section{Conclusion}
In this paper, we explore a functional differentiability approach for a class of statistical optimal allocation
problems.
Inspired by \cite{chernozhukov2018sorted}, Hadamard differentiability is facilitated by a study of the general properties of the sorting operator.
We provide a derivation that makes use of the concept of Hausdorff measure and the area and coarea formulas from geometric measure theory.
Based on our general Hadamard differentiability results, in section \ref{constrained and roc} we derive
the asymptotic properties of the plug-in estimators for both the value function process of a binary constrained optimal allocation problem and the ROC curve from the functional delta method.
When the first step propensity score model is correctly specified, a computationally feasible bootstrap procedure is validated for the plug-in ROC estimator.

Importantly, we build on the partial convexity of the social welfare potential function, i.e. the value function of the optimal allocation problem,
to demonstrate the degeneracy of the first order derivative
of the social welfare potential function with respect to the policy.
These intriguing results provide us with insights
when we
combine techniques from the literature of nonsmooth method of moment, plug-in classification and the recent development of double / debiased machine learning
to develop a debiased estimator of the social welfare potential function
in section \ref{fast convergence rates}.
Here, the conditions required for Hadamard differentiability validate the margin assumption, which leads to a faster convergence rate.


\section{Acknowledgments}

We thank the editor and two anonymous referees for insightful comments.
We also acknowledge helpful and encouraging conversations with
collaborators and colleagues
including Chunrong Ai, Timothy Armstrong, Haoge Chang, Qihui Chen, Xiaohong Chen,
Victor Chernozhukov, Michael Fan, Yanqin Fan, Yue Fang, Yingjie Feng,
Robin Han, Toru Kitagawa, Michael P. Leung, Hongjun Li, Jessie Li, Ruixuan Liu, Ye Luo, Thomas MaCurdy, Chen Qiu, Shuyang Sheng, Zhentao Shi, Kurt Sweat, Liangjun Su, Ke Tang, Guanyi Wang, Haitian Xie and Ping Yu, as well as participants at various seminars and conferences.
We are most grateful to Yuhao Xue for enlightening us
with his knowledge of geometry measure theory and to Yichuan Zhang for research assistance.
We acknowledge funding support from the National Science Foundation (SES 1658950 to Han Hong).

\FloatBarrier
\phantomsection
\addcontentsline{toc}{section}{References}
{
\bibliography{forarxiv}
\bibliographystyle{aer}
}


\newpage
\setstretch{1.25}
{\centering{