Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
78,843 characters · 16 sections · 89 citation commands
Data-Driven Policy Learning for Continuous Treatments
Economists are increasingly interested in learning optimal policy from experimental and observational data. The optimal policy maximizes the population welfare over a (possibly restricted) policy space, where a policy maps individual characteristics into a policy treatment. Computing the optimal policy may encounter two challenges. The population welfare may be unknown to the policymaker, and the other is that the policy space may be infinite-dimensional and complex. One general approach is to apply various methods, including the general methodology proposed by ainon, to estimate the population welfare from observational data to obtain empirical welfare and approximate the complex and infinite-dimensional policy space with a sequence of finite spaces ai2003efficient. The learned policy then maximizes the empirical welfare over the approximating space. This general approach may introduce tuning parameters in estimating the population welfare (e.g., bandwidth in kernel estimation of the population welfare) and the approximation (e.g., the dimension of the approximating space). The learned policy requires careful calibration of the tuning parameters to achieve an oracle inequality of welfare regret.
The existing literature on policy learning from observational data has followed the general approach above, but focused mainly on binary treatments. The binary treatment setting has two advantages over general settings. First, the empirical welfare is a simple sample average that is unbiased and free of tuning parameters. Second, the optimal policy may have an analytical expression, thereby does not need approximation. For example, when the policy space is unrestricted, the optimal policy is an indicator function of the conditional average treatment effect (hereafter, CATE), which can be machine-learned from observational data manski2004statistical,manski2007admissible,manski2007minimax,stoye2009minimax,stoye2012minimax,tetenov2012statistical,bhattacharya2012inferring. Under the condition that the machine-learned CATE converges to the truth fast, these studies established a sharp upper bound of the welfare regret. However, when the policy space is restricted, the optimal policy generally does not have an analytical expression, even in the binary setting. Instead of approximating the policy space, kitagawa2018should,kitagawa2021equality parameterized it as a finite, fixed-dimensional space. athey2021policy,zhou2023offline also parameterized it, but allowed the dimension to grow with the sample size. None of those studies has any tuning parameters. Under sufficient conditions, they all established a sharp (i.e., minimax-optimal rate) bound of the welfare regret. mbakop2021model, on the other hand, did not parameterize the policy space and used approximations. The approximation introduces one tuning parameter: the dimension of the approximating space. They suggested a penalized algorithm to data-automate the dimension. However, due to the approximation error, they could not achieve the same rate as in kitagawa2018should,athey2021policy, obtaining instead an oracle inequality of the welfare regret that balances approximation and estimation errors.\footnote{fang2025model used doubly robust moment conditions for welfare estimation and obtained similar results for multivalued discrete treatments.}
In a real-world context, policymakers often use complex policy treatments. For example, they set carbon tax rates or allocate pollution permits in environmental policy, determine the duration of training for various demographic groups in job training programs, adjust cash transfer amounts across different households in conditional cash transfer programs, and set different price levels for different customer groups in retailing. All these policy treatments are continuous. Yet, learning the optimal continuous policy from observational data has received scant attention in the literature.\footnote{Several recent studies have examined continuous treatments in various causal frameworks, such as su2019non,callaway2021difference,xie2024nonlinear,ColangeloLee2025, with a focus on estimating treatment effects or dose–response functions, which differs from the policy learning objective considered here.} The difficulty is in estimating the population welfare because there are few observations at each level of treatment. In principle, policymakers can use observations in the neighborhood of each treatment level to evaluate the policy. But doing so introduces the bias and another tuning parameter (e.g., the neighborhood's size). kallus2018policy took this approach with kernel estimation but considered a finite, fixed-dimensional policy space. So, they have only one tuning parameter, the bandwidth. They did not data-automate the bandwidth and established only an upper bound of the welfare regret, not the oracle inequality.
We are unaware of any work on learning the optimal continuous policy from observational data, with policy space approximated. This paper intends to fill the literature gap. Specifically, we generalize mbakop2021model to a continuous treatment setting by approximating the policy space with finite, growing spaces and applying the kernel method to obtain the empirical welfare. We consider two cases: known and unknown propensity scores. We allow machine learning propensity scores and modify the empirical welfare to account for the machine learning effect. We then maximize the empirical welfare or modified empirical welfare over the approximating space to obtain the learned policy, which depends on the tuning parameters. We then modify the penalized procedure of mbakop2021model and develop a data-automated algorithm for both tuning parameters. Despite the extra tuning parameter, we still establish an oracle inequality of the welfare regret in known and unknown propensity scores. Our oracle inequality in the known propensity score case is similar to that of mbakop2021model. Still, it is sharper in the unknown propensity score case because we use the double debiasing approach.
The extension is nontrivial because the extra tuning parameter (i.e., bandwidth) complicates the data automation algorithm. The bandwidth and the approximating space dimension play different roles. While the choice of dimension directly impacts policy learning performance, bandwidth directly affects policy evaluation performance. The data automation algorithm must consider the individual effects to maximize the policy evaluation and learning performance separately and the interaction effects of the tuning parameters on the learned policy. Despite the complicated calibration procedure, it is worthwhile to automate the tuning parameters since they adapt to the underlying data-generating process without knowing the model's smoothness condition.
To illustrate the practical value of the proposed policy learning, we re-examine the policy of assigning individuals to job training programs of varying durations. We use the same data from the Job Training Partnership Act (JTPA) study as kitagawa2018should,mbakop2021model. While they analyze the binary treatment (i.e., participation in job training), we examine the continuous treatment (i.e., training duration). flores2012estimating noted that the effects of job training on future earnings may vary with the length of exposure to the training program. It is crucial to consider different training durations (as opposed to a binary participation decision) in policy design. Our findings confirm that the learned policy adapts to and reflects the varying training time duration, further highlighting the benefits of moving beyond binary participation frameworks to develop data-driven approaches to policy design.
We organize the remainder of the paper as follows. Section (ref) sets up the model. Section (ref) introduces the data-automation algorithm in the known propensity score setting and establishes the oracle inequalities. Section (ref) extends the analysis to the unknown propensity score setting. Section (ref) discusses examples of policy space approximations. Section (ref) presents an empirical study. The proofs for theoretical results in the main text are collected in the Appendices.
The model consists of a continuous treatment $T$ with support $\mathcal{T} \subset \mathbb{R}$, a set of potential outcomes $\{Y(t)\}_{t\in\mathcal{T}}$, and a vector of covariates $X$ with support $\mathcal{X} \subset \mathbb{R}^{d_X}$. The researcher only observes the realized outcome $Y\equiv Y(T)$, not all potential outcomes. A policy $\pi$ maps the covariate space $\mathcal{X}$ to the treatment space $\mathcal{T}$.
We define the population welfare of a policy $\pi$ as the expected outcome under this policy:
We aim to find the optimal policy within an infinite-dimensional (and possibly restricted) space $\Pi_\infty$. Let $W^*(\Pi_\infty) \equiv \sup_{\pi \in \Pi_\infty} W(\pi)$ denote the global optimal welfare. Following the literature manski2004statistical,kitagawa2018should,athey2021policy, we assess the performance of a policy $\pi$ by the welfare regret, $W^*(\Pi_\infty) - W(\pi)$, the difference between the global optimal welfare and the welfare achieved by $\pi$.
As mbakop2021model explains, economic theory or intuition often imposes nonparametric restrictions on policy classes, such as monotonicity, convexity, super-modularity, or separability. These restrictions restrict the form of the policies but still allow the policy class to remain infinite-dimensional. Another type of restriction is a parameterization of policy. Although parametric restriction reduces the policy space to finite-dimensional, they are generally ad hoc and seldom driven by economic theory. Below, we present some economically meaningful examples.
Researchers often consider continuous piecewise-linear policies defined by a set of thresholds. These rules are simple to communicate and implement. Block-rate taxes in environmental economics zhou2019would and California’s two-tier price collar for carbon emissions are prominent examples, while analogous “step-up” structures appear in loyalty programmes: multi-threshold linear bonuses fang2018loyalty and tiered status schemes in retail nishio2022joint reward early engagement generously before tapering once customers are “locked in”. Implementing such a policy requires choosing the number and placement of thresholds and the slope within each segment. We formalize this policy class in the next example.
Our theory accommodates all types of covariates—continuous, discrete, or mixed. Discrete covariates are theoretically more straightforward, and when all covariates are discrete, the set of all measurable policies is a finite-dimensional space.
We observe an independent and identically distributed (iid) sample $S_n \equiv \{(Y_i,T_i,X_i): 1 \leq i \leq n\}$ drawn from the distribution of the random variables $(Y,T,X)$.
To estimate $W$, let us first recall that, with a discrete treatment, the IPW formula can determine the welfare (under the unconfoundedness condition; see Assumption (ref) below) as\footnote{Because the following formula applies only when the treatment is discrete, we do not denote it by $W$.}
where $f(t|x)$ denotes the generalized propensity score, i.e., the conditional density of the treatment given the covariates. However, this formula is no longer valid in the continuous setting because the indicator $\mathbf{1}\{T=\pi(X)\}$ equals zero almost surely when $T$ is a continuous variable. We propose a kernel weighting,
with $K$ as a kernel function and $h$ as the bandwidth. This leads to the following empirical welfare estimator:
As directly optimizing the infinite-dimensional policy class $\Pi_\infty$ is not practically feasible, we shall use the sieves approximation to the global policy class \( \Pi_\infty \). This is achieved using a nested sequence of low-complexity policy classes $\{\Pi_k : k \geq 1\}$, where \( \Pi_k \subset \Pi_{k+1} \subset \cdots \subset \Pi_\infty \). We call each \( \Pi_k \) a sieve policy class to distinguish it from the global class \( \Pi_\infty \). Each $\Pi_k$ has finite complexity, measured by the Vapnik–Chervonenkis (VC) dimension\footnote{Let $\mathscr{S}$ be a collection of subsets of a set $\Upsilon$. The VC dimension of $\mathscr{S}$ is defined as the largest cardinality of a subset $\upsilon \subset \Upsilon$ that can be shattered by $\mathscr{C}$. The collection $\mathscr{S}$ is said to shatter $\upsilon$ if for each $\tilde{\upsilon} \subset \upsilon$, there exists a set $\mathcal{S} \in \mathscr{S}$ such that $\tilde{\upsilon} = \upsilon \cap \mathcal{S}$. Notice that this definition of VC dimension is the same as in van2009note,kitagawa2018should,wainwright_2019,mbakop2021model, but is smaller by one than the VC dimension defined in wellner1996, dudley_1999.} of the class of subgraphs $\{ \{(x,t):t<\pi(x) \} : \pi\in\Pi \}$. Throughout the paper, we use $\operatorname{VC}(\Pi)$ to denote the VC-subgraph dimension for a generic policy class $\Pi$.
Each sieve optimal policy estimator $\hat{\pi}_{h,k}$ is obtained by maximizing the estimated welfare $\hat{W}_h$ within each sieve policy class $\Pi_k$ as defined later in ((ref)). We want to data-automate $(h,k)$ in a way that delivers a policy estimator $\hat{\pi}_{\hat{h},\hat{k}}$ with good performance. Below, we provide a heuristic discussion of the challenge.
The roles of $h$ and $k$ are asymmetric: $h$ is involved in the evaluation (welfare estimation) stage, whereas $k$ is used during the policy design stage. Examine the following decomposition of the welfare regret:
where $W^*(\Pi_k) \equiv \sup_{\pi \in \Pi_k} W(\pi)$ denotes the optimal welfare over $\Pi_k$, and the second inequality is (2.2) in kitagawa2018should.
We provide a heuristic argument for how the two tuning parameters affect the three terms on the right-hand side of ((ref)). The welfare-deficiency term captures the loss (compared to global optimal welfare) from restricting to the sieve class $\Pi_k$; it depends only on $k$ and decreases as $k$ increases. The variance term, by empirical-process results in kitagawa2018should, is of order $\sqrt{\operatorname{VC}(\Pi_k)/(n h)}$. The kernel bias, by standard analysis, depends only on the bandwidth and is of order $h^r$, where $r$ denotes the smoothness of the (conditional) dose-response functions defined as
which describes the (conditional) mean potential outcome under each treatment level.
To summarize, the right-hand side of ((ref)) is proportional to the following order-of-magnitude:
In terms of minimizing the above sum of three terms, the optimal $(h,k)$ should be chosen such that the three terms are of the same order:
which yields $h \asymp \left( \frac{\operatorname{VC}(\Pi_k)}{n}\right)^{\frac{1}{2r+1}}$. Hence, the optimal choice of $h$ is dependent on $k$, and vice versa. Intuitively, enlarging the policy class raises the variance term $\sqrt{\frac{\operatorname{VC}(\Pi_k)}{nh}}$,\footnote{The phenomenon that a more complex policy class leads to higher variance is the so-called overfitting issue. When $\Pi_k$ is too rich relative to the sample size, the learned policy can fit sampling noise rather than genuine welfare signals, inflating the estimation error.} requiring a larger bandwidth to mitigate this increase.
This interplay illustrates a central difficulty in continuous-treatment policy learning: the tuning parameters associated with welfare evaluation ($h$) and with policy design ($k$) must be selected jointly. A bandwidth chosen in isolation—for example, by standard mean-squared-error bandwidth rules for welfare estimation—ignores its impact on the downstream optimization problem and is therefore generally sub-optimal.
If one instead fixes $h$ and maximizes the smoothed welfare $W_h$, the bias term vanishes and the model-selection method of mbakop2021model applies directly. Our goal, however, is to optimize the actual welfare $W$, which requires balancing variance and kernel bias by adapting $h$ in tandem with the complexity parameter $k$.
This section examines the case in which the propensity score is known to the econometrician. Section (ref) addresses the case where the propensity score is unknown.
The previous welfare decomposition suggests the following data-automated algorithm. Let $\mathcal{H} \subset (0,1)$ be a countable grid of bandwidth. First, we estimate the optimal policy for each pair $(h,k)$ as
Then, we construct the penalized welfare as
where the three terms in the penalty are defined and explained in detail below. The selector for $h$ and $k$ is the maximizer of $\hat{Q}_{h,k}$:\footnote{In practice, when the exact $\operatorname{VC}(\Pi_k)$ is unknown, one can replace it with an upper bound $V_k \geq \operatorname{VC}(\Pi_k)$. This substitution slightly alters the form of the oracle inequalities in Theorems (ref) and (ref), as the infimum is taken over $\{V_k \leq nh^2\}$, which may cover a smaller range of $k$ compared to $\{\operatorname{VC}(\Pi_k) \leq nh^2\}$. Nevertheless, as we demonstrate in Section (ref), for many widely used policy classes—including highly complex ones such as neural networks—there exist well-established results providing tight or nearly tight upper bounds on VC dimension. We thank an anonymous referee for highlighting this point.}
The final policy estimator is $\hat{\pi} \equiv \hat{\pi}_{\hat{h},\hat{k}}$.
Before introducing the penalty terms, we briefly comment on the reason behind the restriction $\operatorname{VC}(\Pi_k) \leq nh^2$ imposed in the search over $(h,k)$ in ((ref)). First, scanning every $k\in\mathbb{N}_{+}$ is computationally unrealistic in practice; second, the continuous-treatment setting presents a technical hurdle discussed in Section (ref). However, this restriction is not substantive in large samples. As discussed in ((ref)), the optimal variance–bias trade-off itself dictates the relationship $\operatorname{VC}(\Pi_k)\asymp nh^{2r+1}, r \geq 1$, which lies safely within the region defined by $\operatorname{VC}(\Pi_k)\le nh^{2}$. Hence, the restriction is asymptotically non-binding. Appendix (ref) outlines a full search procedure over $k\in\mathbb{N}_{+}$ that provides theoretical regret guarantees.
The first penalty term $\hat{R}_{h,k}$ is set to be
where $\{\text{Rad}_i: 1 \leq i \leq n\}$ represents a sequence of computer-generated i.i.d. Rademacher variables, independent of the sample $S_n \equiv \{(Y_i,T_i,X_i): 1 \leq i \leq n\}$. The expectation is taken over the Rademacher variables and is computed through simulations. The term $\hat{R}_{h,k}$, known as the (empirical) Rademacher complexity, is commonly used to penalize overfitting in policy estimators bartlett2002model,mbakop2021model. Another approach to constructing the overfitting penalty involves using the holdout method described in mbakop2021model, which we present in Section (ref).
The second penalty term $\tau(h,k,n)$ is a user-specified technical term that ensures the penalty grows sufficiently fast with $h$ and $k$. Requirements for $\tau$ and specific choices are provided in the theorems. In general, $\tau$ is of smaller order than the leading term in the oracle inequality.
As noted earlier, fixing \( h \) and focusing on maximizing \( W_h \) allows us to implement the penalty as \( \hat{R}_{h,k} + \tau \) following the method in mbakop2021model. However, when the objective is \( W \), the penalty \( \hat{R}_{h,k} + \tau \) does not penalize against large bandwidth, causing uncontrolled kernel bias. Therefore, we introduce a third penalty term to offset this bias in welfare estimation.
The third penalty term $B(h)$ is set to be
where $\mu$ is the conditional expectation function of $Y$ given $T$:
and $V_{\mu}$ is the total variation of $\mu$, and $r$ is the order of smoothness of $\mu$.\footnote{Total variation is defined as $V_{\mu} \equiv \sup_{m \in \mathbb{N}} \sup_{t_0,\cdots,t_m \in \mathcal{T}} \sum_{j=0}^m |\mu(t_j) - \mu(t_{j-1})|$.} $K^{\operatorname{FT}}$ is the Fourier transform of $K$, i.e., $K^{\operatorname{FT}}(\xi) \equiv \int K(t) e^{\mathbf{i}t\xi}dt$, with $\mathbf{i} \equiv \sqrt{-1}$. To avoid confusion with policy $\pi$, we use $\varsigma$ to denote the mathematical constant, the ratio of a circle's circumference to its diameter, approximately 3.14159.
This bias bound builds on the work of schennach2020bias, establishing a tight upper bound for the nonparametric bias in kernel estimation. In our case, as shown in Lemma (ref), $B(h) \asymp h^r$ provides an upper bound for $|W_{h}(\pi) - W(\pi)|$ for any $\pi$. The quantities $r$ and $V_{\mu}$, and subsequently $B(h)$, can be estimated using the following double-debiased modification of the procedure described in schennach2020bias.
Denote $\mu^{\operatorname{FT}}(\xi) \equiv \int \mu(t)e^{\mathbf{i}\xi t}dt$ as the Fourier transformation of $\mu$. This estimand admits the following double-debiased estimator:\footnote{The double-debiased property of this estimator is demonstrated in the proof in the appendix.}
where $\hat{\mu}$ and $\hat{f}_T$ are estimators of the corresponding nuisance functions. As shown in schennach2020bias, applying Fourier transformation converts the order of smoothness $r$ into the exponent in the frequency domain, yielding the bound $|\mu^{\operatorname{FT}}(\xi)| \leq V_{\mu} |\xi|^{-(r+1)}$. Taking the logarithm of both sides results in a linear expression: $\log |\mu^{\operatorname{FT}}(\xi)| \leq \log V_{\mu} - (r+1) \log|\xi|$. Thus, we can estimate $V_{\mu}$ and $r$ by finding the tightest linear upper bound on $\log |\hat{\mu}^{\operatorname{FT}}(\xi)|$ as a function of $\log|\xi|$, described by the following minimization problem:
Plugging these estimates into the bias bound leads to the estimator $\hat{B}(h) \equiv B(h;\hat{r},\hat{V}_{\mu})$.
This concludes our description of the procedure for data-automation of $(h,k)$. Its statistical properties are introduced next.
The following assumptions are maintained regarding the sieve policy classes, the data-generating process, and the kernel function.
Assumption (ref) establishes the identification of the welfare function, under which we can write $m(t,x) = \mathbb{E}[Y|T=t,X=x]$. Assumption (ref) requires that the sequence of sieve policy spaces approaches the global target eventually. In Assumption (ref), we assume that both the outcome and the inverse propensity are bounded. The discrete-treatment version of this assumption is considered in kitagawa2018should as Assumption 2.1 and in mbakop2021model as Assumption 3.1.\footnote{athey2021policy do not require the outcome to be bounded (only requiring its distribution to exhibit sub-Gaussian tail). Still, they do maintain the requirement for the inverse propensity score to be bounded.}
Assumption (ref) specifies that the kernel is of infinite order. See devroye1992note,politis1999multivariate for examples and discussions. The use of an infinite-order kernel is not essential. Any sufficiently high-order kernel that accommodates the smoothness of the dose-response function would be effective. Assumption (ref) is adopted from schennach2020bias and essentially states that the relationship from the treatment to outcome is $r$th-order smooth uniformly over the covariates. Assumption (ref) imposes standard conditions on the nuisance estimators used in constructing the bias bound, ensuring its consistency.
The first part of Theorem (ref) describes the infeasible performance of the policy estimator when the bias bound is known, while the second part presents an oracle inequality when the bias bound is consistently estimated. When the bias bound is estimated, it needs to be inflated by a factor of $1+\gamma$ to ensure it correctly bounds the bias with high probability. In both cases, the procedure can balance the tradeoff among the three terms in the welfare decomposition of ((ref)), introducing a technical term and a small order term $O_p(n^{-1/(2r+1)})$.\footnote{We use an $O_p$ remainder rather than a nonasymptotic high-probability bound in the oracle inequalities due to the estimation of $r$. See the proof of Theorem (ref)(2) for details.} The technical term can be chosen to be dominated by the variance term, as described subsequently.
The term $\tau$ and condition ((ref)) are inevitable artifacts of applying a union bound to control the random selectors $(\hat h,\hat k)$ in ((ref)) when proving Theorem (ref). This proof device—and the resulting technical term—is standard (e.g., mbakop2021model,bartlett2002model,koltchinskii2001rademacher,koltchinskii2011oracle,boucheron2005theory); our setting is even more delicate because it involves two tuning parameters rather than one. Condition ((ref)) imposes a lower bound on $\tau$ (since $nh\tau^{2}$ must be large), but $\tau$ must also remain small relative to the terms in the oracle inequality. Below we provide specific choices of $\tau$, based on feasible choices of the bandwidth grid, to satisfy ((ref)) while staying negligible compared to $\sqrt{\operatorname{VC}(\Pi_k)/(nh)}$:
for any sequences $\lambda_k\uparrow \infty$ as $k \rightarrow \infty$ and $\lambda_h' \uparrow \infty$ as $h \rightarrow 0$. It is worth noting that when the global policy class has a finite VC dimension—e.g., in Example (ref) with $K<\infty$—the technical term reduces to $\tau(h,n)$; the dependence on $k$ disappears because the sum over $k$ is finite. If the bandwidth grid $\mathcal{H}$ is also finite, this term is no longer needed at all.
In large samples, the constraint $\operatorname{VC}(\Pi_k) \le nh^{2}$ in the oracle inequality’s infimum is asymptotically non-restrictive. Let $(h_n^{*},k_n^{*})$ solve the unconstrained problem:
Then
Hence, the unconstrained optimal choice automatically satisfies $ \operatorname{VC}(\Pi_{k_n^{*}}) \le n(h_n^{*})^{2}$, rendering the restriction non-binding.
From ((ref)), we can obtain the rate for the unconstrained optimal bandwidth
Because $\Pi_\infty$ is infinite-dimensional, driving the welfare deficiency to zero requires $k_n^*\to\infty$, so $h_n^*$ is of larger order than the standard optimal $n^{-1/(2r+1)}$ rate for nonparametric kernel regression. This justifies setting $h_{\min}\gtrsim n^{-1/(2r+1)}$ and then searching upward from there.
\paragraph{Comparison with the literature}
Although the work of athey2021policy addressed continuous treatment, their approach examines infinitesimal nudges through outcome derivatives, effectively reducing the problem to a binary treatment scenario. They did not consider a sieve approximation of the policy space.
In contrast, mbakop2021model considered sieve approximation for the binary treatment case. Their Proposition 3.2 derives the IPW (with known propensity score) welfare regret bound, which is
Comparing our bound in Theorem (ref) to mbakop2021model's bound, the differences are as follows. First, our bound includes an additional bias term of order \( h^r \), and the variance term is larger by a factor of \( 1/\sqrt{h} \). This difference arises from the nonparametric kernel estimation required for the continuous treatment. Similarly, the order of the remainder term increases from $n^{-1/2}$ to \( n^{-r/(2r+1)} \), reflecting the standard minimax rate of nonparametric estimation under smoothness $r$ without approximation of the policy space. Additionally, the technical term has been adjusted to account for the data-driven selection of bandwidth.\footnote{In fact, as shown by bartlett2002model, in the binary treatment case, the technical term in the binary treatment case can be made as small as $\sqrt{(\log k)/n}$ instead $\sqrt{k/n}$.}
Second, our bound directly applies to the regret itself, whereas mbakop2021model's bound applies to the expected regret \(\mathbb{E}[W^*(\Pi_\infty) - W(\hat{\pi})]\). That is, we additionally bound (in probability) the deviation of the regret from its mean. However, it is important to note that our result does not imply convergence in expectation, as the \(O_p\)-remainder terms in Theorem (ref) may only converge in the (weaker) notation of convergence probability. This limitation is due to the complexities in estimating the order of smoothness, which typically exhibits a slow convergence rate.\footnote{See, for example, sun2005adaptive, the minimax rate for the order of smoothness is logarithmic in $n$.}
kallus2018policy studied policy learning with continuous treatment, obtaining a regret bound of order $\mathcal{R}_n(\Pi_\infty)/h^2 + \text{bias}(h)$, where $\mathcal{R}_n(\Pi_\infty)$ is the Rademacher complexity of $\Pi_\infty$, defined as $\mathcal{R}_n(\Pi_\infty) \equiv \mathbb{E}\left[ \sup_{\pi \in \Pi_\infty} 2\sum_{i=1}^n \text{Rad}_i \pi(X_i)/n \right]$. There are several key differences between their result and ours. First, they do not consider sieve approximation of the policy space. Second, even without sieve approximation, the bound derived by kallus2018policy is less sharp than our Theorem (ref). This can be seen in a global policy class with a finite VC dimension $\Pi_\infty = \Pi_k$. In this more straightforward scenario, both procedures aim to find the optimal policy within a fixed policy class of finite VC dimension, leading to a welfare deficiency of zero. The resulting regret bounds are respectively
Observe that our variance term is smaller by a factor of $h^{-3/2}$. This difference arises because in the analysis of kallus2018policy, they only utilize the Rademacher contraction comparison lemma ledoux1991probability to obtain the bound. However, this approach only leverages the contraction property of the function $hK(\cdot/h)$ and does not fully exploit the following structure of the kernel term:
For the kernel term, this implies that although it is uniformly bounded by \( 1/h \), its second moment is also bounded by \( 1/h \) rather than \( 1/h^2 \) due to a standard change of variables argument. If we apply uniform bounds, which are sufficiently sharp in discrete-treatment policy learning, the variance term becomes \( \frac{1}{h}\sqrt{\frac{\text{VC}(\Pi_k)}{n}} \). Thus, using second-moment-type bounds to estimate \( \mathbb{E}[\hat{R}_{h,k}] \) yields significantly sharper results than relying on uniform bounds. Unlike our approach, kallus2018policy did not leverage this “small second-moment property” of the kernel term, leading to a much larger bound on the regret.
\paragraph{Theoretical challenges} In connection with the kernel properties in ((ref)), establishing that the penalized welfare closely approximates the true welfare—analogous to Lemma 3.2 in mbakop2021model—requires a sharper concentration result. Specifically, we rely on Talagrand’s inequality talagrand1996new to control the tail probability, rather than the bounded difference inequality mcdiarmid1989method. The use of Talagrand’s inequality is essential here, as it incorporates the second moment of the empirical process and avoids the overly conservative bounds that arise from the bounded difference inequality, which relies solely on a uniform bound for the kernel term.
However, Talagrand’s inequality introduces an additional term in the denominator, which grows with the sieve index. To ensure that this term does not dominate, we impose the technical condition \(\operatorname{VC}(\Pi_k) \leq nh^2\) in our procedure. As discussed in Appendix (ref), increasing \(\tau\) offers an alternative solution to this technical issue. Nonetheless, this remains a technical concern, as practically searching over policy spaces with complexities up to \(nh^2\) is more than adequate.
In contrast, in the discrete case, it suffices to use the bounded difference inequality, as in Lemma 3.2 of mbakop2021model, because a uniform bound on the empirical IPW welfare yields a tail bound that is as sharp as one based on the second moment.
As noted in mbakop2021model, the holdout penalty is an alternative to using Rademacher complexity for penalizing overfitting. This approach involves setting aside a portion of the sample to assess the performance of sieve policy estimators.
The holdout procedure is described below. Let $n_E=\lfloor (1-\iota) n \rfloor$ and $n_T = n - n_E$ for some fixed number $\iota\in(0,1)$.\footnote{For concreteness, one can consider $\iota = 1/2, 1/4$ as in the numerical examples of mbakop2021model.} The original sample $S_n=\{(Y_i,D_i,X_i)\}_{i=1}^n$ is split into an estimating sample $S_{n_E}\equiv\{(Y_i,D_i,X_i)\}_{i=1}^{n_E}$ and a testing sample $S_{n_T}\equiv\{(Y_i,D_i,X_i)\}_{i=n_E+1}^n$. The estimating sample is used to identify the optimal policy within each \( \Pi_k \), while the testing sample evaluates the performance of the estimated policy. Let \( \hat{W}^E_{h}(\pi) \) represent the empirical welfare on the estimating sample and \( \hat{\pi}^E_{h,k} \equiv \arg\max_{\pi \in \Pi_k} \hat{W}^E_{h}(\pi) \) be the empirical welfare maximizer. Let \( \hat{W}^T_{h} \) denote the empirical welfare obtained on the testing sample.
The holdout procedure constructs the penalized welfare as
Intuitively, this procedure penalizes overfitting using the difference between in-sample and out-of-sample estimated welfare. The procedure remains the same as in Section (ref).
The Rademacher penalty, which exploits the full sample, yields lower regret but is harder to analyze theoretically. In contrast, the holdout penalty is theoretically simpler because the independence of the held-out sample allows the use of Bernstein’s inequality, without requiring the more technically involved Talagrand’s inequality. Yet, the holdout method sacrifices efficiency because the held-out data are not directly used to estimate the policy function.
In the previous section, we considered the case where the propensity density $f$ is known. This section studies the case where the propensity density is unknown. Using double debiasing techniques, we can achieve a welfare regret bound similar to the one derived in the previous section.
Define the double-debiased welfare function as
This expression represents the IPW welfare studied in Section (ref) with an added adjustment term. The adjustment term has zero mean, i.e., \( \mathbb{E}[\Gamma_h] = W_h \), but introduces additional variability. To manage this added variance, we impose the following assumption.
Assumption (ref) places a mild requirement on the dose-response function $m$. By restricting $m$ to be of bounded variation in the first argument, we can bound the complexity of the function class
by using the VC dimension of $\Pi_k$, thus controlling the additional variance introduced by the adjustment term in ((ref)). This technique is standard in nonparametric estimation. For example, GINE2002rates uses this condition on the kernel function to ensure that the function class corresponding to the kernel density estimator is of finite VC dimension.
An alternative approach to formulating the double-debiased moment function involves modifying $m(\pi(X),X)$ instead of the IPW expression:\footnote{kallus2018policy briefly mentioned, without providing formal results, the double-debiased approach using this formulation instead of our ((ref)).}
In this case, the adjustment term introduces additional estimation bias beyond that of the IPW estimand, which is difficult to control. For this reason, we do not pursue this version of the double-debiased formula.
Estimating the welfare based on the double-debiased moment function in ((ref)) requires first-stage estimation of $f$ and $m$. For notation simplicity, we will use $g \equiv 1/f$ to denote the inverse propensity. Suppose we have consistent (under the sup-norm) estimators $\hat{m}$ and $\hat{g}$ for $m$ and $g$, respectively. They are assumed to satisfy the following conditions.
Assumption (ref)(i) concerns the mean-squared convergence rate of \( \hat{g} \) and \( \hat{m} \) in the \( L_\infty \) space.\footnote{Alternatively, we could impose assumptions of convergence rate on $L_2$ norm for $X$, but infinity norm for $T$ is still needed. Previous literature assumes $L_2$ convergence rate for discrete treatment, while we consider continuous treatment in contrast, and hence, we need the rate to hold uniformly for all treatment levels.} Unlike in athey2021policy, these rate constraints depend on the smoothness \( r \), as the remainder term in the welfare regret is of order \( n^{-r/(2r+1)} \) rather than \( n^{-1/2} \). Since \( r \) is unknown, one could either use the estimate \( \hat{r} \) to assess this condition or adopt the conservative choice of having \( \rho_g, \rho_m \geq 1/4 \).\footnote{It is difficult to directly compare our rate requirement on the nuisance estimators with that of athey2021policy because of the additional continuous argument \( T \) in the nuisance functions.} The other conditions in Assumption (ref) require that the estimators share the same properties as their target.
Given sufficient regularity, we can construct an estimator of $m$ that satisfies the rate condition in Assumption (ref) by employing, for example, sieve-based methods chen2007large, local polynomial methods calonico2018effect, or modern machine-learning techniques such as random forests, lasso, ridge, deep neural nets, boosted trees, and ensembles of these methods chernozhukov2018double. We can also use recent advances in linear and nonlinear partitioning-based methods CattaneoFarrellFeng2020large,CattaneoChandakKlusowski2024convergence,CattaneoFengShigida2024uniform, which encompass certain decision-tree and recursive-partitioning approaches. To estimate the conditional density $f$, one may apply the techniques developed by cattaneo2024boundary or ColangeloLee2025. Appendix (ref) provides guidance on constructing the nonparametric estimators $\hat g$ and $\hat m$ based on cattaneo2024boundary and CattaneoFengShigida2024uniform and outlines sufficient conditions under which these estimators satisfy Assumption (ref).
For welfare estimation, we implement the following cross-fitting procedure. Divide the data equally into $L$ folds, using the size of each fold $n/L$. For $\ell = 1, \cdots, L$, let $I_\ell$ denote the set of observation indices in the $\ell$th fold and $I^c_\ell = \bigcup_{\ell' \ne \ell} I_{\ell'}$ the set of observation indices not in the $\ell$th fold. With a slight abuse of notation, denote $S_\ell$ as the set of observations with indices $i \in I_\ell$. For observation $(T_i,X_i)$ in $S_\ell$, we use the observations with indices in $I^c_\ell$ to construct the nonparametric estimators $\hat{m}_\ell(T_i,X_i)$ and $\hat{g}_\ell(T_i,X_i)$. The subscript $\ell$ signifies that the estimators are constructed using data in $I_\ell^c$. The double-debiased empirical welfare is constructed as
where the superscript DD indicates double debias. The sieve empirical welfare maximizer with double-debiased welfare is obtained as
The penalized welfare is set to be
where the Rademacher complexity is now computed using the double-debiased moment with cross-fitting
The rest of the procedure remains the same as in Section (ref), and we denote the resulting policy estimator as $\hat{\pi}^{\operatorname{DD}}$.
Comparing Theorem (ref) with Theorem (ref), the variance bound is higher in two ways. First, the constant \( C_v' > C_v \) accounts for the extra variation introduced by the double-debiased adjustment term in ((ref)). Second, there is a factor \( \sqrt{L} \) due to the Rademacher complexity being constructed through cross-fitting; this factor also appears in zhou2023offline (e.g., their Lemma 3).
The bias term is slightly larger because, although the double-debiasing procedure mitigates bias, a small residual remains owing to kernel smoothing of the continuous treatment. In standard double-debiasing calculations for discrete treatments, the bias includes the expectation of the following term
whose expectation is zero. With a continuous treatment, this term becomes
whose expectation is generally non-zero and depends on the bandwidth $h$ and the convergence rate of $\hat{g}$. See Lemma (ref) for details.
Despite these two differences, the result for the observational setting shows that double-debiased policy estimators can achieve a comparable welfare regret to the IPW setting with a known propensity, provided the first-stage estimates converge at a sufficiently fast rate.
In this section, we discuss the implementation of sieve approximation of the global policy class, including traditional sieves and neural networks. In each case, we discuss the VC dimension and the welfare approximation rate for the specific sieve class.
Specifically, let $\alpha_k \downarrow 0$ denote the rate at which the sieve sequence $\Pi_\infty$ approximates $\Pi_k$, which is defined by
If $m$ is Lipschitz in $t$ with a Lipschitz constant uniform in $x$ (e.g., its first derivative in $t$ is bounded), then the welfare deficiency $W^*(\Pi_\infty) - W^*(\Pi_k)$ is $O(\alpha_k)$. When $W^*(\Pi_\infty)$ is achieved at an interior maximizer satisfying the first-order condition, the welfare deficiency rate can be sharpened to $O(\alpha_k^2)$. We present upper bounds on welfare deficiency only for illustration; the oracle inequalities, as shown in the theorems, ensure that the data-driven estimator automatically balances variance and kernel bias against the actual welfare deficiency without needing to know its exact rate.\footnote{Similar data-driven sieve-selection ideas have been explored in other adaptive estimation problems (e.g., breunig2024adaptive and chen2025adaptive for nonparametric instrumental variables), though the goals there differ from policy learning.}
As in the empirical approach of mbakop2021model, a standard way for restricting \( \Pi_\infty \) is to apply shape constraints driven by economic principles, such as fairness. We introduce formulations of \( \Pi_\infty \) and \( \Pi_k \) that extend the structure used in the empirical study of mbakop2021model to our continuous-treatment context. In Section (ref), this policy class formulation promotes fairness within job training programs.
Denote the $p$-th component of $X$ by $X_p$ and the support of $X_p$ by $\mathcal{X}_p\subset \mathbb{R}$. Let $h_p:\mathcal{X}_p\rightarrow \mathbb{R}$ be a monotone function and Lipschitz continuous with constant $L_p$, for $1\leq p \leq d_X$. We consider a particular type of policy, which transforms each coordinate of $X$ and then takes the sum as the treatment level: $\pi(X) = \sum_{p=1}^{d_X}h_p(X_p)$. Let $\Pi_{\infty}$ be the set of all such policies:
To construct the sieve policy class $\Pi_k$, we define $\varphi_{k,k^\prime}$ as the triangular kernel shifted by $k^\prime$ and scaled by $k$, for $k'=1,\dots,k$:
Given a vector of coefficients $\theta \equiv (\theta_1,\cdots,\theta_{d_X})$, where $\theta_p\equiv (\theta_{p,0},\cdots,\theta_{p,k})^\top$ for $p=1,\dots,d_X$, the policy $\pi_{k,\theta}$ is defined as \[\pi_{k,\theta}(x)\equiv \sum_{p=1}^{d_X}\sum_{k^\prime=0}^k \theta_{p,k^\prime}\varphi_{k,k^\prime}(x_p).\] To ensure that $\sum_{k^\prime=0}^k \theta_{p,k^\prime}\varphi_{k,k^\prime}$ is monotone, additional constraints on the coefficients are needed, specifically $\theta_{p,k^\prime+1}\geq\theta_{p,k^\prime}, \forall k', p$. Denote $E$ as the following $k\times(k+1)$ matrix:
Then, the sieve policy class $\Pi_k$ is described
Since $\Pi_k$ lies within a finite-dimensional vector space, its VC dimension is bounded by its dimension $(k+1)d_X$. The sieve approximation rate in condition ((ref)) can be taken to be $\alpha_k = O(k^{-1})$, as established in Lemma (ref) in Appendix (ref).
Neural networks have emerged as a viable option for approximating continuous functions. We briefly explain how to construct deep neural nets to implement policy learning.
A function $\psi_{\text{NN}}$ on $\mathcal{X}$ implemented by a ReLU neural network can be represented as $x \xrightarrow[]{\mathcal{L}_0} hd_1 \xrightarrow[]{\text{ReLU}} \widetilde{hd}_1 \cdots \xrightarrow[]{\mathcal{L}_{k-1}} hd_{k} \xrightarrow[]{\text{ReLU}} \widetilde{hd}_{k} \xrightarrow[]{\mathcal{L}_k} hd_{k+1} = \psi_{\text{NN}}(x)$, or more compactly expressed as $\psi_{\text{NN}} = \mathcal{L}_k \circ \text{ReLU} \circ \mathcal{L}_{k-1} \circ \text{ReLU} \circ \cdots \circ \mathcal{L}_1 \circ \text{ReLU} \circ \mathcal{L}_0$, where $\mathcal{L}_{k'}$ is an affine transformation for each $k'=0,\cdots,k$, that is, $\mathcal{L}_{k'}(\cdot) = \Omega_{k'} \cdot + \omega_{k'}$ for some $\Omega_{k'} \in \mathbb{R}^{N_{k'+1} \times N_{k'}}$ and $\omega_{k'} \in \mathbb{R}^{N_{k'+1}}$. The ReLU activation function takes $\max\{\cdot,0\}$. In the literature, the matrix $\Omega_{k'}$ is called the weight and $\omega_{k'}$ the bias. The sieve index $k$ is the depth of the network. The integer $N_{k'}$ represents the width of the $k'$th layer, which is the number of neurons in the $k'$th layer. In particular, $N_0 = d_X$ and $N_{L+1}=1$, indicating $d_X$ inputs and a single output. A deep neural network is characterized by increasing depth while maintaining fixed width, i.e., $\Pi_k = \{\psi_{\text{NN}} \text{ with } k \text{ layers and fixed width} \}$.
Recent work has established theoretical properties for deep neural networks. Theorem 7 of bartlett2019nearly shows that the VC dimension of $\Pi_k$ admits a nearly tight upper bound of order $O(k^{2}\log k)$. The approximation rate of deep neural networks for smooth functions is provided in shen2021deep,shen2022optimal. For example, by Corollary 1.3 in shen2022optimal, for the H\"older space of continuous function of order $\gamma \in (0,1]$,\footnote{Here, fix a constant $L>0$, the H\"older space is the set of all functions $f$ satisfying $|f(x) - f(x')| \leq L \lVert x - x' \rVert_2^\gamma$.} condition ((ref)) holds with $\alpha_k \propto k^{-2\gamma/d_X}$.
In this section, we apply our proposed method to assigning individuals to job training of varying lengths, using data from the Job Training Partnership Act (JTPA) study.\footnote{The data is sourced from kitagawa2018should, with background information extracted from the expbif.dta dataset, publicly available on the W.E. Upjohn Institute for Employment Research website. Observations with missing values for the included covariates have been excluded. The code for our empirical study is available on \url{https://github.com/yuefang11/continuous_policy_learning.git}.} Individuals often enroll in job training programs for varying durations. In the dataset, $22\%$ of individuals enrolled in job training for less than one month, $34.2\%$ received training for more than one month but less than three months, and $20\%$ for more than six months. We plot the frequency of the training time (measured in weeks) and its estimated density in Figure (ref), demonstrating that the treatment variable is continuous.
For policymakers, the challenge is not only to determine who should receive job training but also to tailor the duration of training to meet individual needs effectively. Building on prior studies, such as kitagawa2018should and mbakop2021model, which focus on binary eligibility for job training, our study addresses how long participants should receive training, treating the duration as a continuous variable. This shift introduces two distinctions. First, the propensity score becomes unknown, making our double-debiased method particularly effective for this setting. Second, identification requires justification, as we are no longer in an experimental context.
We argue that the unconfoundedness assumption is reasonable, as supported by flores2012estimating, hsu2022counterfactual,ColangeloLee2025 in their analysis of training duration using the Job Corps (JC) dataset. Given the close alignment between the two programs, the identification strategy developed for the JC dataset can be effectively applied to the JTPA study dataset. Both programs operated under the JTPA framework and shared similar institutional features, including open entry and exit policies, individualized training plans, and self-paced structures bloom1993national, bloom1997benefits, doolittle1993design. This strong alignment, combined with rich pre-treatment demographic data and comparable participant characteristics, supports the validity of the unconfoundedness assumption for the JTPA dataset. Additionally, following flores2012estimating, we restrict our analysis to individuals who have completed at least one week (40 hours) of training. This criterion ensures that the sample includes participants who have engaged with the training program to a minimal degree, enabling a more meaningful assessment of the training's effects.
Following ColangeloLee2025, we define the continuous treatment $T$ as the total time spent in academic and vocational training (measured in weeks).\footnote{Participants are encouraged to search for employment while still in training, so the actual length of participation need not coincide with the nominal period assigned at entry. Hence, our policy in this setting is best interpreted as a recommended duration rather than a rigidly enforced requirement.} The outcome variable $Y$ is the applicants' earnings for $30$ months following the program, subtracting the training program cost, which is, on average, $\$5$ per hour bloom1997benefits. The policy is based on three variables: years of education ($X_1$), pre-treatment earnings ($X_2$, measured by dollars per year), and working experience ($X_3$, measured by total weeks worked in the previous year). While other covariates are utilized to estimate the nuisance functions, due to legal and ethical considerations, they are only used for de-confounding but are not included in the policy.\footnote{Those covariates include gender, race, age, location, and site of enrollment, etc.} There are 2740 observations in our sample. The average participant is 32.39 years old, with a 40.58% likelihood of being male, has 11.73 years of education, \$2952.80 of pre-treatment earnings, and 22.08 weeks of work experience in the previous year.
The policy spaces we consider are those defined in Section (ref). This monotonicity constraint reflects the assumption that individuals with lower levels of education, pre-treatment earnings, and work experience should receive at least as much training time as those with higher levels of these attributes. The rationale is that individuals with fewer resources or lower baseline levels in these areas may benefit more from extended training, helping them achieve outcomes comparable to those of their more advantaged peers.
The welfare is estimated based on the double-debiased method. The estimated order of smoothness is equal to one. The propensity score $f$ is estimated using nearest neighbor kernel density estimation, and the conditional outcome function $m$ is estimated with linear regression. More complicated methods (e.g., random forests) could be adopted at the cost of increasing optimization time. For the Rademacher penalty, 100 random draws are used to simulate the Rademacher complexity for each fold, with one optimization problem solved for each draw. For the holdout penalty, 20% of the sample is used as the testing sample. The optimization model is implemented with Gurobi 11.0 in a Python 3.11.9 environment. The factor $\gamma$ in the term $(1+\gamma)\hat{B}(h)$ is set to be 0.1. The bandwidth set $\mathcal{H}$ is the exponential sequence provided in Section (ref).
Figures (ref) and (ref) depict the policy learning results obtained using the Rademacher and holdout penalties, respectively. For each method, \(\hat{\pi}_{k,\hat{h}_k}\) is presented for \(k = 1, \dots, 9\), where \(\hat{h}_k\) denotes the bandwidth that maximizes the penalized welfare for the corresponding \(k\). The selected policies are similar, with \((k=4, h=0.1)\) chosen under the Rademacher penalty and \((k=5, h=0.1)\) under the holdout penalty. Figure (ref) illustrates slices of the learned policy.
These results offer several insights. First, the learned policy yields an average training duration of approximately 11 weeks in both cases, with extended training periods—particularly those exceeding 12 weeks—rarely assigned. This observation could be attributed to the tendency of longer training durations to reduce participants' motivation to seek employment, as the training often provides subsidies or support services that may diminish the urgency of job searching.
Second, education level emerges as the primary factor influencing the assignment of training durations, whereas earnings and work experience appear to have insignificant effects. The complementary relationship between education and training may explain this distinction. Job training enhances and builds upon the foundational skills acquired through education, suggesting that individuals with lower levels of education require longer training durations to address skill gaps effectively. In contrast, earnings and work experience are typically outcomes of an individual's existing skills and reflect their current position in the labor market rather than their potential to benefit from additional training.
From an econometric perspective, we note the following observations: First, the policies learned through the Rademacher and holdout procedures are similar, yielding consistent results. Second, for a fixed sieve index \(k\), the bandwidth has a relatively minor impact on penalized welfare. Third, across the complete set of results, we observe a tendency for the selected bandwidth to increase with \(k\), aligning with our theoretical understanding of the relationship between the two tuning parameters.
This paper investigates policy learning in settings where the treatment variable is continuous. Following the framework of mbakop2021model, we approximated the infinite-dimensional policy space using a sequence of finite-dimensional subspaces. However, in contrast to the binary treatment setting, the welfare function for continuous treatments required nonparametric estimation, even with a known propensity score. This nonparametric estimation introduced an additional tuning parameter—the bandwidth—which interacted with the dimension of the approximating space. We developed a data-automated penalization procedure for simultaneously selecting the tuning parameters. The penalty term was designed to control overfitting and account for the bias induced by the nonparametric estimation. We established oracle inequalities to demonstrate that the policy learned through this procedure effectively balanced the trade-offs between variance, the bias arising from the policy space approximation, and the kernel estimation bias. We proposed a double-debiased procedure for cases with an unknown propensity score that achieved a similar oracle inequality. We applied the proposed methodology to the JTPA dataset to determine the optimal training durations for participants based on their profiles. Compared to binary decisions regarding program participation, our approach offered more personalized recommendations for training durations. These findings suggest that decision-makers in continuous treatment settings should strongly consider using data-driven policy learning procedures.
The authors contributed equally to this work and are listed in alphabetical order. We appreciate the Co-Editor Michael Jansson, an associate editor, and three referees whose suggestions have significantly improved this article. The National Natural Science Foundation of China (No. T2261160400, No. 72133005) supported Chunrong Ai's work, the National Natural Science Foundation of China (No. 72503208) supported Yue Fang's work, and the National Natural Science Foundation of China (No. 72403008, No. 72495123) supported Haitian Xie's work.