The exact contents of citations.db main_text.text for this paper — one flattened LaTeX string, title through conclusion, appendix excluded, unmodified except for removing email addresses. This is what our citation measures are computed over.
74,438 characters
Experimental Design for Policy Choice
\title{Experimental Design for Policy Choice}
\author{Samuel D. Higbee}
\institute{
University of North Carolina at Chapel Hill
}
\date{September 9, 2026}
\abstract{
We show how to optimally design experiments when the
resulting data will be used to choose a welfare-maximizing policy subject to constraints.
A decision maker seeks to maximize Bayes expected welfare by choosing a policy
whose effects depend on an unknown finite-dimensional parameter.
The decision maker has access to a first wave of experimental data with a fixed design
but may choose the design of a second wave that will be collected before choosing the policy.
The resulting experimental design--policy choice problem
is a very high-dimensional dynamic program that is generally intractable
in finite samples.
We propose a tractable approximation based on the limit experiment
and show it is asymptotically optimal using a new asymptotic representation
theorem for adaptive experiments with continuous treatments.
We apply the method to a conditional cash transfer experiment and demonstrate the potential for large
gains from tailoring the experiment to the policy choice.
}
\acknowledgements{
I thank St\'ephane Bonhomme, Max Tabord-Meehan, Guillaume Pouliot,
Arun Chandrasekhar, Alex Torgovitsky, Lars Peter Hansen,
Azeem Shaikh, Giovanni Compiani, Ali Horta\c{c}su,
and numerous seminar participants for their helpful comments.
Author's email: \texttt{[email removed]}
}
\maketitle
\newpage
\section{Introduction}
\label{section:introduction}
Economists often use experiments to evaluate, compare, and choose policies.
Given data from an experiment and an econometric model,
decision makers can evaluate counterfactual policies and choose the one
that maximizes some measure of welfare, subject to budgetary or other constraints.
Experimental design can play an important role in this process,
since some experimental designs will be more informative about the optimal policy than others.
Such experiments should be preferred when the ultimate goal of the experiment is to
inform policy choice.
Despite the fact that choosing policies is a ubiquitous objective for decision
makers,
very little is known about how to optimally design experiments for this purpose.
Instead, much of the guidance on experimental design assumes the objective of
maximizing the precision of parameter or treatment effect estimates
(see \textcite{atheyEconometricsRandomizedExperiments2017},
as well as Section \ref{section:literature} below for a review).
However, not all parameters are equally important for choosing policies.
For example, estimating the average effect of a welfare benefits program may not
be helpful for choosing how benefits should vary based on a recipient's income or
number of dependents.
Instead, the experiment should focus on learning about the effects of
counterfactual policies on welfare,
prioritizing policies that are more likely to be welfare-enhancing.
This paper provides a method for designing experiments specifically for the
purpose of informing policy choice.
Given an econometric model, a welfare objective, and a set of feasible policies,
our proposed method uses a first wave of experimental data to characterize which
model parameters are most valuable to learn about for the purpose of policy
choice, and then designs a second wave to optimally learn about these
policy-relevant parameters.
The resulting experiment delivers data which is maximally informative
for choosing the best policy.
We consider a decision maker who must choose a policy to maximize
expected welfare subject to budgetary or other constraints.
The effects of counterfactual policies are described by a structural
econometric model governed by an unknown finite-dimensional parameter.
The decision maker has a first wave of experimental data
and can design and run a second wave before choosing the
policy.
The resulting joint experimental design--policy choice problem is
a very high-dimensional dynamic decision problem.
Since the policy depends on the realized data,
a naive backward-induction approach requires the decision maker to specify which policy
they would choose for every possible dataset that the experiment could
generate.
As a practical matter, solving this problem directly is not possible.
We propose a tractable method for finding the optimal experiment using
two approximations to the decision problem.
First, we replace the high-dimensional data with a Gaussian estimate of the model parameters from each wave.
Selecting a design then reduces to choosing the covariance matrix of the second-wave estimate.
Second, we approximate the nonlinear welfare objective with a quadratic function.
Under these two approximations, the policy choice problem
depends only on a Gaussian estimate of a particular linear function of the model parameters---
namely, the marginal effect of making small adjustments to the policy.
These approximations yield a low-dimensional dynamic program that is simple to solve in practice.
We show that our proposed method is asymptotically optimal
in the sense that a decision maker who
(i) uses our method to select the experimental design, and
(ii) uses the resulting data to choose a policy,
will have the highest possible limiting welfare
across all feasible experimental designs.
We show this by validating the two approximations underlying our method
in an asymptotic regime in which both waves of the experiment grow large.
First, we establish a new asymptotic representation theorem for
adaptive experiments with continuous treatments, which justifies the Gaussian information structure.
Second, we show that the constrained policy choice problem is locally approximated by a quadratic program.
We use these results to derive an asymptotic lower bound on the welfare achievable by any experimental design--policy choice procedure
and show that our proposed method achieves this bound.
We demonstrate the method in an application to the Progresa conditional cash
transfer experiment.
We consider a decision maker who must choose a school attendance subsidy policy to maximize the graduation rate of students.
The decision maker has a first wave of data from the Progresa experiment,
which she uses to design a second wave to learn about the optimal subsidy policy.
We estimate that our experimental design
reduces the regret of the downstream policy choice by as much as $70\%$ compared to the original experimental design.
This translates to graduation rates that are about $1.5$ percentage points higher than
what could be attained by policies estimated with the original Progresa experiment.
Our approach applies to a wide range of models and objectives.
While we focus on parametric models, we do not impose further restrictions on the model beyond standard regularity conditions.
Parametric structural models are widely used in economics to evaluate counterfactual policies
not directly observed in the data
(see \textcite{toddAssessingImpactSchool2006} and \textcite{attanasioEducationChoicesMexico2012} for examples in Progresa).
We require the welfare functions to be strongly concave in the policy in a neighborhood of the optimal policy,
but do not assume a specific functional form for the welfare function.
This allows the decision maker to target objectives that may not be simple reduced-form functions of observed outcomes,
such as agents' subjective utility or long-run profits.
The methods of this paper therefore enable researchers to take advantage
of the ``best of both worlds'' described by
\textcite{toddBestBothWorlds2023} to leverage experimental variation and
economic structure to efficiently learn about optimal policies.
\subsection{Related Literature}
\label{section:literature}
This paper intersects with two complementary literatures in econometrics: experimental
design and policy choice.
The questions of how to design experiments, and of how to choose policies
based on the results of experiments, are often treated separately in econometrics.
The bandit literature, which does treat these questions jointly,
does not generally achieve the asymptotic optimality results that we achieve here
in our constrained, nonlinear, continuous-treatment setting.
The optimal design of experiments constitutes a vast literature
in many fields.
Classic approaches include
\textcite{silveyOptimalDesignIntroduction2013},
\textcite{pukelsheimOptimalDesignExperiments2006},
and
{
\DeclareDelimFormat[textcite]{finalnamedelim}
{\addspace\bibstring{and}\linebreak\space}
\textcite{chalonerBayesianExperimentalDesign1995}
}.
More recent works in econometrics have focused on experimental
design with the aim of efficiently estimating treatment effects
with binary or discrete treatments in semiparametric settings.
This literature is reviewed in
\textcite{atheyEconometricsRandomizedExperiments2017}.
Examples include
\textcite{hahnAdaptiveExperimentalDesign2011},
\textcite{baiOptimalityMatchedPairDesigns2022},
\textcite{vivianoExperimentalDesignNetwork2020},
\textcite{tabord-meehanStratificationTreesAdaptive2023},
and \textcite{cytrynbaumFineStratificationSurvey2021}.
The methods proposed by these papers, like the one proposed here,
rely on large first-wave samples to inform the design of a second wave.
When this assumption fails, such methods can have poor finite-sample
properties (\cite{caiPerformanceNeymanAllocation2024}).
\textcite{armstrongAsymptoticEfficiencyBounds2022a} provides
asymptotic efficiency bounds for ATE estimation across experimental designs.
Our setting is parametric, but we view the contribution of
an asymptotically optimal experiment for policy choice in parametric models as both a novel and
necessary first step towards designing experiments for policy choice in more flexible settings.
Another large literature in econometrics focuses on policy choice,
given experimental or observational data
(\cite{manskiStatisticalTreatmentRules2004},
\cite{stoyeMinimaxRegretTreatment2009,stoyeMinimaxRegretTreatment2012a},
\cite{bhattacharyaInferringWelfareMaximizing2012},
\cite{kitagawaWhoShouldBe2018},
\cite{atheyPolicyLearningObservational2021},
\cite{mbakopModelSelectionTreatment2021}).
Counterfactual evaluation and policy choice is also a central
goal of structural econometric models;
we motivate our setting with the examples of
\textcite{toddAssessingImpactSchool2006} and
\textcite{attanasioEducationChoicesMexico2012},
which study the Progresa program that we use in our application.
This literature takes the data-generating process
to be outside the control of the decision maker.
We complement this approach by showing how to choose the most favorable data-generating process
for the policy choice problem from a given class of experiments.
To do this, we extend the asymptotic analysis of treatment choice
rules as in \textcite{hiranoAsymptoticsStatisticalTreatment2009},
\textcite{hiranoAsymptoticAnalysisStatistical2020a},
and \textcite{xuAsymptoticAnalysisPoint2026}
to general nonlinear constrained decision problems
and optimize the value of this problem across experimental designs.
Adaptive experiments are commonly used to optimize a measure of welfare,
with the multi-armed bandit literature providing many algorithms
for both in-sample and out-of-sample welfare maximization
(\textcite{lattimoreBanditAlgorithms2020} and
\textcite{russoTutorialThompsonSampling2018} provide surveys,
while
\textcite{vivianoPolicyDesignExperiments2020},
\textcite{katoAdaptiveExperimentalDesign2025},
and \textcite{cesa-bianchiAdaptiveMaximizationSocial2025}
provide applications to policy choice in economics).
Much of the bandit literature proposes heuristic algorithms and proves
rate-optimality results for expected regret.
We take a different approach, related to
\textcite{hiranoAsymptoticRepresentationsSequential2025} and
\textcite{adusumilliRiskOptimalPolicies2025,adusumilliHowSampleWhen2025,adusumilliContinuousTimeAsymptotic2026a},
in which we characterize the limiting behavior of adaptive experiments and derive designs and policies which are
optimal in the limit experiment,
rather than rate-optimal in finite samples.
Our asymptotic representation theorem differs from prior work by allowing for continuous treatments.
Our decision problem is also distinct,
focusing on strongly concave welfare objectives and nonlinear constraints,
as opposed to a linear objective such as treatment choice with finitely many arms.
\subsection{Outline}
In Section \ref{section:decision}, we describe the general framework
and the decision problem faced by the decision maker,
and find that an exact solution is infeasible.
In Section \ref{section:method}, we describe a tractable solution method
in which the difficult decision problem is replaced by a simpler
Gaussian experiment with quadratic loss.
In Section \ref{section:limit}, we show the proposed method is asymptotically
optimal, using a limit-of-experiments framework.
In Section \ref{section:progresa}, we apply the method to the Progresa conditional cash transfer experiment.
\section{Environment and decision problem}
\label{section:decision}
The decision maker must choose two things: how to design the second
wave of the experiment, and what policy to choose after observing the results of
the experiment.
\subsection{Welfare and expected regret}
The decision maker has a welfare function
\begin{align*}
W(\pi, \theta)
\end{align*}
where $\pi\in\R^k$ is a policy which the decision maker can choose and
$\theta\in\R^\ell$ is a finite-dimensional unknown parameter.
The welfare function is not assumed to have a specific structure
beyond smoothness and local concavity assumptions specified in Section~\ref{section:limit}.
The mapping from policy and parameter to welfare is context-specific,
depending both on the model being used to predict the effects of counterfactual policies
and the preferences of the decision maker.
The set of feasible policies is described by nonlinear constraints
\begin{align*}
\Pi = \{\pi \in \R^k : g(\pi) \leq 0 \}
,
\end{align*}
which may include budget constraints, fairness constraints,
or other exogenous constraints on the policy,
such as a nonnegativity constraint on a subsidy or price.
Since the true value of $\theta$ is not known,
the decision maker evaluates policies based on their expected regret.
Regret is the welfare loss from choosing a policy $\pi$
and is given by
\begin{align*}
R(\pi, \theta) = \max_{\tilde \pi \in \Pi} W(\tilde \pi, \theta) - W(\pi,\theta)
\end{align*}
where the first term is the maximum welfare achieveable if $\theta$ were known.
Expected regret is calculated by averaging regret across possible
values of $\theta$ using a prior density $q(\theta)$,
resulting in the Bayes regret objective
\begin{align*}
\E\left[R(\pi, \theta)\right] =
\int R(\pi, \theta) q(\theta) d\theta
.
\end{align*}
Minimizing Bayes regret is equivalent to maximizing Bayes welfare
because the recentering term in regret does not affect the choice of policy.
However, recentering the welfare function in this way is useful for
asymptotic analysis.
While we use Bayes regret as the objective,
the method proposed in this paper does not depend on the prior,
and hence a user of our method does not need to be able to articulate their prior.
This is because the decision maker uses the first wave to inform future decisions,
and as the size of the first wave grows the influence of the prior vanishes
under the conditions given in Section~\ref{section:limit}.
\subsection{The data-generating process}
The decision maker has access to an experimental sample that arrives in two waves.
Of the $n$ total units, $n_1$ arrive in the first wave $\mathcal{I}_{1,n} = \{1, \dots, n_1\}$,
and $n_2 = n - n_1$ arrive in the second wave $\mathcal{I}_{2,n} = \{n_1 + 1, \dots, n\}$.
Each unit is associated with a vector of covariates $x_i$.
Our analysis is conditional on covariates, and we treat them as fixed
throughout.
During the experiment, each unit is assigned a treatment $z_i$,
after which an outcome $y_i$ is observed.
In the first wave, the treatment $z_i$ is assigned according to a fixed experimental
design which is not chosen by the decision maker.
It is drawn from the density
\begin{align*}
p_{z \mid x}(z_i \mid x_i; \delta_1),
\quad
i \in \mathcal{I}_{1,n}
\end{align*}
where $\delta_1$ is a fixed parameter describing the design of the first wave.
The dominating measure for the density $p_{z \mid x}$ may be Lebesgue measure or another measure,
so that treatment may be discrete or continuous, and potentially multidimensional.
After treatment is assigned in the first wave, the outcome $y_i$ is observed for these units.
Outcomes are generated according to the density
\begin{align*}
p_{y \mid z, x}(y_i \mid z_i, x_i; \theta),
\quad
i \in \mathcal{I}_{1,n}
\end{align*}
where $\theta \in \Theta$ is an unknown finite-dimensional parameter
and $\Theta$ is an open subset of $\R^\ell$.
The outcome may not directly measure welfare,
but observations of $y_i$ are informative about $\theta$,
which determines the welfare effects of policies.
While we refer to the observations $i \in \mathcal{I}_{1,n}$ as the first wave
of the experiment, this data can take many forms in practice.
It may arise from an experiment conducted by the decision maker or by another entity.
It may be an observational dataset if
$\theta$ is identified from the observational data.
It can even be replaced by an asymptotically informative prior as in
\textcite{adusumilliRiskOptimalPolicies2025}.
The first wave represents existing data available to the decision maker
that may be informative for policy decisions,
but does not reveal the optimal policy with perfect certainty.
In the second wave,
the decision maker may choose the design $\delta_2$
governing the conditional distribution of treatment
for the units in the second wave.
It is restricted to lie in a compact set
\begin{align*}
\delta_2 \in \Delta
\end{align*}
which describes constraints on the experimental design,
such as budget constraints, ethical constraints, etc.
Once $\delta_2$ has been decided, the treatment for units in the second wave is drawn
according to
\begin{align*}
p_{z \mid x}(z_i \mid x_i; \delta_2),
\quad
i \in \mathcal{I}_{2,n}
.
\end{align*}
Thus, $\delta_2$ can specify how the probability of treatment depends on covariates or
how the value of a continuous treatment (such as a price) depends on covariates.
Once the decision maker assigns treatment, outcomes for the second wave are observed.
These outcomes are drawn according to the same conditional density as in the first wave,
\begin{align*}
p_{y \mid z, x}(y_i \mid z_i, x_i; \theta),
\quad
i \in \mathcal{I}_{2,n},
\end{align*}
so both experimental waves can be combined to learn about $\theta$.
\subsection{Policy choice and the dynamic decision problem}
After observing the outcomes of both waves of the experiment, the decision maker chooses a policy $\pi$
to minimize posterior expected regret.
Letting the first-wave and second-wave random data be denoted by
\begin{align*}
D_{1,n} = \{y_i, z_i\}_{i \in \mathcal{I}_{1,n}},
\quad
\quad
D_{2,n} = \{y_i, z_i\}_{i \in \mathcal{I}_{2,n}},
\end{align*}
the value of an experimental dataset is given by
\begin{subequations}\label{eq:Vn}
\begin{equation}
V_n(D_{1,n}, D_{2,n}) =
\min_\pi
\quad
\E\left[ R(\pi, \theta) \mid D_{1,n}, D_{2,n} \right]
\quad
\text{s.t.}
\quad
g(\pi) \leq 0.
\end{equation}
By backwards induction, the optimal design of the second wave is the solution to
\begin{equation}
\min_{\delta_2}
\quad
\E_{\delta_2}\left[
V_n(D_{1,n}, D_{2,n}) \mid D_{1,n}
\right]
\quad
\text{s.t.}
\quad
\delta_2 \in \Delta
\end{equation}
where $\E_{\delta_2}$ denotes the expectation with respect to the law of motion for the second wave of the experiment
when the design $\delta_2$ is chosen,
\begin{equation}
y_i, z_i \sim
p_{y \mid z, x}(y_i \mid z_i, x_i; \theta)
p_{z \mid x}(z_i \mid x_i; \delta_2),
\quad
i \in \mathcal{I}_{2,n}
\end{equation}
in which treatment $z_i$ is first assigned according to $\delta_2$
and then the outcome is realized according to the unknown parameter $\theta$.
\end{subequations}
The dynamic program (\ref{eq:Vn}) characterizes the optimal design for the second wave,
but it is infeasible to solve in practice.
One could imagine trying to solve it
by standard dynamic programming methods,
but since $\pi$ is a function of all experimental data,
the state space of $V_n$ is the set of possible
datasets that the second wave of the experiment could generate.
This is extremely high-dimensional even for moderate-sized experiments
and low-dimensional variables\footnote{
Alternatively, one could use the posterior on $\theta$ as the state,
but if the likelihood and prior are not conjugate
the state space will be the
set of probability distributions on $\R^{\ell}$.
}.
This problem is a general feature of dynamic experiments,
and features prominently in multi-armed bandit literature.
More broadly, it is a feature of statistical decision problems
where a decision maker must specify a decision rule which may depend
on realizations of the data
(\cite{manskiEconometricsDecisionMaking2021}, \cite{hiranoWaldsStatisticalDecision2026}).
\section{Approximate solution method}
\label{section:method}
Since the finite-sample problem is intractable, we propose an approximation to
this problem that is low-dimensional and tractable.
This approximation has two components:
first, replacing the nonlinear regret with a quadratic function,
and second, replacing the data-generating process with a Gaussian observation.
This section presents the method as it is implemented in practice.
Formal justification of the approximations and the asymptotic optimality of the
method are given in Section \ref{section:limit}.
\subsection{Quadratic approximation of expected regret}
We replace the nonlinear expected regret loss function with
a quadratic approximation.
Under this approximation, the optimal policy depends on $\theta$
only through a particular linear function,
which is the marginal welfare effect of changing the policy.
This reduces the state of the dynamic program to the posterior mean of this marginal effect.
Our quadratic approximation uses a second-order Taylor expansion
of welfare around a reference point.
Suppose that the true parameter $\theta$ is in a neighborhood of
some $\theta_0$ (to be formalized in Section \ref{section:limit}).
Let the optimal policy under $\theta_0$ be
\begin{align}
\label{eq:nlp-reference}
\pi_0 = \arg \min_{\pi} \quad R(\pi, \theta_0)
\quad
\text{s.t.} \quad
g(\pi) \leq 0.
\end{align}
The centering term in regret makes $R(\pi, \theta)$ potentially nondifferentiable in $\theta$.
Our method instead uses derivatives of the welfare function $W(\pi, \theta)$, which is smooth in $\theta$,
and ignores the policy-irrelevant centering term in regret.
Define
\begin{align*}
D = - \nabla_\pi W(\pi_0, \theta_0),
\quad\quad
B = - \nabla^2_{\pi\theta} W(\pi_0, \theta_0),
\quad\quad
H = - \nabla^2_{\pi\pi} W(\pi_0, \theta_0)
.
\end{align*}
Since the choice of policy is subject to constraints,
it will not generally be true that $D=0$,
so this first-order term must be included in the approximation.
The quadratic function we minimize is then given by
\begin{align*}
\tilde R(\pi, \beta) &=
(\pi - \pi_0)' D
+ (\pi - \pi_0)' (\beta - \beta_0)
+ \frac{1}{2} (\pi - \pi_0)' H (\pi - \pi_0)
\end{align*}
where $\beta = B \theta$ and $\beta_0 = B \theta_0$.
The approximation $\tilde R$ depends on $\theta$ only through the linear functional $\beta$.
The marginal effect of changing the policy in a neighborhood of $(\pi_0, \theta_0)$ is
\begin{align*}
\nabla_\pi R(\pi_0, \theta) &\approx D + (\beta - \beta_0)
\end{align*}
and so the parameter $\beta$ gives an approximation to the marginal effect of the policy
in this neighborhood.
When choosing a policy, $\theta$ is only relevant through the effects of counterfactual policies.
For policies close to $\pi_0$,
the marginal effect of changing the policy characterizes these counterfactuals.
Thus, $\beta$ serves as a sufficient statistic for policy choice in this neighborhood.
Further tractability comes from the linearity of $\tilde R$ in $\beta$.
Denoting the posterior mean of $\beta$ after both waves of the experiment by $\hat\beta_{2,n}$,
we have
\begin{align*}
\E\left[ \tilde R(\pi, \beta) \mid D_{1,n}, D_{2,n} \right]
&=
\tilde R(\pi, \hat\beta_{2,n})
\end{align*}
and so when choosing a policy to minimize the expectation of $\tilde R(\pi, \beta)$,
the decision maker only needs to consider the posterior mean of $\beta$.
However, the law of motion for $\hat\beta_{2,n}$ is still complicated,
motivating our next approximation.
\subsection{Gaussian approximation of estimates}
Our second approximation is to replace the full data with a Gaussian observation
from each wave, with precision determined by $\delta_2$.
This allows the decision maker to reduce the optimization over treatment distributions
to optimization over posterior variance-covariance matrices for $\hat\beta_{2,n}$.
We approximate the prior distribution and experimental data with a flat prior
and Gaussian observations, respectively.
After the first wave, suppose the decision maker only observes a Gaussian estimate $\hat\theta_{1,n}$ of $\theta$,
which satisfies
\begin{align*}
\hat \theta_{1,n} \overset{a}{\sim} N\left(\theta, \frac{1}{n_1} J_{1,n}^{-1}\right)
\end{align*}
where
\begin{align*}
J_{1,n} = \frac{1}{n_1}\sum_{i \in \mathcal{I}_{1,n}} \E\left[
\nabla_\theta \log p_{y \mid z, x}(y_i \mid z_i, x_i; \theta_0)
\nabla_\theta \log p_{y \mid z, x}(y_i \mid z_i, x_i; \theta_0)'
\right]
\end{align*}
is the Fisher information matrix at $\theta_0$ for the first wave of the experiment.
After choosing $\delta_2$ and running the second wave of the experiment,
suppose the decision maker observes a second Gaussian estimate $\hat \theta_{2,n}$
from the second wave, which satisfies
\begin{align*}
\hat \theta_{2,n} \overset{a}{\sim} N\left(\theta, \frac{1}{n_2} J_{2,n}(\delta_2)^{-1} \right)
\end{align*}
where
\begin{align*}
J_{2,n}(\delta_2) = \frac{1}{n_2} \sum_{i \in \mathcal{I}_{2,n}} \E_{\delta_2} \left[
\nabla_\theta \log p_{y \mid z, x}(y_i \mid z_i, x_i; \theta_0)
\nabla_\theta \log p_{y \mid z, x}(y_i \mid z_i, x_i; \theta_0)'
\right]
\end{align*}
is the Fisher information matrix when the design $\delta_2$ is chosen,
and $\E_{\delta_2}$ indicates that $z_i$ is assigned according to the design $\delta_2$.
By varying the experimental design $\delta_2$,
the decision maker varies the variance-covariance matrix of the resulting
estimate $\hat\theta_{2,n}$.
The information matrix $J_{2,n}(\delta_2)$ affects the welfare of the
policy chosen at the end of the experiment by determining the information
available to the decision maker when choosing the policy.
This is summarized by the posterior distribution of $\beta$
conditional on the Gaussian estimates $\hat{\theta}_{1,n}$ and $\hat{\theta}_{2,n}$.
Under the flat-prior approximation,
this posterior is also Gaussian and is given by
\begin{align*}
\hat\beta_{2,n} =
B
\left(n_1 J_{1,n} + n_2 J_{2,n}(\delta_2)\right)^{-1}
\left(n_1 J_{1,n} \hat \theta_{1,n} + n_2 J_{2,n}(\delta_2) \hat \theta_{2,n} \right)
.
\end{align*}
Equivalently, since we are using a flat prior approximation,
$\hat\beta_{2,n}$ is the pooled maximum likelihood estimate of $\beta$ from the two waves of the experiment.
Therefore, our method does not depend on the choice of prior $q(\theta)$.
\footnote{This may be desirable if the decision maker has difficulty articulating their prior,
but it may be undesirable if the decision maker has a strong prior belief that is not reflected by a flat or weakly informative prior.
The method here can easily extend to informative Gaussian priors.}
Designs which are more informative about particular parameters
will lead to posteriors which are more informative about those parameters.
Whether a particular $\delta_2$ is optimal in terms of its information
depends on the objective and constraints of the decision problem.
In particular, our quadratic approximation implies that $\beta$
is a sufficient statistic for the policy choice.
However, due to constraints on the policy choice and the local curvature of welfare,
the decision maker may not care about all components of $\beta$ equally.
If some margins described by $\beta$ are infeasible,
or if the first wave suggests that some margins would almost certainly decrease welfare,
then the decision maker may not want to spend their experimental budget to learn about those margins.
Finally, we note that our method still works if $\delta_2$ is chosen such that
$J_{2,n}(\delta_2)$ is singular,
as long as the resulting posterior information $n_1 J_{1,n} + n_2 J_{2,n}(\delta_2)$ is non-singular.
In this case, the MLE $\hat\theta_{2,n}$ may not be well-defined,
but the pooled MLE from both waves, and the posterior mean $\hat\beta_{2,n}$, will still be well-defined.
In fact, choosing a singular $J_{2,n}(\delta_2)$ may be optimal in some cases,
since the decision maker may prefer a design which is informative about $\beta$
but which provides no information about other components of $\theta$.
\subsection{Feasible analog and value function}
The approximation described above is constructed around a reference point
$(\pi_0, \theta_0)$ which is unknown to the decision maker.
To make this method feasible, we replace the unknown reference point with estimates
from the first wave.
We use these estimates to construct empirical analogues of the Gaussian and quadratic approximations above.
After collecting the first wave of data, the decision maker constructs
the maximum likelihood estimator $\hat\theta_{1,n}$,
which is approximately normally distributed.
The decision maker then constructs an estimate of the optimal policy from the first wave, given by
\begin{align*}
\hat \pi_{1,n} = \arg\min_\pi \quad R(\pi, \hat\theta_{1,n}) \quad \text{s.t.} \quad g(\pi) \leq 0
.
\end{align*}
The first-wave estimates $(\hat\pi_{1,n}, \hat\theta_{1,n})$ are then used as plug-in estimates for the unknown reference point $(\pi_0, \theta_0)$.
The quadratic approximation to regret is estimated by
\begin{align*}
\hat R_n(\pi, \beta) &=
(\pi - \hat \pi_{1,n})' \hat D_n
+ (\pi - \hat \pi_{1,n})' (\beta - \hat\beta_{1,n})
+ \frac{1}{2} (\pi - \hat\pi_{1,n})' \hat H_n (\pi - \hat\pi_{1,n})
\end{align*}
where $\hat D_n$, $\hat \beta_{1,n} = \hat B_n \hat\theta_{1,n}$, and $\hat H_n$
are all given by plug-in estimates.
Likewise, the Gaussian approximation is estimated by replacing the unknown
Fisher information matrices $J_{1,n}$ and $J_{2,n}(\delta_2)$ with plug-in estimates.
These are given by
\begin{align*}
\hat J_{1,n} &= \frac{1}{n_1} \sum_{i \in \mathcal{I}_{1,n}} \nabla_\theta \log p_{y \mid z, x}(y_i \mid z_i, x_i; \hat\theta_{1,n})
\nabla_\theta \log p_{y \mid z, x}(y_i \mid z_i, x_i; \hat\theta_{1,n})'
\\
\hat J_{2,n}(\delta_2) &= \frac{1}{n_2} \sum_{i \in \mathcal{I}_{2,n}} \E_{\delta_2} \left[
\nabla_\theta \log p_{y \mid z, x}(y_i \mid z_i, x_i; \hat\theta_{1,n})
\nabla_\theta \log p_{y \mid z, x}(y_i \mid z_i, x_i; \hat\theta_{1,n})'
\right]
\end{align*}
where the expectation in the second line is taken over the distribution of $z_i$ when the design is $\delta_2$
and the distribution of $y_i$ when the parameter is $\hat\theta_{1,n}$,
both of which are known to the decision maker after the first wave of the experiment.
Thus, the feasible approximation to the value function
which the decision maker can solve in practice is
\begin{subequations}\label{eq:hatVn}
\begin{equation}
\hat V_n(\hat \beta_{2,n}) = \min_\pi
\quad \hat R_n(\pi, \hat \beta_{2,n})
\quad \text{s.t.} \quad g(\pi) \leq 0
\end{equation}
after which the decision maker chooses the design of the second wave to minimize the expected value function:
\begin{equation}
\min_{\delta_2} \quad
\E\left[ \hat V_n(\hat\beta_{2,n}) \mid \hat \theta_{1,n} \right]
\quad \text{s.t.} \quad \delta_2 \in \Delta
\end{equation}
where the expectation integrates over the second-wave distribution of $\hat \beta_{2,n}$ using
a plug-in estimate of the Gaussian Bayesian updating formula for the law of motion:
\begin{equation}
\hat\beta_{2,n} \sim N\left(
\hat\beta_{1,n},
\hat B_n
\left( \frac{1}{n_1} \hat J_{1,n}^{-1} - (n_1 \hat J_{1,n} + n_2 \hat J_{2,n}(\delta_2))^{-1} \right)
\hat B_n'
\right)
\mid \hat \theta_{1,n}
.
\end{equation}
\end{subequations}
We denote the solution to this problem by $\hat\delta_{2,n}$.
This value function is much lower-dimensional than the original value function $V_n$.
The state is simply $\hat\beta_{2,n}$, rather than the full experimental data $D_{1,n}$ and $D_{2,n}$.
In contrast to $V_n$, the state of $\hat V_n$ does not grow with the sample size.
Moreover, $\hat\beta_{2,n}$ is of the same dimension as the policy $\pi$, which is often low-dimensional for practical reasons.
Solving the value function $\hat V_n$ in practice is straightforward.
While algorithmic details are deferred to Appendix \ref{appendix:progresa},
the main steps are (i) evaluate $\hat V_n(\beta)$ for many values of $\beta$,
(ii) interpolate to construct a smooth surrogate for $\hat V_n$,
(iii) compute $\E_\delta[\hat V_n(\hat\beta_{2,n}) \mid \hat\theta_{1,n}]$
as a function of $\delta$ by Monte Carlo integration,
and (iv) optimize over $\delta_2$ using a nonlinear optimization routine.
Solving $\hat{V}_n$ is especially tractable when the constraints $g(\pi) \leq 0$ are linear,
as is the case in the Progresa application in Section \ref{section:progresa}.
Then $\hat{V}_n(\hat \beta)$ is a quadratic program for any value of $\hat \beta$,
making construction of $\hat V_n$ very fast.
\section{Asymptotic optimality}
\label{section:limit}
The solution method presented in the previous section is asymptotically optimal.
We establish this by showing that the
approximations made in the previous section have negligible error as
the sizes of the two waves of the experiment grow large.
These results imply that the design $\hat \delta_{2,n}$ that comes from
solving (\ref{eq:hatVn}) leads to the lowest possible regret asymptotically.
\subsection{Local asymptotic framework}
\label{subsection:localasymptotics}
The Gaussian and quadratic approximations are justified in an
asymptotic decision environment called the limit experiment.
The limit experiment describes the possible asymptotic behaviors of any
design and policy in the finite-sample environment.
Solving for the best design in the limit experiment
motivates finite-sample procedures that achieve asymptotic optimality.
The limit experiment is constructed in a local asymptotic framework.
Local asymptotics are commonly used to study
optimality of estimators and tests,
among other econometric and statistical problems
(\cite{le1972limits}, \cite{vandervaartAsymptoticStatistics2000}).
In local asymptotics, we analyze the performance of our method under
alternative parameter values that are difficult to distinguish from each other
even in large samples.
In particular, we model
\begin{align*}
\theta = \theta_0 + h/\sqrt{n}
&&
\pi = \pi_0 + c/\sqrt{n}
\end{align*}
where $(h,c)$ is a local parameter and policy and
$(\theta_0, \pi_0)$ is a reference parameter and policy.
The $1/\sqrt{n}$ scaling means that the difference between $\theta$ and
$\theta_0$ is of the same order of magnitude
as the standard error of typical estimators of $\theta$.
Regret can be written in terms of local alternatives as
\begin{align*}
R_n(c, h) = n R(\pi_0 + c/\sqrt{n}, \theta_0 + h/\sqrt{n})
\end{align*}
where the $n$ scaling ensures a nontrivial limit as $n \to \infty$.
Local asymptotics are essential for our framework to capture
finite-sample uncertainty about which constraints are binding.
The set of $\theta$ for which a particular constraint holds with weak complementary slackness
generally has Lebesgue measure zero.
This means that for almost every $\theta$,
the decision problem behaves asymptotically as if the set of binding constraints is known.
In this case, the decision problem in the limit experiment reduces to
minimizing a quadratic function over a linear subspace,
and hence is equivalent to point estimation.
In contrast, our approach of localizing around a reference parameter $\theta_0$
where weak complementary slackness holds preserves nontrivial constraints in the limit
and ensures that even asymptotically, the decision maker does not know
which constraints bind at the optimum.
This uncertainty is essential for capturing the distinctive aspects
of experimental design for policy choice.
This localization has an intuitive interpretation in the cash-transfer example of Section \ref{section:progresa}.
The decision maker is most likely to make a mistake when it is difficult to detect
differences in the marginal effects of the cash transfer across groups.
When this is the case, noisy experimental data may lead the decision maker to allocate extra money to boys even
when girls would benefit more.
In this scenario, $\theta_0$ may be a point where the marginal effects of the cash transfer are positive and equal across groups,
and $\pi_0$ is a vertex of the feasible set.
Then at $\pi_0$, the cash is allocated to only one of the four groups,
but local perturbations $h \neq 0$ may cause the optimal policy $c$ to move away from the vertex and allocate some cash to the other groups.
Our asymptotic framework does not require that $\theta_0$ be specified, but it is intended
to capture this statistically ``hard'' instance of the decision problem.
This is analogous to localizing around an average treatment effect of zero in a binary
treatment choice problem (\cite{hiranoAsymptoticsStatisticalTreatment2009}).
\subsection{Limit experiment}
The bound we establish on the limiting regret of any sequence of designs
is given by the value of an optimization problem in the limit experiment.
This limiting decision problem is a quadratic program with linear constraints.
The constraints in the limit experiment are given by linear approximations
to the constraints in the finite-sample problem.
Define $\lambda_0$ to be the unique Lagrange multiplier (guaranteed to exist under the assumptions given below)
satisfying
\begin{align*}
D + \lambda_0' \nabla g(\pi_0) = 0
&&
\lambda_{0j} \geq 0
&&
\lambda_{0j} g_j(\pi_0) = 0
\end{align*}
for $j=1, \dots, J$.
Let the set of indices $j$ for which
strict/weak complementary slackness holds at $\pi_0$ be
\begin{align*}
\mathcal{J}_s = \{
j : g_j(\pi_0) = 0, \; \lambda_{0j} > 0
\}
&&
\mathcal{J}_w = \{
j : g_j(\pi_0) = \lambda_{0j} = 0
\},
\end{align*}
respectively.
The feasible set in the limit experiment is the convex cone
\begin{align*}
\mathcal{G} = \{
c : \;
\nabla g_j(\pi_0)' c = 0, j \in \mathcal{J}_s, \;
\nabla g_j(\pi_0)' c \leq 0, j \in \mathcal{J}_w, \;
\}
.
\end{align*}
The quadratic objective arises from the quadratic approximation to regret,
but also incorporates the curvature of the constraints through the Hessian of the Lagrangian.
Define the Hessian of the Lagrangian at $(\pi_0, \lambda_0)$ as
\begin{align*}
\bar H = H + \lambda_0' \nabla^2 g(\pi_0)
\end{align*}
and define the quadratic objective
\begin{align*}
r(c, b) = c' b + \frac{1}{2} c' \bar H c
\end{align*}
where $b = B h$ is the local alternative for $\beta$.
This quadratic objective, like $\tilde R$ in Section \ref{section:method}, omits the policy-irrelevant centering term in regret,
which simplifies the dynamic program in the limit experiment.
The asymptotic lower bound, however, is given by the regret in the limit experiment,
which is
\begin{align*}
R_\infty(c, b) = r(c, b) - \min_{c \in \mathcal{G}} r(c, b).
\end{align*}
We now describe the information structure in the limit experiment.
We begin with the prior.
Fix $K > 0$ and define the localized finite-sample prior as
\begin{align*}
q_{n,K}(h) \propto q(\theta_0 + h/\sqrt{n}) \1\{ \|h\| \leq K \}
\end{align*}
normalized to integrate to one.
The corresponding density in the limit experiment is
\begin{align*}
q_{\infty,K}(h) \propto \1\{ \|h\| \leq K \}
\end{align*}
which is uniform on the ball of radius $K$.
This localization of the prior serves two purposes.
First, it ensures that the limit of expected regret is finite,
and hence the asymptotic lower bound is well-defined.
Second, it asymptotically concentrates around $\theta_0$,
a point of possible weak complementary slackness, and hence
reflects the finite-sample property that some constraints may be on the margin of binding or nonbinding.
In the results that follow, we will show our method, which is motivated by a flat-prior approximation,
is asymptotically optimal as $K$ grows large, and hence as our local prior becomes weakly informative.
The observed data in the limit experiment comes from a two-wave adaptive experiment with Gaussian signals.
Let $\rho$ denote the asymptotic fraction of the sample in the second wave of the experiment.
In the first wave of the limit experiment, the decision maker observes a random variable
\begin{align*}
A_1 \sim N((1-\rho) J_1 h, (1-\rho) J_1)
.
\end{align*}
Note that when $J_1$ is nonsingular,
observing $A_1$ is equivalent to observing a Gaussian signal
$\hat h_1 = (1-\rho)^{-1} J_1^{-1} A_1 \sim N(h, (1-\rho)^{-1}J_1^{-1})$,
which is the limiting distribution of the first-wave maximum likelihood estimator.
After observing $A_1$, the decision maker chooses a design $\delta_2$ and observes
\begin{align*}
A_2 \sim N(\rho J_2(\delta_2) h, \rho J_2(\delta_2))
.
\end{align*}
As with the first wave, when $J_2(\delta_2)$ is nonsingular, observing $A_2$ is equivalent to observing a Gaussian signal
$\hat h_2 = \rho^{-1} J_2(\delta_2)^{-1} A_2 \sim N(h, \rho^{-1} J_2(\delta_2)^{-1})$.
After observing $A_1$ and $A_2$, the decision maker chooses a policy $c$
to minimize the posterior expected regret.
Let $\hat b_{2,K}$ be the posterior mean of $b$ conditional on $A_1$ and $A_2$ under the prior $q_{\infty,K}$.
The value function in the limit experiment (omitting the centering term) is
\begin{align*}
V(\hat b_{2,K}) =
\min_c \quad r(c, \hat b_{2,K})
\quad \text{s.t.} \quad
c \in \mathcal{G}
\end{align*}
and the optimal design in the limit experiment is given by
\begin{align*}
\min_{\delta_2} \quad \E_{\delta_2,K}\big[
V(\hat b_{2,K})
\mid A_1
\big]
\quad \text{s.t.} \quad
\delta_2 \in \Delta
.
\end{align*}
We denote the optimal design and policy in the limit experiment by $(\delta_{2,K}^*,c^*_K)$.
We evaluate the asymptotic performance of a sequence of designs and policies
by their Bayes regret under the truncated prior $q_{n,K}$.
We consider experimental designs which are described by
a fixed design rule $\phi_n$ so that $\delta_{2,n} = \phi_n(D_{1,n}, U)$ where $U$ is auxiliary randomness independent of the data.
The ex-ante Bayes regret under $q_{n,K}$ is given by
\begin{align*}
\mathcal{R}_{n,K}(\delta_{2,n}, c_n)
= \int \E_h^{\phi_n}\left[ R_n(c_n, h) \right] q_{n,K}(h) dh
\end{align*}
where $\E_h^{\phi_n}$ denotes the expectation over the data when the local parameter is $h$ and the design rule is $\phi_n$.
When $c_n = c_{n,K}^*$ is the Bayes optimal policy for $\delta_{2,n}$, we abbreviate
to $\mathcal{R}_{n,K}^*(\delta_{2,n})$, and when $\delta_{2,n} = \delta_{2,n,K}^*$ is the Bayes optimal design,
we abbreviate to $\mathcal{R}_{n,K}^{**}$.
The fixed-$K$ regret of a design $\delta_2$ and policy $c$ in the limit experiment is
\begin{align*}
\mathcal{R}_{\infty,K}(\delta_2, c)
= \int \E_h^\phi\left[ R_\infty(c, B h) \right] q_{\infty,K}(h) dh
\end{align*}
where $\phi$ is the limit experiment design rule governing $\delta_2 = \phi(A_1, U)$.
The quantities $\mathcal{R}_{\infty,K}^*(\delta_2)$ and $\mathcal{R}_{\infty,K}^{**}$ are defined analogously.
\subsection{Asymptotic representation theorem}
Our first result is an asymptotic representation theorem which justifies the Gaussian approximation
to the information environment.
It states that the statistical behavior of any sequence of designs and policies in the finite-sample experiment
is asymptotically approximated by a design and policy in the Gaussian limit experiment.
To apply asymptotic approximations to both waves of the experiment,
we require that both waves of the experiment grow large.
\begin{assumption}
\label{assumption:largewaves}
Wave sizes satisfy $\lim_{n\to\infty} n_2/n = \rho$ and $\lim_{n\to\infty} n_1/n = 1 - \rho$ for some $\rho \in (0,1)$.
\end{assumption}
This assumption implies that the first wave cannot be an uninformative pilot that is vanishingly small
compared to the second wave--- it must be large enough to yield a $\sqrt{n}$-consistent estimate of $\theta$.
A key (and standard) condition for local asymptotic analysis is that the model is smooth in $\theta$ in the sense of differentiability in quadratic mean (DQM),
allowing for local approximations to the likelihood.
\begin{assumption}
\label{assumption:dqm}
There exists a function $\psi(y \mid z, x)$,
called the score of $p_{y \mid z, x}$ at $\theta_0$,
such that for all $h$,
\begin{align*}
\sup_{z, x} \int &\left[
\sqrt{p_{y \mid z, x}(y \mid z, x; \theta_0 + h)}
- \sqrt{p_{y \mid z, x}(y \mid z, x; \theta_0)}
\right. \\ & \left.
- \frac{1}{2} h' \psi(y \mid z, x) \sqrt{p_{y \mid z, x}(y \mid z, x; \theta_0)}
\right]^2 \mu(dy)
= o(\|h\|^2)
\end{align*}
as $h \to 0$, for some measure $\mu$ which dominates $p_{y \mid z, x}$ for all $(z, x)$.
\end{assumption}
At a high level, this means the square root of the density is differentiable in $\theta$ at $\theta_0$.
This condition allows us to use the score to linearly approximate the likelihood at $\theta$ near $\theta_0$.
Typically, the score is the gradient of the log-likelihood of the model
(see Lemma 7.6 of \textcite{vandervaartAsymptoticStatistics2000} for sufficient conditions for this formulation).
To show that the Gaussian approximation holds uniformly across the set of designs,
we need to link the conditional distributions
$p_{z \mid x}(\cdot \mid x; \delta)$ and $p_{y \mid z, x}(\cdot \mid z, x; \theta)$
across different designs through a potential outcomes model.
\begin{assumption}
\label{assumption:potentialoutcomes}
There exist functions $z(x, \nu; \delta)$ and $y(z, x, \epsilon; \theta)$ such that
\begin{align*}
z_i = z(x_i, \nu_i; \delta), \quad\quad y_i = y(z_i, x_i, \epsilon_i; \theta)
\end{align*}
where $\nu_i$ and $\epsilon_i$ are independent of each other and independent across $i$.
\end{assumption}
The structural error $\epsilon_i$ captures the unobserved determinants of the outcome $y_i$,
while the randomization device $\nu_i$ is generated by the decision maker to determine the treatment assignment $z_i$.
With this potential outcomes model,
we can define the score of an observation $i$ as a function of the design $\delta$ as
\begin{align*}
\psi_i(\delta)
=
\psi\bigg(
y\Big(
z(x_i, \nu_i; \delta),
x_i, \epsilon_i; \theta
\Big),
z(x_i, \nu_i; \delta),
x_i
\bigg).
\end{align*}
We also define the score process and log-likelihood ratio in the second wave as
\begin{align*}
A_{2,n}(\cdot) = \frac{1}{\sqrt{n}} \sum_{i \in \mathcal{I}_{2,n}} \psi_i(\cdot)
, \quad\quad
\Lambda_{2,n}^h(\cdot) = \sum_{i \in \mathcal{I}_{2,n}} \log \frac{p_{y \mid z, x}(y_i(\cdot) \mid z_i(\cdot), x_i; \theta_0 + h/\sqrt{n})}{p_{y \mid z, x}(y_i(\cdot) \mid z_i(\cdot), x_i; \theta_0)}
.
\end{align*}
The sample score $A_{1,n}$ and log-likelihood ratio $\Lambda_{1,n}^h$ in the first wave are defined analogously, but with the fixed design $\delta_1$.
The specific smoothness required for uniform approximations of the decision environment
is stochastic equicontinuity of the log-likelihood ratio and the score
as processes in $\Delta$.
\begin{assumption}
\label{assumption:stocheq}
$\Delta$ is a compact subset of Euclidean space.
$A_{2,n}(\cdot)$ is stochastically equicontinuous as a process in $\Delta$ under $\theta_0$.
For any $h$, $\Lambda_{2,n}^h(\cdot)$ is stochastically equicontinuous as a process in $\Delta$ under $\theta_0$.
\end{assumption}
Stochastic equicontinuity is a high-level condition that ensures that the
score and log-likelihood have relatively smooth sample paths for large enough $n$.
Many lower-level sufficient conditions are available in the literature
(e.g. \textcite{andrewsChapter37Empirical1994}, \textcite{vandervaartWeakConvergenceEmpirical2013}).
While Assumption \ref{assumption:stocheq} is imposed only under $\theta_0$,
we show in Appendix \ref{appendix:art} that the laws of the adaptive experiments
under local alternatives are contiguous, so that approximations that
vanish in probability under $\theta_0$ also vanish in probability under local alternatives.
Assumption \ref{assumption:stocheq} is a joint restriction on both the treatment assignment mechanism
and the outcome model. Both the treatment assignment and the model must be smooth enough that small changes in $\delta$
lead to small changes in the data and associated likelihood.
In Appendix \ref{appendix:stocheq} we give lower-level sufficient conditions which are natural for our setting,
and verify that they are satisfied in the Progresa example in Section \ref{section:progresa}.
Finally, we need the sample information matrices to have well-defined limits.
\begin{assumption}
\label{assumption:information}
There exists a matrix $J_1$
and a covariance kernel $J_2(\cdot, \cdot)$ such that for any $d_1, d_2 \in \Delta$,
\begin{align*}
\frac{1}{n_1} \sum_{i \in \mathcal{I}_{1,n}} \E_{\theta_0} \big[ \psi_i(\delta_1) \psi_i(\delta_1)' \big] &\to J_1
\\
\frac{1}{n_2} \sum_{i \in \mathcal{I}_{2,n}} \E_{\theta_0} \big[ \psi_i(d_1) \psi_i(d_2)' \big] &\to J_2(d_1, d_2).
\end{align*}
$J_1$ is positive definite.
When $d_1 = d_2 = d$, we write $J_2(d) := J_2(d, d)$.
The pooled information matrix is defined as $J(\delta_2) = (1-\rho) J_1 + \rho J_2(\delta_2)$.
\end{assumption}
Our asymptotic representation theorem states that any convergent
sequence of designs and policies in the finite-sample experiment
converges to a design and policy in the limit experiment.
\begin{theorem}[Asymptotic representation theorem]
\label{theorem:art}
Suppose \Crefrange{assumption:largewaves}{assumption:information} hold.
Let $(\delta_{2n}, c_n)$ be a sequence of adaptive designs and policies in the finite-sample experiment which converges in distribution
under the sequence $\theta_0 + h/\sqrt{n}$ for every $h$.
Then there exist statistics $(\delta_2, c)$ and random variables
$A_1, A_2, U$
such that
\begin{enumerate}
\item $A_1 \sim N( (1-\rho) J_1 h, (1-\rho) J_1)$ independently of $U$,
and $\delta_2 = \delta_2(A_1, U)$
\item $A_2 | A_1, U \sim N(\rho J_2(\delta_2) h, \rho J_2(\delta_2))$
and $c = c(A_1, A_2, U)$
\item $(\delta_{2n}, c_n) \overset{h}{\leadsto} (\delta_2, c)$.
\end{enumerate}
\end{theorem}
Asymptotic representation theorems are often used to provide simple approximations to statistical decision problems.
For the classic, non-adaptive case, see \textcite{le1972limits} and \textcite{vandervaartAsymptoticStatistics2000}.
\textcite{hiranoAsymptoticsStatisticalTreatment2009} use such a theorem to derive asymptotically optimal
rules for binary treatment choice.
\textcite{hiranoAsymptoticRepresentationsSequential2025} establish a similar result for batched multi-armed bandits.
Our result, while stated for a two-wave adaptive experiment, can be extended to multiple waves.
The main distinction between our result and \textcite{hiranoAsymptoticRepresentationsSequential2025} is that we allow for continuous treatments.
Our result builds on this work by extending their argument to these more complex design spaces
in which $\delta$ can determine not just the fraction of units assigned to a treatment,
but also the values of continuous treatments assigned to each unit, which can depend on covariates.
This generality comes at the cost of the additional Assumption \ref{assumption:stocheq},
which is satisfied in the multi-armed bandit setting of \textcite{hiranoAsymptoticRepresentationsSequential2025}
because the score and log-likelihood are partial-sum processes.
Finally, \textcite{hiranoAsymptoticRepresentationsSequential2025} do not solve for optimal policies,
which we consider next.
Theorem \ref{theorem:art} applies to sequences of policies $c_n$ that converge at the parametric rate.
While this can fail in nonparametric empirical welfare maximization problems
(\cite{kitagawaWhoShouldBe2018}, \cite{atheyPolicyLearningObservational2021}),
it is much less restrictive in the parametric models considered here.
In particular, Appendix \ref{appendix:lowerbound} verifies that the Bayes-optimal policy
is uniformly tight.
We note that the usefulness of Theorem \ref{theorem:art} extends beyond the
specific decision problem considered here.
It does not depend on the objective function or constraints of the decision problem,
nor does it impose a Bayesian framework--- it is stated for fixed local alternatives.
Thus, Theorem \ref{theorem:art} can be used to analyze efficiency of estimators
and tests in adaptive experiments with continuous treatments.
We leave this to future work.
\subsection{Asymptotic optimality}
Our main result states that the procedure described in Section \ref{section:method}
is asymptotically optimal.
We first establish a lower bound on the regret of any sequence of designs and policies in the finite-sample experiment.
Then we show that the procedure described in Section \ref{section:method} achieves this lower bound, and is therefore asymptotically optimal.
We first require some regularity on the decision maker's objective and constraints.
\begin{assumption}[Nonlinear program regularity]
\label{assumption:nlp}
\begin{enumerate}
\item $W$ is continuous on $\Pi \times \Theta$ and twice continuously differentiable
in a neighborhood of $(\pi_0, \theta_0)$.
Each $g_j$ is twice continuously differentiable in a neighborhood of $\pi_0$.
\item $\pi_0$ is the unique global maximizer of $W(\pi, \theta_0)$ subject to $g(\pi) \leq 0$.
There exists some $\alpha < W(\pi_0, \theta_0)$, a neighborhood $N$ of $\theta_0$, and a compact set $C_\pi$ such that
$\{ \pi \in \Pi : W(\pi, \theta) \geq \alpha\} \neq \emptyset$ and
$\{\pi \in \Pi : W(\pi, \theta) \geq \alpha\} \subseteq C_\pi$ for all $\theta \in N$.
$\Pi$ is closed and convex.
\item The gradients $\{ \nabla g_j(\pi_0) : g_j(\pi_0) = 0 \}$ are linearly independent.
\item There is a $\kappa > 0$ such that
$c' \bar H c \geq \kappa \| c \|^2$
for all $c \in \text{span}(\mathcal{G})$
and $c' H c \geq \kappa \| c \|^2$
for all $c \in \text{span}(\Pi - \Pi)$.
\end{enumerate}
\end{assumption}
These conditions are standard in the literature on sensitivity analysis for nonlinear programs
under weak complementary slackness (\cite{shapiroSecondOrderSensitivity1985}, \cite{bonnansPerturbationAnalysisOptimization2013}).
They imply that the value of the reference decision problem (\ref{eq:nlp-reference}) is locally approximated by a quadratic program.
Item 3 ensures that the Lagrange multiplier $\lambda_0$ is unique, but not necessarily nonzero.
This means that some constraints in the limit experiment will be inequality constraints,
and the decision maker may not know whether these constraints are binding or nonbinding at the true optimal policy.
Item 4 restricts our attention to strongly concave welfare functions;
this rules out binary treatment choice with a linear welfare function.
We also require smoothness of the prior.
\begin{assumption}
\label{assumption:prior}
$q$ is positive and continuous in a neighborhood of $\theta_0$.
\end{assumption}
Under these assumptions, the optimal design and policy $(\delta_{2,K}^*, c_K^*)$ in the limit experiment provides
a lower bound on the regret of any sequence of designs and policies in the finite-sample experiment.
\begin{theorem}[Lower bound]
\label{theorem:lowerbound}
Suppose \Crefrange{assumption:largewaves}{assumption:prior} hold.
Let $(\delta_{2,n}, c_n)$ be any sequence of designs and policies in the finite-sample experiment.
Then for any fixed $K < \infty$,
\begin{align*}
\liminf_{n\to\infty} \mathcal{R}_{n,K}(\delta_{2,n}, c_n) \geq \mathcal{R}_{\infty,K}^{**}
.
\end{align*}
\end{theorem}
Theorem \ref{theorem:lowerbound} combines Theorem \ref{theorem:art},
which justifies the Gaussian information environment,
with a second-order expansion of the oracle problem (\ref{eq:nlp-reference}).
The lower bound is more than just a Taylor approximation to welfare;
it is a joint approximation of the objective and constraints of the decision problem.
This approximation is related to existing results on sensitivity analysis of nonlinear programs under weak complementary slackness
(\cite{shapiroSecondOrderSensitivity1985}, \cite{bonnansPerturbationAnalysisOptimization2013}).
In particular, $V(b)$ provides a second-order directional derivative of the oracle problem (\ref{eq:nlp-reference})
in the direction $b$, up to policy-invariant centering terms.
Theorem \ref{theorem:lowerbound} then shows that the pointwise directional derivative
provides an asymptotic lower bound
in the dynamic decision problem in the limit experiment.
We now show that our proposed experimental design $\hat\delta_{2,n}$
is asymptotically optimal in the sense that it attains the lower bound in Theorem \ref{theorem:lowerbound}.
To establish asymptotic optimality,
we construct a feasible sequence of terminal policies which attain the lower bound in Theorem \ref{theorem:lowerbound}.
Let $\hat A_{1,n}$ and $\hat A_{2,n}(\delta_2)$ be the feasible analogues of $A_{1,n}$ and $A_{2,n}(\delta_2)$,
where the score is evaluated at $\hat \theta_{1,n}$ rather than $\theta_0$.
Let $\hat J_n(\delta_2) = \frac{n_1}{n} \hat J_{1,n} + \frac{n_2}{n} \hat J_{2,n}(\delta_2)$ be the feasible analogue of $J(\delta_2)$.
Define the one-step estimator\footnote{
The maximum likelihood estimator could also be used after strengthening the classical assumptions
(e.g. Section 3.2 of \textcite{vandervaartWeakConvergenceEmpirical2013})
to ensure uniform $\sqrt{n}$-consistency and asymptotic normality of the MLE across the set of designs.
Given stochastic equicontinuity of the score process, the one-step estimator yields a simpler proof.
}
$\hat \theta_{2,n}^\text{os} = \hat\theta_{1,n} + \frac{1}{\sqrt n} \hat J_n(\hat\delta_{2,n})^{-1} \hat A_{2,n}(\hat\delta_{2,n})$
and $\hat \beta_{2,n}^\text{os} = \hat \beta_{1,n} + \hat B_n (\hat \theta_{2,n}^\text{os} - \hat \theta_{1,n})$.
The localized version is $\hat h_n^\text{os} = \sqrt{n}(\hat\theta_{2,n}^\text{os} - \theta_0)$.
Define
\begin{align*}
\hat \pi_n^\text{os} = \arg\min_{\pi \in \Pi} \hat R_n(\pi, \hat\beta_{2,n}^\text{os}) \quad \text{s.t.} \quad g(\pi) \leq 0
\end{align*}
and let $\hat c_n^\text{os} = \sqrt{n}(\hat\pi_n^\text{os} - \pi_0)$ be the corresponding localized policy.
The one-step estimator must be regular in the following sense:
\begin{assumption}[Plug-in estimator]
\label{assumption:plugin}
\begin{enumerate}
\item The first-wave MLE is $\sqrt{n}$ consistent \linebreak and interior to the parameter space.
\item The score $\psi_\theta$ exists in a neighborhood of $\theta_0$ and satisfies
\begin{align*}
\sup_{\|h\|\leq K}
\left\| \frac{1}{\sqrt{n}} \sum_{i \in \mathcal{I}_{1,n}} \left[\psi_{i,h}(\delta_1) - \psi_i(\delta_1)\right]
+ (1-\rho) J_1 h \right\|
&= o_{P_0}(1)
\\
\sup_{d\in\Delta, \|h\| \leq K}
\left\| \frac{1}{\sqrt{n}} \sum_{i \in \mathcal{I}_{2,n}} \left[\psi_{i,h}(d) - \psi_i(d) \right]
+ \rho J_2(d) h \right\|
&= o_{P_0}(1)
.
\end{align*}
\item $\|\hat J_{1,n} - J_1\| = o_{P_0}(1)$ and $\sup_{d\in\Delta} \|\hat J_{2,n}(d) - J_2(d)\| = o_{P_0}(1)$.
\item The sequence $R_n(\hat c_n^\text{os}, h)$ is uniformly integrable under $q_{\infty,K}$ for any $K < \infty$.
\end{enumerate}
\end{assumption}
This assumption ensures that the feasible one-step estimator $\hat \beta^\text{os}_{2,n}$ behaves
like the Gaussian limit experiment estimator $\hat b_{2,K}$ for large $K$.
Interiority of $\hat \theta_{1,n}$ implies that the first-wave score $\hat A_{1,n} = 0$.
Together, items 1-3 imply that the one-step process $\hat A_{2,n}(\cdot)$ is asymptotically equivalent to the Gaussian limit experiment score process $A_2(\cdot)$,
just as Assumption \ref{assumption:stocheq} implies that the true second-wave score process $A_{2,n}(\cdot)$ is asymptotically equivalent to $A_2(\cdot)$.
The uniform score and information conditions are implied by uniform versions of standard
differentiability and dominance conditions on the likelihood.
We now show that a decision maker who uses $\hat\delta_{2,n}$ to design the second wave of the experiment
and subsequently chooses the one-step estimator $\hat \pi_n^\text{os}$ for the final policy
attains the asymptotic lower bound in Theorem \ref{theorem:lowerbound}.
\begin{theorem}[Asymptotic optimality]
\label{theorem:optimality}
Suppose that for any $K$, \Crefrange{assumption:largewaves}{assumption:plugin} hold.
Then
\begin{align*}
\lim_{K\to\infty} \limsup_{n\to\infty} \left\{ \mathcal{R}_{n,K}(\hat\delta_{2,n}, \hat c_n^\text{os}) - \mathcal{R}_{\infty,K}^{**}\right\} = 0
.
\end{align*}
\end{theorem}
This is the main result of the paper, which justifies the use of the method proposed in Section \ref{section:method}.
While Theorem \ref{theorem:lowerbound} states that no feasible design and policy in the finite-sample experiment can
achieve lower regret than the optimal design and policy in the limit experiment,
this bound is useful only insofar as the design and policy used in practice can attain it.
Theorem \ref{theorem:optimality} shows that the proposed method does attain this bound\footnote{
Since $\mathcal{R}_{n,K}^*(\hat \delta_{2,n}) \leq \mathcal{R}_{n,K}(\hat\delta_{2,n}, \hat c_n^\text{os})$,
Theorem \ref{theorem:optimality} implies that a decision maker who uses $\hat\delta_{2,n}$ to run
the experiment and then chooses the fully ex-post Bayes optimal policy $c_{n,K}^*$
will also attain the asymptotic lower bound;
however, the one-step estimator is simpler to compute and is asymptotically equivalent.
}.
Since the limit experiment is defined under a truncated prior,
the flat-prior procedure in Section \ref{section:method} is not asymptotically optimal for any fixed $K$,
since it disregards the information of the truncated prior.
However, as $K$ grows large and the truncation becomes less informative,
the optimality gap between the flat-prior and truncated-prior procedure vanishes.
Previous asymptotic optimality results in adaptive experiments apply to decisions such as point estimation,
hypothesis testing, or discrete treatment choice
(\cite{hiranoAsymptoticRepresentationsSequential2025,adusumilliRiskOptimalPolicies2025,adusumilliHowSampleWhen2025}).
To our knowledge, Theorem \ref{theorem:optimality} is the first asymptotic optimality result for adaptive experiments with continuous treatments,
nonlinear objectives, and nonlinear constraints.
A limitation of this result is that it is restricted to Bayes regret.
Theorem \ref{theorem:optimality} may be useful in studying asymptotic optimality under minimax regret, but we leave this to future work.
\section{Application to Progresa}
\label{section:progresa}
We apply our method to the Progresa cash transfer experiment in Mexico.
We consider the problem of choosing the size and targeting
of the cash transfer to maximize the secondary school completion rate
among children in the program.
When our proposed method is used to design a second wave of the Progresa
experiment, we find that the expected regret of the policy choice problem
is reduced by up to 70\% relative to a second wave which uses the original
Progresa experimental design.
\subsection{Model}
We use a dynamic school choice model to learn about the effects
of counterfactual policies that the decision maker may choose.
This approach has been taken by
\textcite{toddAssessingImpactSchool2006} and \textcite{attanasioEducationChoicesMexico2012}.
One reason this model is useful for policy choice is that
the school completion rate is a long-term outcome which is not
immediately observed in the experiment.
Instead, only current schooling enrollment decisions are observed.
The dynamic model allows the decision maker to use the observed effects
of the program to learn about the long-term effects of different policies
In describing the model, we suppress the unit index $i$.
At each age $\tau$ before age $18$, households may choose whether a child in grade $s_\tau$
will attend school, denoted $y_\tau = 1$ or $y_\tau = 0$.
If the child attends school, they receive a subsidy $z_{s_\tau}$
which depends on the grade of enrollment.
The utility a household receives from a child enrolling in school
depends on the subsidy $z$ as well as
the child's age $\tau$,
current grade level $s_\tau$,
a female sex indicator $f$,
and an indicator for attending secondary school.
The difference at age $\tau$ in the utility from enrolling in school
versus not enrolling is given by
\begin{align*}
u_{\tau}(s, f, z) =&
\gamma_0 + \gamma_1 \tau
+\gamma_2 s
+ \gamma_3 f
+ \gamma_4 \1[s \geq 7] +
\\
&
\gamma_5 (1-f) z
+ \gamma_6 (1-f) \tau z
+ \gamma_7 f z
+ \gamma_8 f \tau z
+ \epsilon_{\tau}
.
\end{align*}
Households choose a sequence of school attendance $y_\tau$ to maximize long-term
utility.
Each period, if the child was enrolled in school they may advance a grade
with probability $p_\text{pass}(\tau, s_\tau)$ depending on age and grade,
or they may fail and remain at the same grade.
Failure is assumed to be exogenous.
Finally, at age $\tau=18$ households obtain a terminal value
\begin{align*}
v_{18} = \gamma_9 \1[s_{18} \geq 10]
\end{align*}
which depends on whether the child completed secondary school.
Thus, households solve
\begin{align*}
\max_{\{y_\tau\}_{\tau=6}^{17}} &\sum_{\tau=6}^{17}
\eta^\tau \E\left[ y_\tau u_{\tau}(s_\tau, f, z_{s_\tau}) \right]
+ \eta^{18} \E\left[ v_{18} \right]
\\
s_{\tau+1} &= s_\tau + y_\tau \text{Bernoulli}(p_\text{pass}(\tau, s_\tau))
\end{align*}
where $\eta$ is a calibrated discount factor and households may observe $\epsilon_\tau$ before choosing $y_\tau$.
We assume that $\epsilon_{\tau}$ has a logistic
distribution, which
enables us to compute the value function for the households by backwards
induction.
\subsection{Decision problem}
The decision problem is to choose
(i) an experimental design and
(ii) a policy that maximizes the fraction of children graduating from
secondary school.
A policy $\pi$ is a four-dimensional vector specifying the subsidy amounts
$z_s$ for boys and girls in primary and secondary school.
The welfare function is
\begin{align*}
W(\pi, \theta) =
P_{\pi, \theta} [s_{18} \geq 10]
\end{align*}
where $P_{\pi,\theta}$ represents the probability distribution
of $s_{18}$ when $\theta$ is the true parameter value and
subsidy amounts are determined by $\pi$.
We suppose the decision maker has access to a first wave of experimental data
and wants to run a second wave to help choose the optimal policy.
The first wave consists of
a sample of $n_1$ children from the original Progresa experiment.
The design of the second wave can differ from the original Progresa experiment in two ways.
First, the decision maker can change the peso amount of the subsidy
offered to boys and girls in each grade.
Second, the decision maker can let the probability of treatment vary by gender.
Thus, the design $\delta_2$ is a six-dimensional vector describing
the amount of the subsidy offered to each group
and the probability of being treated for boys and girls.
We consider $n_1 = n_2 \in \{250, 500, 1000\}$.
Both the experimental subsidy and the chosen policy subsidy
are subject to constraints.
First, the total amount of the chosen subsidy cannot exceed the total amount
of the original Progresa subsidy, although the decision maker can
allocate the subsidy differently across gender and school level.
The decision maker may also allocate the subsidy differently across the treatment and control groups in the experiment,
but the treated-share-weighted average of the experimental subsidy cannot exceed that of the Progresa experiment.
Second, the subsidy must be nonnegative,
preventing the decision maker from taxing children in some grades
to subsidize children in other grades.
We also constrain the subsidy in each grade to be less than $1000$ pesos.
\subsection{Optimal experiment}
We now present the result of applying our proposed experimental design
to this decision problem.
Following Section \ref{section:limit}, we focus on the performance of the proposed method
in statistically challenging settings where the optimal policy and the active set of constraints
are difficult to learn.
Specifically, we evaluate the expected regret of our method when the true parameter is in
a neighborhood of a point $\theta_0$ where (i) the optimal policy $\pi_0$ is a vertex of the feasible set and
(ii) the nonnegativity constraints are weakly binding, so the decision maker is unsure
which groups should receive a positive subsidy.
We calibrate these points by estimating $\theta_0$ by maximum likelihood on a large holdout set,
subject to the constraints that the KKT conditions of the policy choice problem
hold with $\lambda_{j0} = 0$ for the nonnegativity constraints $j$.
This yields four points $\theta_0$ corresponding to the four vertices of the feasible set
that lie on the budget constraint.
For each sample size and each $\theta_0$ estimated on the holdout set,
we run a Monte Carlo simulation to compute the expected regret of the proposed method.
We sample $\theta$ from a Uniform distribution on a neighborhood of $\theta_0$ and
generate a first wave of experimental data using the original Progresa design.
This first wave is used to estimate $\hat\delta_{2,n}$ following Section \ref{section:method}.
We then generate a second wave of experimental data using the $\hat\delta_{2,n}$
and use both waves to estimate the optimal policy $\hat \pi_{2,n}$.
For comparison, we also generate a second wave of experimental data using the original Progresa design
and a second wave using an oracle optimal design which uses the true $\theta$ and Fisher information
to choose the optimal design, rather than using first-wave plugins for these quantities.
\begin{figure}
\centering
\caption{Expected regret by experimental design}
\label{figure:regret}
\includegraphics[width=0.9\textwidth]
{figures/regret_by_vertex_and_sample_size.pdf}
\par\smallskip
\begin{minipage}{1.0\textwidth}
\footnotesize
\setstretch{1.0}
\textit{Note: }
Each panel shows expected regret and 95\% Monte Carlo confidence intervals
when $\theta$ is local to one of four reference points $\theta_0$ corresponding to four vertices of the feasible set.
Each vertex is characterized by a different group of children receiving a positive subsidy.
``Pilot only'' disregards the second wave.
``Progresa'' generates the second wave using the original Progresa design, just like the first wave.
``Proposed method'' generates the second wave using the method proposed in Section \ref{section:method}.
``Oracle'' uses the true $\theta$ and Fisher information to construct the approximations.
Expected regret is computed based on the $>99\%$
of replications that were successfully solved.
\end{minipage}
\end{figure}
Figure \ref{figure:regret}
shows the headline expected regret results.
More detailed results are provided in Appendix \ref{appendix:progresa}.
Across all sample sizes and reference points, the proposed method substantially
reduces expected regret relative to the original Progresa design.
The gain is particularly large for $n_1 = n_2 \in \{500, 1000\}$,
in which regret is reduced by 50-70\%.
The gains at $n_1 = n_2 = 250$ are smaller, suggesting that sampling error in the
first wave is enough to degrade the quality of the estimated design,
but even this noisily estimated design is more valuable than doubling the sample size of the original Progresa design.
We use the same method to estimate the optimal policy in each experimental design,
so this difference is due to the value of the dataset generated by different designs.
Appendix \ref{appendix:progresa} provides additional details on how the proposed design
differs from the original Progresa design and how these differences affect the quality of the chosen policy.
The optimal design tends to focus exclusively on secondary school and give large experimental subsidies to a small treatment group.
This is despite $\theta_0$ being chosen so that the marginal effect of increasing the subsidy
is equalized across all four groups, so that allocating the subsidy to primary school children is optimal for some values of $\theta$.
This is therefore not an artifact of primary school being unimportant for the policy choice
but rather a reflection of how informative different experimental subsidies are for learning about the optimal policy,
which is jointly determined by the objective, constraints, and information of the model.
In turn, the more informative designs improve the quality of the chosen policy by increasing the
probability of allocating more subsidy to the group with the highest response.
By focusing the experiment on learning about the decision-relevant
marginal effects of policy changes, the optimal experiment is able to
more effectively inform policy choice.
\begingroup
\setlength{\emergencystretch}{1em}
\printbibliography
\endgroup