Justin Whitehouse, Emma Brunskill, Vasilis Syrgkanis
arXiv 11 Oct 2026 · Statistics — Machine Learning
arXiv:2610.15789 · PDF · Extracted main text
In this paper, we develop an asymptotically normal estimator for the value of policy improvement (VPI) under adaptively collected data. This value, which is defined as the gap between the value of the best treatment policy in a class and that of a baseline policy, gauges whether or not it is worth deploying a personalized treatment policy. Building an estimator for the VPI is naturally more difficult than estimating the value of a fixed policy. This is because, in addition to contending with the correlated data and decaying propensities associated with adaptive experiments, any estimator must also handle the non-regularity, or lack of pathwise differentiability, of the underlying estimand when the optimal policy is non-unique. Our proposed estimator works by sequentially fitting a weighted empirical welfare maximizing policy on historical data and then evaluating it using inverse propensity weighting on future samples. We neither assume that the optimal policy is unique nor make a margin assumption on the underlying heterogeneous treatment effects. A key difficulty in analyzing our estimator is showing that the regret incurred through policy learning is asymptotically smaller than the standard error of the estimator. We accomplish this by showing faster regret rates for policy learning under decaying propensities, which may be of independent interest. We not only evaluate the coverage of our estimator on a mix of synthetic and semi-synthetic data, but also employ it to estimate the VPI in an adaptive experiment evaluating the efficacy of job assistance programs in Jordan due to Caria et al. Although our estimates are generally more conservative than their hierarchical Bayesian analysis, which assumes that the optimal policy is unique, we still corroborate the benefits of personalization amongst Syrian refugees.
appendix boundary found by appendix_command · 38% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Luedtke, Alex and Chambaz, Antoine (2020) Performance guarantees for policy learning | 1.000 | 7 | 4 | 100% |
| 2 | Luedtke, Alexander R and Van Der Laan, Mark J (2016) Statistical inference for the mean outcome under a possibly non-unique optimal treatment strategy | 0.961 | 18 | 6 | 89% |
| 3 | Bibaut, Aurélien and Dimakopoulou, Maria and Kallus, Nathan and Cham… (2021) Post-contextual-bandit inference | 0.941 | 18 | 8 | 83% |
| 4 | Fang, Zheng and Santos, Andres (2019) Inference on directionally differentiable functions | 0.941 | 6 | 4 | 83% |
| 5 | Zhan, Ruohan and Hadad, Vitor and Hirshberg, David A and Athey, Susan (2021) Off-policy evaluation via adaptive weighting with data from contextual bandits | 0.920 | 9 | 6 | 78% |
| 6 | Zhan, Ruohan and Ren, Zhimei and Athey, Susan and Zhou, Zhengyuan (2024) Policy learning with adaptively collected data | 0.909 | 20 | 8 | 75% |
| 7 | Bibaut, Aurélien and Kallus, Nathan and Dimakopoulou, Maria and Cham… (2021) Risk minimization from adaptively collected data: Guarantees for supervised and policy learning | 0.874 | 9 | 6 | 67% |
| 8 | Athey, Susan and Wager, Stefan (2021) Policy learning with observational data | 0.843 | 4 | 4 | 75% |
| 9 | Cárcamo, Javier and Cuevas, Antonio and Rodr\'iguez, Luis-Alberto (2020) Directional differentiability for supremum-type functionals | 0.843 | 4 | 4 | 75% |
| 10 | Girard, Samuel and Bibaut, Aurélien and Zenati, Houssam (2025) Online Policy Learning via a Self-Normalized Maximal Inequality | 0.843 | 4 | 4 | 75% |
Showing the top 10 of 80 scored citations.