arXiv 18 Jul 2026 · Econometrics
arXiv:2607.16982 · PDF · Extracted main text
Experimenters often run pilots, but how much a small pilot should shape the main-wave design has no settled answer. This paper shows how noisy pilot evidence should guide treatment assignment probabilities in two-wave experiments. Two canonical rules mark the extremes. Balanced assignment guards against worst cases but ignores evidence that one arm is noisier. Feasible Neyman allocation adapts, but with a finite pilot it can overreact to noise, producing arbitrarily large precision losses. We propose a Conditional Minimax Regret (CMR) rule that minimizes worst-case regret over a finite-sample confidence set for the treatment and control variances. CMR retains balance's worst-case protection with high probability, converges to the Neyman allocation as the pilot grows, and attains the minimax-regret rate up to constants. It extends to multi-arm and stratified designs, and simulations calibrated to four field experiments show it avoids feasible Neyman's severe small-pilot losses while capturing most of its large-pilot gains.
appendix boundary found by appendix_command · 36% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Timothy B. Armstrong (2026) Asymptotic efficiency bounds for a class of experimental designs, 2026 | 0.928 | 4 | 3 | 100% |
| 2 | Yong Cai and Ahnaf Rafi (2024) On the performance of the Neyman allocation with small pilots | 0.843 | 4 | 3 | 75% |
| 3 | Jinyong Hahn, Keisuke Hirano, and Dean Karlan (2009) Adaptive experimental design using the propensity score | 0.843 | 4 | 3 | 75% |
| 4 | Isaiah Andrews and Jiafeng Chen (2025) Certified decisions, 2025 | 0.811 | 4 | 2 | 100% |
| 5 | Andreas Maurer and Massimiliano Pontil (2009) Empirical Bernstein bounds and sample variance penalization | 0.737 | 3 | 3 | 67% |
| 6 | Leonard J Savage (1951) The theory of statistical decision | 0.737 | 3 | 2 | 100% |
| 7 | Jörg Stoye (2009) Minimax regret treatment choice with finite samples | 0.737 | 3 | 2 | 100% |
| 8 | Jerzy Neyman (1934) On the two different aspects of the representative method: The method of stratified sampling and the method of purposive selection | 0.644 | 2 | 2 | 100% |
| 9 | Charles F Manski (2021) Econometrics for decision making: Building foundations sketched by Haavelmo and Wald | 0.585 | 3 | 1 | 100% |
| 10 | Max Tabord-Meehan (2023) Stratification trees for adaptive randomisation in randomised controlled trials | 0.511 | 2 | 2 | 50% |
Showing the top 10 of 50 scored citations.