arXiv 9 Jun 2022 · Econometrics · publishedJournal of Econometrics (2024) · 2 citations (OpenAlex)
arXiv:2206.04643 · PDF · DOI · OpenAlex · Extracted main text
The Neyman Allocation is used in many papers on experimental design, which typically assume that researchers have access to large pilot studies. This may be unrealistic. To understand the properties of the Neyman Allocation with small pilots, we study its behavior in an asymptotic framework that takes pilot size to be fixed even as the size of the main wave tends to infinity. Our analysis shows that the Neyman Allocation can lead to estimates of the ATE with higher asymptotic variance than with (non-adaptive) balanced randomization. In particular, this happens when the outcome variable is relatively homoskedastic with respect to treatment status or when it exhibits high kurtosis. We provide a series of empirical examples showing that such situations can arise in practice. Our results suggest that researchers with small pilots should not use the Neyman Allocation if they believe that outcomes are homoskedastic or heavy-tailed. Finally, we examine some potential methods for improving the finite sample performance of the FNA via simulations.
appendix boundary found by appendix_command · 67% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Cytrynbaum, M (2021) Designing representative and balanced experiments by local randomization | 0.874 | 8 | 2 | 100% |
| 2 | Hahn, J., K. Hirano, and D. Karlan (2011) Adaptive experimental design using the propensity score | 0.874 | 5 | 2 | 100% |
| 3 | Cytrynbaum, M (2023) Optimal stratification of survey experiments | 0.811 | 4 | 2 | 100% |
| 4 | Ashraf, N., D. Karlan, and W. Yin (2006) Tying odysseus to the mast: Evidence from a commitment savings product in the philippines | 0.693 | 9 | 1 | 100% |
| 5 | Avvisati, F., M. Gurgand, N. Guyon, and E. Maurin (2014) Getting parents involved: A field experiment in deprived schools | 0.693 | 7 | 1 | 100% |
| 6 | Blackwell, M., N. E. Pashley, and D. Valentino (2022) Batch adaptive designs to improve efficiency in social science experiments | 0.644 | 2 | 2 | 100% |
| 7 | Bai, Y (2022) Optimality of matched-pair designs in randomized controlled trials | 0.585 | 3 | 1 | 100% |
| 8 | Tabord-Meehan, M (2021) Stratification trees for adaptive randomization in randomized controlled trials | 0.511 | 2 | 1 | 100% |
| 9 | Canay, I. A. and V. Kamat (2017, 10) (2017) Approximate Permutation Tests and Induced Order Statistics in the Regression Discontinuity Design | 0.405 | 1 | 1 | 100% |
| 10 | Azriel, D., M. Mandel, and Y. Rinott (2012) Optimal allocation to maximize the power of two-sample tests for binary response | 0.405 | 1 | 1 | 100% |
Showing the top 10 of 36 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.