arXiv 15 Sep 2023 · Statistics — Methodology · 7 citations (OpenAlex)
arXiv:2309.08808 · PDF · DOI · OpenAlex · Extracted main text
In the experimental design literature, Neyman allocation refers to the practice of allocating units into treated and control groups, potentially in unequal numbers proportional to their respective standard deviations, with the objective of minimizing the variance of the treatment effect estimator. This widely recognized approach increases statistical power in scenarios where the treated and control groups have different standard deviations, as is often the case in social experiments, clinical trials, marketing research, and online A/B testing. However, Neyman allocation cannot be implemented unless the standard deviations are known in advance. Fortunately, the multi-stage nature of the aforementioned applications allows the use of earlier stage observations to estimate the standard deviations, which further guide allocation decisions in later stages. In this paper, we introduce a competitive analysis framework to study this multi-stage experimental design problem. We propose a simple adaptive Neyman allocation algorithm, which almost matches the information-theoretic limit of conducting experiments. We provide theory for estimation and inference using data collected from our adaptive Neyman allocation algorithm. We demonstrate the effectiveness of our adaptive Neyman allocation algorithm using both online A/B testing data from a social media site and synthetic data.
appendix boundary found by none_found · 100% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Carpentier A, Munos R (2011) Finite time analysis of stratified sampling for monte carlo | 1.000 | 9 | 7 | 100% |
| 2 | Khamaru K, Zhang CH (2024) Inference with the upper confidence bound algorithm | 1.000 | 9 | 3 | 100% |
| 3 | Antos A, Grover V, Szepesvári C (2010) Active learning in heteroscedastic noise | 1.000 | 8 | 6 | 100% |
| 4 | Grover V (2009) Active learning and its application to heteroscedastic problems | 1.000 | 8 | 6 | 100% |
| 5 | Chen Y, Lu J (2025) A characterization of sample adaptivity in ucb data | 1.000 | 8 | 3 | 100% |
| 6 | Hahn J, Hirano K, Karlan D (2011) Adaptive experimental design using the propensity score | 1.000 | 7 | 3 | 100% |
| 7 | Xiong R, Athey S, Bayati M, Imbens GW (2019) Optimal experimental design for staggered rollouts | 1.000 | 6 | 3 | 100% |
| 8 | Lattimore T, Szepesvári C (2020) Bandit algorithms | 0.928 | 4 | 3 | 100% |
| 9 | Hu F, Zhang LX (2004) Asymptotic properties of doubly adaptive biased coin designs for multitreatment clinical trials | 0.874 | 6 | 2 | 100% |
| 10 | Neyman J (1934) On the two different aspects of the representative method: The method of stratified sampling and the method of purposive selection | 0.874 | 5 | 2 | 100% |
Showing the top 10 of 138 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.
| Citing paper | Intensity | Mentions | Sections | |
|---|---|---|---|---|
| 1 | Benefits and Costs of Adaptive Sampling | 0.811 | 4 | 2 |
| 2 | Double machine learning and design in batch adaptive experiments | 0.644 | 2 | 2 |
| 3 | On the Performance of the Neyman Allocation with Small Pilots | 0.405 | 1 | 1 |
| 4 | 2.5cm When and How to Pilot: Design Rules for Two-Wave Experiments | 0.405 | 1 | 1 |