Aurelien Bibaut, Nathan Kallus, Michael Lindon
arXiv 29 Dec 2022 · Statistics — Methodology · 2 citations (OpenAlex)
arXiv:2212.14411 · PDF · DOI · OpenAlex · Extracted main text
Sequential tests and their implied confidence sequences, which are valid at arbitrary stopping times, promise flexible statistical inference and on-the-fly decision making. However, strong guarantees are limited to parametric sequential tests that under-cover in practice or concentration-bound-based sequences that over-cover and have suboptimal rejection times. In this work, we consider classic delayed-start normal-mixture sequential probability ratio tests, and we provide the first asymptotic type-I-error and expected-rejection-time guarantees under general non-parametric data generating processes, where the asymptotics are indexed by the test's burn-in time. The type-I-error results primarily leverage a martingale strong invariance principle and establish that these tests (and their implied confidence sequences) have type-I error rates asymptotically equivalent to the desired (possibly varying) $\alpha$-level. The expected-rejection-time results primarily leverage an identity inspired by It\^o's lemma and imply that, in certain asymptotic regimes, the expected rejection time is asymptotically equivalent to the minimum possible among $\alpha$-level tests. We show how to apply our results to sequential inference on parameters defined by estimating equations, such as average treatment effects. Together, our results establish these (ostensibly parametric) tests as general-purpose, non-parametric, and near-optimal. We illustrate this via numerical simulations and a real-data application to A/B testing at Netflix.
appendix boundary found by appendix_command · 67% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | H. Robbins and D. Siegmund (1974) The Expected Sample Size of Some Tests of Power One | 1.000 | 11 | 4 | 100% |
| 2 | Herbert Robbins and David Siegmund (1970) Boundary crossing probabilities for the wiener process and sample sums | 1.000 | 10 | 4 | 100% |
| 3 | A. Wald (1945) Sequential tests of statistical hypotheses | 1.000 | 7 | 3 | 100% |
| 4 | Ian Waudby-Smith, David Arbour, Ritwik Sinha, Edward H. Kennedy, and… (2021) Time-uniform central limit theory, asymptotic confidence sequences, and anytime-valid causal inference, 2021 | 1.000 | 6 | 4 | 100% |
| 5 | Steven R. Howard, Aaditya Ramdas, Jon McAuliffe, and Jasjeet Sekhon (2021) Time-uniform, nonparametric, nonasymptotic confidence sequences | 1.000 | 5 | 4 | 100% |
| 6 | Herbert Robbins (1970) Statistical methods related to the law of the iterated logarithm | 1.000 | 5 | 4 | 100% |
| 7 | A. Wald and J. Wolfowitz (1948) Optimum character of the sequential probability ratio test | 1.000 | 5 | 3 | 100% |
| 8 | A. Wald (1947) Sequential analysis | 0.811 | 4 | 2 | 100% |
| 9 | Volker Strassen (1967) Almost sure behavior of sums of independent random variables and martingales | 0.794 | 6 | 4 | 50% |
| 10 | Donald L McLeish (1974) Dependent central limit theorems and invariance principles | 0.737 | 3 | 2 | 100% |
Showing the top 10 of 76 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.
| Citing paper | Intensity | Mentions | Sections | |
|---|---|---|---|---|
| 1 | Demistifying Inference after Adaptive Experiments | 0.693 | 6 | 1 |