arXiv 2 Oct 2025 · Econometrics
arXiv:2510.02507 · PDF · DOI · OpenAlex · Extracted main text
Pre-analysis plans (PAPs) have become standard in experimental economics research, but it is nevertheless common to see researchers deviating from their PAPs to supplement preregistered estimates with non-prespecified findings. While such ex-post analysis can yield valuable insights, there is broad uncertainty over how to interpret -- or whether to even acknowledge -- non-preregistered results. In this paper, we consider the case of a truth-seeking researcher who, after seeing the data, earnestly wishes to report additional estimates alongside those preregistered in their PAP. We show that, even absent "nefarious" behavior, conventional confidence intervals and point estimators are invalid due to the fact that non-preregistered estimates are only reported in a subset of potential data realizations. We propose inference procedures that account for this conditional reporting. We apply these procedures to Bessone et al. (2021), which studies the economic effects of increased sleep among the urban poor. We demonstrate that, depending on the reason for deviating, the adjustments from our procedures can range from having no difference to an economically significant difference relative to conventional practice. Finally, we consider the robustness of our procedure to certain forms of misspecification, motivating possible heuristic checks and norms for journals to adopt.
appendix boundary found by appendix_command · 57% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Olken, B. A (2015) Promises and perils of pre-analysis plans | 1.000 | 7 | 3 | 100% |
| 2 | Bessone, P., Rao, G., Schilbach, F., Schofield, H., and Toma, M (2021) The economic consequences of increasing sleep among the urban poor | 0.969 | 22 | 8 | 91% |
| 3 | McCloskey, A (2024) Hybrid confidence intervals for informative uniform asymptotic inference after model selection | 0.956 | 8 | 3 | 88% |
| 4 | Andrews, I., Kitagawa, T., and McCloskey, A (2024) Inference on winners | 0.928 | 10 | 5 | 80% |
| 5 | Lee, J. D., Sun, D. L., Sun, Y., and Taylor, J. E (2016) Exact post-selection inference, with application to the lasso | 0.928 | 5 | 3 | 80% |
| 6 | Fithian, W., Sun, D., and Taylor, J (2014) Optimal inference after model selection | 0.843 | 4 | 3 | 75% |
| 7 | Banerjee, A., Duflo, E., Finkelstein, A., Katz, L. F., Olken, B. A.,… (2020) In praise of moderation: Suggestions for the scope and use of pre-analysis plans for rcts in economics | 0.811 | 4 | 2 | 100% |
| 8 | Pfanzagl, J (1994) Parametric Statistical Theory | 0.794 | 6 | 3 | 50% |
| 9 | Christensen, G. and Miguel, E (2018) Transparency, reproducibility, and the credibility of economics research | 0.644 | 2 | 2 | 100% |
| 10 | Miguel, E (2021) Evidence on research transparency in economics | 0.644 | 2 | 2 | 100% |
Showing the top 10 of 68 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.
| Citing paper | Intensity | Mentions | Sections | |
|---|---|---|---|---|
| 1 | Integrating Diagnostic Checks into Estimation | 0.737 | 3 | 2 |
| 2 | Inference on effect size after multiple hypothesis testing | 0.644 | 2 | 2 |
| 3 | Dynamically Consistent Statistical Decisions | 0.405 | 1 | 1 |