Yong Cai, Agathe Pernoud, Boli Xu
arXiv 4 Sep 2026 · Econometrics
arXiv:2609.05372 · PDF · Extracted main text
This paper studies the effects of p-hacking on the bias of published estimates when papers with statistically significant results are selectively published. We show that fast p-hacking---actions that lead to large changes in p-values---always exacerbates the bias from selective publication. On the other hand, slow p-hacking---actions that lead to small changes in p-values---exacerbates bias when selection is weak, but mitigates it when selection is strong. In a model featuring both types of p-hacking, we show that a normality assumption identifies the true distribution of effects as well as the counterfactual mean that would obtain under selective publication without p-hacking. Applying the model to meta-analyses on the effects of behavioral nudges and development aid, we find suggestive evidence that both mitigation and exacerbation can arise in practice.
appendix boundary found by appendix_command · 32% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Andrews, Isaiah and Kasy, Maximilian (2019) Identification of and correction for publication bias | 1.000 | 6 | 3 | 100% |
| 2 | Brodeur, Abel and Lé, Mathias and Sangnier, Marc and Zylberberg, Yanos (2016) Star wars: The empirics strike back | 0.874 | 6 | 2 | 100% |
| 3 | Simonsohn, Uri (2020) p-hacking fast and slow: Evaluating a forthcoming AER paper deeming some econ literatures less trustworthy | 0.843 | 4 | 4 | 75% |
| 4 | Masicampo, EJ and Lalande, Daniel R (2012) A peculiar prevalence of p values just below. 05 | 0.843 | 3 | 3 | 100% |
| 5 | Doucouliagos, Hristos and Paldam, Martin (2011) The ineffectiveness of development aid on growth: An update | 0.817 | 11 | 3 | 55% |
| 6 | Mertens, Stephanie and Herberz, Mario and Hahnel, Ulf JJ and Brosch,… (2022) The effectiveness of nudging: A meta-analysis of choice architecture interventions across behavioral domains | 0.817 | 11 | 3 | 55% |
| 7 | Friese, Malte and Frankenbach, Julius (2020) p-Hacking and publication bias interact to distort meta-analytic effect size estimates. | 0.811 | 4 | 2 | 100% |
| 8 | Hedges, Larry V (1984) Estimation of effect size under nonrandom sampling: The effects of censoring studies yielding statistically insignificant mean d… | 0.737 | 3 | 2 | 100% |
| 9 | Iyengar, Satish and Greenhouse, Joel B (1988) Selection models and the file drawer problem | 0.737 | 3 | 2 | 100% |
| 10 | Jagadeesan, Ravi and Viviano, Davide (2024) Publication Design with Incentives in Mind | 0.737 | 3 | 2 | 100% |
Showing the top 10 of 64 scored citations.