EconBase
← All papers

Identification of and correction for publication bias

Isaiah Andrews, Maximilian Kasy

arXiv 28 Nov 2017 · Econometrics · publishedAmerican Economic Review (2019) · 380 citations (OpenAlex)

arXiv:1711.10527 · PDF · DOI · OpenAlex · Extracted main text

Abstract

Some empirical results are more likely to be published than others. Such selective publication leads to biased estimates and distorted inference. This paper proposes two approaches for identifying the conditional probability of publication as a function of a study's results, the first based on systematic replication studies and the second based on meta-studies. For known conditional publication probabilities, we propose median-unbiased estimators and associated confidence sets that correct for selective publication. We apply our methods to recent large-scale replication studies in experimental economics and psychology, and to meta-studies of the effects of minimum wages and de-worming programs.

Citation extraction

51
references
153
in-text mentions
51
distinct cited
0
self-citations
14,464
main-text words

appendix boundary found by appendix_command · 48% of the source is main text. Read the extracted text to check this.

Most heavily cited references

The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.

ReferenceIntensityMentionsSectionsMain text
1Brodeur, A., Lé, M., Sangnier, M., and Zylberberg, Y (2016) Star wars: The empirics strike back0.92843100%
2Camerer, C. F., Dreber, A., Forsell, E., Ho, T.-H., Huber, J., Johan… (2016) Evaluating replicability of laboratory experiments in economics0.89922773%
3Croke, K., Hicks, J. H., Hsu, E., Kremer, M., and Miguel, E (2016) Does mass deworming affect child nutrition? Meta-analysis, cost-effectiveness, and statistical power0.87412567%
4McCrary, J., Christensen, G., and Fanelli, D (2016) Conservative tests under satisficing models of publication bias0.87452100%
5Wolfson, P. J. and Belman, D (2015) 15 years of research on us employment and the minimum wage0.85113562%
6Card, D. and Krueger, A. B (1995) Time-series minimum-wage studies: A meta-analysis0.84333100%
7Open Science Collaboration (2015) Estimating the reproducibility of psychological science0.80425752%
8Gilbert, D. T., King, G., Pettigrew, S., and Wilson, T. D (2016) Estimating the reproducibility of psychological science0.7373367%
9Christensen, G. S. and Miguel, E (2016) Transparency, reproducibility, and the credibility of economics research0.73732100%
10Hedges, L. V (1992) Modeling publication selection effects in meta-analysis0.6443267%

Showing the top 10 of 51 scored citations.

Cited by, within the corpus

arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.

Citing paperIntensityMentionsSections
1Literature Review and Evidence Aggregation: a Toolkit for Applied Micro1.000163
2Testing for Underpowered Literatures1.00074
3Do t-Statistic Hurdles Need to be Raised?1.00064
4Should We Adjust for the Test for Pre-trends in Difference-in-Difference Designs?0.84343
5The Power of Tests for Detecting $p$-Hacking0.64422
6Inference After Ranking with Applications to Economic Mobility0.64422
7Optimal Post-Hoc Theorizing0.64422
8IMF Programs and Growth: A Source-Informed Robustness Reanalysis0.64422
9An Overview of Meta-Analytic Methods for Economic Research0.51121
10Publication Design with Incentives in Mind0.51121