arXiv 20 Aug 2022 · Econometrics · 1 citations (OpenAlex)
arXiv:2208.09638 · PDF · DOI · OpenAlex · Extracted main text
What is the purpose of pre-analysis plans, and how should they be designed? We model the interaction between an agent who analyzes data and a principal who makes a decision based on agent reports. The agent could be the manufacturer of a new drug, and the principal a regulator deciding whether the drug is approved. Or the agent could be a researcher submitting a research paper, and the principal an editor deciding whether it is published. The agent decides which statistics to report to the principal. The principal cannot verify whether the analyst reported selectively. Absent a pre-analysis message, if there are conflicts of interest, then many desirable decision rules cannot be implemented. Allowing the agent to send a message before seeing the data increases the set of decision rules that can be implemented, and allows the principal to leverage agent expertise. The optimal mechanisms that we characterize require pre-analysis plans. Applying these results to hypothesis testing, we show that optimal rejection rules pre-register a valid test, and make worst-case assumptions about unreported statistics. Optimal tests can be found as a solution to a linear-programming problem.
appendix boundary found by appendix_command · 77% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Lehmann, Erich L and Joseph P Romano (2006) Testing statistical hypotheses | 0.843 | 4 | 4 | 75% |
| 2 | Gneiting, Tilmann and Adrian E Raftery (2007) Strictly proper scoring rules, prediction, and estimation | 0.843 | 4 | 3 | 75% |
| 3 | DellaVigna, Stefano and Devin Pope (2018) What motivates effort? evidence and expert forecasts | 0.811 | 4 | 2 | 100% |
| 4 | Myerson, Roger B (1986) Multistage games with communication | 0.811 | 4 | 2 | 100% |
| 5 | Savage, Leonard J (1971) Elicitation of personal probabilities and expectations | 0.737 | 3 | 2 | 100% |
| 6 | Andrews, Isaiah and Jesse M Shapiro (2021) A model of scientific communication | 0.644 | 2 | 2 | 100% |
| 7 | Coffman, Lucas C. and Muriel Niederle (2015) Pre-analysis plans have limited upside, especially where replications are feasible | 0.644 | 2 | 2 | 100% |
| 8 | Frankel, Alexander and Maximilian Kasy (2022) Which findings should be published? | 0.644 | 2 | 2 | 100% |
| 9 | Olken, Benjamin A (2015) Promises and perils of pre-analysis plans | 0.644 | 2 | 2 | 100% |
| 10 | Duflo, Esther, Abhijit V Banerjee, Amy Finkelstein, Lawrence F Katz,… (2020) In praise of moderation: Suggestions for the scope and use of pre-analysis plans for RCTs in economics | 0.644 | 2 | 2 | 100% |
Showing the top 10 of 44 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.
| Citing paper | Intensity | Mentions | Sections | |
|---|---|---|---|---|
| 1 | Publication Design with Incentives in Mind | 0.737 | 3 | 2 |
| 2 | “Post” Pre-Analysis Plans: Valid Inference for Non-Preregistered Specifications | 0.511 | 2 | 1 |
| 3 | A model of multiple hypothesis testing | 0.405 | 1 | 1 |
| 4 | Do t-Statistic Hurdles Need to be Raised? | 0.405 | 1 | 1 |
| 5 | Testing the Fairness-Accuracy Improvability of Algorithms | 0.405 | 1 | 1 |
| 6 | Optimal Post-Hoc Theorizing | 0.405 | 1 | 1 |
| 7 | Identification Design | 0.405 | 1 | 1 |
| 8 | Dynamically Consistent Statistical Decisions | 0.405 | 1 | 1 |