EconBase
← All papers

Starting Small: Prioritizing Safety over Efficacy in Randomized Experiments Using the Exact Finite Sample Likelihood

Neil Christy, A. E. Kowalski

arXiv 25 Jul 2024 · Econometrics

arXiv:2407.18206 · PDF · DOI · OpenAlex · Extracted main text

Abstract

We use the exact finite sample likelihood and statistical decision theory to answer questions of “why?” and “what should you have done?” using data from randomized experiments and a utility function that prioritizes safety over efficacy. We propose a finite sample Bayesian decision rule and a finite sample maximum likelihood decision rule. We show that in finite samples from 2 to 50, it is possible for these rules to achieve better performance according to established maximin and maximum regret criteria than a rule based on the Boole-Frechet-Hoeffding bounds. We also propose a finite sample maximum likelihood criterion. We apply our rules and criterion to an actual clinical trial that yielded a promising estimate of efficacy, and our results point to safety as a reason for why results were mixed in subsequent trials.

Citation extraction

50
references
59
in-text mentions
50
distinct cited
0
self-citations
6,822
main-text words

appendix boundary found by appendix_titled_section at “Appendix” · 86% of the source is main text. Read the extracted text to check this.

Most heavily cited references

The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.

ReferenceIntensityMentionsSectionsMain text
1Manski, C. F (2004) Statistical treatment rules for heterogeneous populations0.92843100%
2Ben-Michael, E., K. Imai, and Z. Jiang (2024) Policy learning with asymmetric counterfactual utilities0.81142100%
3Stoye, J (2009) Minimax regret treatment choice with finite samples0.64422100%
4Zabet, M. H., M. Mohammadi, M. Ramezani, and H. Khalili (2016) Effect of high-dose ascorbic acid on vasopressor's requirement in septic shock0.64422100%
5Tian, J. and J. Pearl (2000) Probabilities of causation: Bounds and identification0.51121100%
6Balke, A. and J. Pearl (1997) Bounds on treatment effects from studies with imperfect compliance0.40511100%
7Boole, G (1854) Of statistical conditions0.40511100%
8Canner, P. L (1970) Selecting one of two treatments when the responses are dichotomous0.40511100%
9Copas, J. B (1973) Randomization models for the matched and unmatched 2 x 2 tables0.40511100%
10Cox, D. R (1958) Planning of Experiments0.40511100%

Showing the top 10 of 50 scored citations.

Cited by, within the corpus

arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.

Citing paperIntensityMentionsSections
1Counting Defiers: A Design-Based Model of an Experiment Can Reveal Evidence Beyond the Average Effect0.40511
23emHippocratic Utility0.40511