EconBase
← All papers

Learning What to Learn: Experimental Design when Combining Experimental with Observational Evidence

Aristotelis Epanomeritakis, Davide Viviano

arXiv 27 Oct 2025 · Econometrics

arXiv:2510.23434 · PDF · Extracted main text

Abstract

Experiments deliver credible treatment-effect estimates but, because they are costly, are often restricted to specific sites, small populations, or particular mechanisms. A common practice across several fields is therefore to combine experimental estimates with reduced-form or structural external (observational) evidence to answer broader policy questions such as those involving general equilibrium effects or external validity. We develop a unified framework for the design of experiments when combined with external evidence, i.e., choosing which experiment(s) to run and how to allocate sample size under arbitrary budget constraints. Because observational evidence may suffer bias unknown ex-ante, we evaluate designs using a minimax proportional-regret criterion that compares any candidate design to an oracle that knows the observational study bias and jointly chooses the design and estimator. This yields a transparent bias-variance trade-off that does not require the researcher to specify a bias bound and relies only on information already needed for conventional power calculations. We illustrate the framework by (i) designing cash-transfer experiments aimed at estimating general equilibrium effects and (ii) optimizing site selection for microfinance interventions.

Citation extraction

90
references
163
in-text mentions
90
distinct cited
1
self-citations
17,275
main-text words

appendix boundary found by appendix_command · 78% of the source is main text. Read the extracted text to check this.

Most heavily cited references

The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.

ReferenceIntensityMentionsSectionsMain text
1Andrews, I., M. Gentzkow, and J. M. Shapiro (2020) Transparency in structural research1.00084100%
2Todd, P. E. and K. I. Wolpin (2006) Assessing the impact of a school subsidy program in mexico: Using a social experiment to validate a dynamic behavioral model of…1.00083100%
3Egger, D., J. Haushofer, E. Miguel, P. Niehaus, and M. Walker (2022) General equilibrium effects of cash transfers: Experimental evidence from Kenya1.00053100%
4Gechter, M., K. Hirano, J. Lee, M. Mahmud, O. Mondal, J. Morduch, S.… (2024) Selecting experimental sites for external validity1.00053100%
5Tsybakov, A. B (1998) Pointwise and sup-norm sharp adaptive estimation of functions on the sobolev classes0.9285380%
6Armstrong, T. B. and M. Kolesár (2018) Optimal inference in a class of regression models0.92843100%
7Attanasio, O. P., C. Meghir, and A. Santiago (2012) Education choices in mexico: using a structural model and a randomized experiment to evaluate progresa0.92843100%
8Banerjee, A., E. Breza, A. G. Chandrasekhar, E. Duflo, M. O. Jackson… (2024) Changes in social network structure in response to exposure to formal credit markets0.874102100%
9Armstrong, T. B., P. Kline, and L. Sun (2024) Adapting to misspecification0.86011464%
10Gerber, A. S. and D. P. Green (2012) Field Experiments: Design, Analysis, and Interpretation0.84333100%

Showing the top 10 of 90 scored citations.

Cited by, within the corpus

arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.

Citing paperIntensityMentionsSections
1Better Measurement or Larger Samples? Data Collection for Policy Learning with Unobserved Heterogeneity0.51121