EconBase
← All papers

Generalizability with ignorance in mind: learning what we do (not) know for archetypes discovery

Emily Breza, Arun G. Chandrasekhar, Davide Viviano

arXiv 23 Jan 2025 · Econometrics

arXiv:2501.13355 · PDF · DOI · OpenAlex · Extracted main text

Abstract

When studying policy interventions, researchers often pursue two goals: i) identifying for whom the program has the largest effects (heterogeneity) and ii) determining whether those patterns of treatment effects have predictive power across environments (generalizability). We develop a framework to learn when and how to partition observations into groups of individual and environmental characterstics within which treatment effects are predictively stable, and when instead extrapolation is unwarranted and further evidence is needed. Our procedure determines in which contexts effects are generalizable and when, instead, researchers should admit ignorance and collect more data. We provide a decision-theoretic foundation, derive finite-sample regret guarantees, and establish asymptotic inference results. We illustrate the benefits of our approach by reanalyzing a multifaceted anti-poverty program across six countries.

Citation extraction

55
references
94
in-text mentions
55
distinct cited
2
self-citations
15,293
main-text words

appendix boundary found by appendix_command · 52% of the source is main text. Read the extracted text to check this.

Most heavily cited references

The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.

ReferenceIntensityMentionsSectionsMain text
1Banerjee, A., E. Duflo, N. Goldberg, D. Karlan, R. Osei, W. Parienté… (2015) A multifaceted program causes lasting progress for the very poor: Evidence from six countries1.00084100%
2Kitagawa, T. and A. Tetenov (2018) Who should be treated? empirical welfare maximization methods for treatment choice1.00053100%
3Bonhomme, S. and E. Manresa (2015) Grouped patterns of heterogeneity in panel data0.92843100%
4Venkateswaran, A., A. Sankar, A. G. Chandrasekhar, and T. H. McCormick (2024) Robustly estimating heterogeneity in factorial data using rashomon partitions0.92843100%
5Wager, S. and S. Athey (2018) Estimation and inference of heterogeneous treatment effects using random forests0.92843100%
6Gechter, M., K. Hirano, J. Lee, M. Mahmud, O. Mondal, J. Morduch, S.… (2024) Selecting experimental sites for external validity0.84333100%
7Chernozhukov, V., M. Demirer, E. Duflo, and I. Fernandez-Val (2018) Generic machine learning inference on heterogeneous treatment effects in randomized experiments, with an application to immuniza…0.81142100%
8Athey, S. and S. Wager (2021) Policy learning with observational data0.7374350%
9Broderick, T., R. Giordano, and R. Meager (2020) An automatic finite-sample robustness metric: when can dropping a little data make a big difference?0.64422100%
10Manski, C. F (2004) Statistical treatment rules for heterogeneous populations0.64422100%

Showing the top 10 of 55 scored citations.

Cited by, within the corpus

arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.

Citing paperIntensityMentionsSections
1Decision Theory for the Archetype Discovery Problem1.000104
2Better Measurement or Larger Samples? Data Collection for Policy Learning with Unobserved Heterogeneity0.51121
3Binary Classification with the Maximum Score Model and Linear Programming0.40511
4Learning What to Learn: Experimental Design when Combining Experimental with Observational Evidence0.40511
5Misspecification-Averse Estimation0.40511