EconBase
← All papers

The Transfer Performance of Economic Models

Isaiah Andrews, Drew Fudenberg, Lihua Lei, Annie Liang, Chaofeng Wu

arXiv 10 Feb 2022 · Theoretical Economics · 12 citations (OpenAlex)

arXiv:2202.04796 · PDF · DOI · OpenAlex · Extracted main text

Abstract

Economists often estimate models using data from a particular domain, e.g. estimating risk preferences in a particular subject pool or for a specific class of lotteries. Whether a model's predictions extrapolate well across domains depends on whether the estimated model has captured generalizable structure. We provide a tractable formulation for this "out-of-domain" prediction problem and define the transfer error of a model based on how well it performs on data from a new domain. We derive finite-sample forecast intervals that are guaranteed to cover realized transfer errors with a user-selected probability when domains are iid, and use these intervals to compare the transferability of economic models and black box algorithms for predicting certainty equivalents. We find that in this application, the black box algorithms we consider outperform standard economic models when estimated and tested on data from the same domain, but the economic models generalize across domains better than the black-box algorithms do.

Citation extraction

99
references
403
in-text mentions
302
distinct cited
7
self-citations
14,231
main-text words

appendix boundary found by appendix_command · 52% of the source is main text. Read the extracted text to check this.

Most heavily cited references

The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.

ReferenceIntensityMentionsSectionsMain text
1Meager, R (2019) Understanding the Average Impact of Microcredit Expansions: A Bayesian Hierarchical Analysis of Seven Randomized Experiments0.84333100%
2Meager, R (2022) Aggregating Distributional Treatment Effects: A Bayesian Hierarchical Analysis of the Microcredit Literature0.84333100%
3Harless, D. W. and C. F. Camerer (1994) The Predictive Utility of Generalized Expected Utility Theories0.81142100%
4Plonsky, O., R. Apel, E. Ert, M. Tennenholtz, D. Bourgin, J. Peterso… (2019) Predicting human decisions with behavioral theories and machine learning0.81142100%
5Bruhin, A., H. Fehr-Duda, and T. Epper (2010) Risk and rationality: Uncovering heterogeneity in probability distortion0.7374350%
6Hsieh, S.-L., S. Ke, C. Zhao, and Z. Wang (2023) A Logit Neural-Network Utility Model0.7373367%
7Bernheim, B. D. and C. Sprenger (2020) On the empirical validity of cumulative prospect theory: Experimental evidence of rank-independent probability weighting0.7373367%
8Fudenberg, D., J. Kleinberg, A. Liang, and S. Mullainathan (2022) Measuring the Completeness of Economic Models self0.73732100%
9Hey, J. D. and C. Orme (1994) Investigating Generalizations of Expected Utility Theory Using Experimental Data0.73732100%
10Peysakhovich, A. and J. Naecker (2017) Using methods from machine learning to evaluate behavioral models of choice under risk and ambiguity0.73732100%

Showing the top 10 of 302 scored citations.

Cited by, within the corpus

arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.

Citing paperIntensityMentionsSections
1From Predictive Algorithms to Automatic Generation of Anomalies0.84333
2Transfer Estimates for Causal Effects across Heterogeneous Sites0.40511
32307.051220.40511
4Inference for Synthetic Controls via Refined Placebo Tests0.40511
5Randomization Inference: Theory and Applications0.40511
6Externally Valid Selection of Experimental Sites via the k-Median Problem0.40511
7Generalizability with ignorance in mind: learning what we do (not) know for archetypes discovery0.40511
8Training and Testing with Multiple Splits: A Central Limit Theorem for Split-Sample Estimators0.40511
9How Well Do LLMs Predict Human Behavior? A Measure of their Pretrained Knowledge0.40511