EconBase
← All papers

Assessing External Validity Over Worst-case Subpopulations

Sookyo Jeong, Hongseok Namkoong

arXiv 5 Jul 2020 · Statistics — Machine Learning · 2 citations (OpenAlex)

arXiv:2007.02411 · PDF · DOI · OpenAlex · Extracted main text

Abstract

Study populations are typically sampled from limited points in space and time, and marginalized groups are underrepresented. To assess the external validity of randomized and observational studies, we propose and evaluate the worst-case treatment effect (WTE) across all subpopulations of a given size, which guarantees positive findings remain valid over subpopulations. We develop a semiparametrically efficient estimator for the WTE that analyzes the external validity of the augmented inverse propensity weighted estimator for the average treatment effect. Our cross-fitting procedure leverages flexible nonparametric and machine learning-based estimates of nuisance parameters and is a regular root-$n$ estimator even when nuisance estimates converge more slowly. On real examples where external validity is of core concern, our proposed framework guards against brittle findings that are invalidated by unanticipated population shifts.

Citation extraction

85
references
140
in-text mentions
85
distinct cited
0
self-citations
8,952
main-text words

appendix boundary found by appendix_command · 33% of the source is main text. Read the extracted text to check this.

Most heavily cited references

The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.

ReferenceIntensityMentionsSectionsMain text
1V. Chernozhukov, D. Chetverikov, M. Demirer, E. Duflo, C. Hansen, W.… (2018) Double/debiased machine learning for treatment and structural parameters1.00084100%
2A. W. van der Vaart and J. A. Wellner (1996) Weak Convergence and Empirical Processes: With Applications to Statistics0.6939333%
3M. R. Rosenzweig and C. Udry (2020) External validity in a stochastic world: Evidence from low-income countries0.58531100%
4E. A. Stuart, S. R. Cole, C. P. Bradshaw, and P. J. Leaf (2011) The use of propensity scores to assess the generalizability of results from randomized trials0.58531100%
5E. Tipton (2013) Improving generalizations from experiments using propensity score subclassification: Assumptions, properties, and contexts0.58531100%
6P. J. Bickel, C. A. Klaassen, Y. Ritov, and J. A. Wellner (1993) Efficient and adaptive estimation for semiparametric models, volume 40.5115220%
7A. W. van der Vaart (1998) Asymptotic Statistics0.5115220%
8R. T. Rockafellar and S. Uryasev (2000) Optimization of conditional value-at-risk0.5112250%
9I. Andrews and E. Oster (2017) Weighting for external validity0.51121100%
10J. Angrist and I. Fernandez-Val (2010) Extrapolate-ing: External validity and overidentification in the late framework0.51121100%

Showing the top 10 of 85 scored citations.