EconBase
← All papers

Treatment Effect Risk: Bounds and Inference

Nathan Kallus

arXiv 15 Jan 2022 · Statistics — Methodology · publishedManagement Science (2023) · 5 citations (OpenAlex)

arXiv:2201.05893 · PDF · DOI · OpenAlex · Extracted main text

Abstract

Since the average treatment effect (ATE) measures the change in social welfare, even if positive, there is a risk of negative effect on, say, some 10% of the population. Assessing such risk is difficult, however, because any one individual treatment effect (ITE) is never observed, so the 10% worst-affected cannot be identified, while distributional treatment effects only compare the first deciles within each treatment group, which does not correspond to any 10%-subpopulation. In this paper we consider how to nonetheless assess this important risk measure, formalized as the conditional value at risk (CVaR) of the ITE-distribution. We leverage the availability of pre-treatment covariates and characterize the tightest-possible upper and lower bounds on ITE-CVaR given by the covariate-conditional average treatment effect (CATE) function. We then proceed to study how to estimate these bounds efficiently from data and construct confidence intervals. This is challenging even in randomized experiments as it requires understanding the distribution of the unknown CATE function, which can be very complex if we use rich covariates so as to best control for heterogeneity. We develop a debiasing method that overcomes this and prove it enjoys favorable statistical properties even when CATE and other nuisances are estimated by black-box machine learning or even inconsistently. Studying a hypothetical change to French job-search counseling services, our bounds and inference demonstrate a small social benefit entails a negative impact on a substantial subpopulation.

Citation extraction

58
references
90
in-text mentions
58
distinct cited
6
self-citations
8,093
main-text words

appendix boundary found by appendix_command · 60% of the source is main text. Read the extracted text to check this.

Most heavily cited references

The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.

ReferenceIntensityMentionsSectionsMain text
1Edward H Kennedy (2020) Optimal doubly robust estimation of heterogeneous causal effects1.00063100%
2Xinkun Nie and Stefan Wager (2021) Quasi-oracle estimation of heterogeneous treatment effects0.81142100%
3Stefan Wager and Susan Athey (2018) Estimation and inference of heterogeneous treatment effects using random forests0.73732100%
4Susan Athey and Guido Imbens (2016) Recursive partitioning for heterogeneous causal effects0.73732100%
5Nathan Kallus, Xiaojie Mao, and Masatoshi Uehara (2019) Localized debiased machine learning: Efficient inference on quantile treatment effects and beyond self0.73732100%
6Kosuke Imai and Marc Ratkovic (2013) Estimating treatment effect heterogeneity in randomized program evaluation0.73732100%
7Sören R Künzel, Jasjeet S Sekhon, Peter J Bickel, and Bin Yu (2019) Metalearners for estimating heterogeneous treatment effects using machine learning0.73732100%
8Richard K Crump, V Joseph Hotz, Guido W Imbens, and Oscar A Mitnik (2008) Nonparametric tests for treatment effect heterogeneity0.64422100%
9Jacob Dorn, Kevin Guo, and Nathan Kallus (2021) Doubly-Valid/Doubly-Sharp Sensitivity Analysis for Causal Inference with Unmeasured Confounding0.64422100%
10Nathan Kallus, Xiaojie Mao, and Angela Zhou (2021) Assessing algorithmic fairness with unobserved protected class using data combination self0.64422100%

Showing the top 10 of 58 scored citations.

Cited by, within the corpus

arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.

Citing paperIntensityMentionsSections
1What's the Harm? Sharp Bounds on the Fraction Negatively Affected by Treatment0.87462
2Welfare at Risk: Distributional impact of policy interventions0.58531
3Doubly-Valid/Doubly-Sharp Sensitivity Analysis for Causal Inference with Unmeasured Confounding0.40511
4Policy Learning with Distributional Welfare0.40511
5On the Lower Confidence Band for the Optimal Welfare in Policy Learning0.40511
6Individual Treatment Effect: Prediction Intervals and Sharp Bounds0.40511