EconBase
← All papers

Machine Learning Who to Nudge: Causal vs Predictive Targeting in a Field Experiment on Student Financial Aid Renewal

Susan Athey, Niall Keleher, Jann Spiess

arXiv 12 Oct 2023 · Econometrics · publishedJournal of Econometrics (2025) · 12 citations (OpenAlex)

arXiv:2310.08672 · PDF · DOI · OpenAlex · Extracted main text

Abstract

In many settings, interventions may be more effective for some individuals than others, so that targeting interventions may be beneficial. We analyze the value of targeting in the context of a large-scale field experiment with over 53,000 college students, where the goal was to use "nudges" to encourage students to renew their financial-aid applications before a non-binding deadline. We begin with baseline approaches to targeting. First, we target based on a causal forest that estimates heterogeneous treatment effects and then assigns students to treatment according to those estimated to have the highest treatment effects. Next, we evaluate two alternative targeting policies, one targeting students with low predicted probability of renewing financial aid in the absence of the treatment, the other targeting those with high probability. The predicted baseline outcome is not the ideal criterion for targeting, nor is it a priori clear whether to prioritize low, high, or intermediate predicted probability. Nonetheless, targeting on low baseline outcomes is common in practice, for example because the relationship between individual characteristics and treatment effects is often difficult or impossible to estimate with historical data. We propose hybrid approaches that incorporate the strengths of both predictive approaches (accurate estimation) and causal approaches (correct criterion); we show that targeting intermediate baseline outcomes is most effective in our specific application, while targeting based on low baseline outcomes is detrimental. In one year of the experiment, nudging all students improved early filing by an average of 6.4 percentage points over a baseline average of 37% filing, and we estimate that targeting half of the students using our preferred policy attains around 75% of this benefit.

Citation extraction

36
references
52
in-text mentions
36
distinct cited
8
self-citations
11,476
main-text words

appendix boundary found by appendix_command · 81% of the source is main text. Read the extracted text to check this.

Most heavily cited references

The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.

ReferenceIntensityMentionsSectionsMain text
1Hitsch, Günter, Sanjog Misra, and Walter Zhang (2023) Heterogeneous treatment effects and optimal targeting policy evaluation0.9285380%
2Wager, Stefan and Susan Athey (2018) Estimation and inference of heterogeneous treatment effects using random forests0.84333100%
3Chernozhukov, Victor, Mert Demirer, Esther Duflo, and Iván Fernández… (2019) Generic machine learning inference on heterogenous treatment effects in randomized experiments0.73732100%
4Yadlowsky, Steve, Scott Fleming, Nigam Shah, Emma Brunskill, and Ste… (2021) Evaluating treatment prioritization rules via rank-weighted average treatment effects0.73732100%
5Ascarza, Eva (2018) Retention futility: Targeting High-Risk customers might be ineffective0.64422100%
6Athey, Susan and Guido Imbens (2016) Recursive partitioning for heterogeneous causal effects self0.64422100%
7Athey, Susan, Julie Tibshirani, and Stefan Wager (2019) Generalized random forests self0.64422100%
8Athey, Susan and Stefan Wager (2021) Policy learning with observational data self0.64422100%
9Athey, Susan, Peter J Bickel, Aiyou Chen, Guido W Imbens, and Michae… (2023) Semi-parametric estimation of treatment effects in randomised experiments self0.64422100%
10Fernández-Lorá, Carlos and Foster Provost (2022) Causal classification: Treatment effect estimation vs. outcome prediction0.64422100%

Showing the top 10 of 36 scored citations.

Cited by, within the corpus

arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.

Citing paperIntensityMentionsSections
1Denoised IPW-Lasso for Heterogeneous Treatment Effect Estimation in Randomized Experiments0.64422
2Estimating treatment-effect heterogeneity across sites, in multi-site randomized experiments with few units per site0.40511
3Predicting the Distribution of Treatment Effects via Covariate-Adjustment, with an Application to Microcredit0.40511
4Training and Testing with Multiple Splits: A Central Limit Theorem for Split-Sample Estimators0.40511
5Nonparametric Bayesian Policy Learning0.40511