Carlos Fernández-Loría, Foster Provost, Jesse Anderton, Benjamin Carterette, Praveen Chandar
arXiv 24 Apr 2020 · Econometrics · publishedInformation Systems Research (2022) · 11 citations (OpenAlex)
arXiv:2004.11532 · PDF · DOI · OpenAlex · Extracted main text
This study presents a systematic comparison of methods for individual treatment assignment, a general problem that arises in many applications and has received significant attention from economists, computer scientists, and social scientists. We group the various methods proposed in the literature into three general classes of algorithms (or metalearners): learning models to predict outcomes (the O-learner), learning models to predict causal effects (the E-learner), and learning models to predict optimal treatment assignments (the A-learner). We compare the metalearners in terms of (1) their level of generality and (2) the objective function they use to learn models from data; we then discuss the implications that these characteristics have for modeling and decision making. Notably, we demonstrate analytically and empirically that optimizing for the prediction of outcomes or causal effects is not the same as optimizing for treatment assignments, suggesting that in general the A-learner should lead to better treatment assignments than the other metalearners. We demonstrate the practical implications of our findings in the context of choosing, for each user, the best algorithm for playlist generation in order to optimize engagement. This is the first comparison of the three different metalearners on a real-world application at scale (based on more than half a billion individual treatment assignments). In addition to supporting our analytical findings, the results show how large A/B tests can provide substantial value for learning treatment assignment policies, rather than simply choosing the variant that performs best on average.
appendix boundary found by none_found · 100% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Alina Beygelzimer and John Langford (2009) The offset tree for learning with partial labels. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge di… | 1.000 | 9 | 4 | 100% |
| 2 | Lihong Li, Wei Chu, John Langford, and Robert E Schapire (2010) A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on… | 1.000 | 5 | 3 | 100% |
| 3 | Baqun Zhang, Anastasios A Tsiatis, Marie Davidian, Min Zhang, and Er… (2012) Estimating optimal treatment regimes from a classification perspective | 1.000 | 5 | 3 | 100% |
| 4 | Susan Athey and Guido Imbens (2016) Recursive partitioning for heterogeneous causal effects | 0.928 | 4 | 3 | 100% |
| 5 | Susan Athey and Stefan Wager (2021) Policy learning with observational data | 0.874 | 6 | 2 | 100% |
| 6 | Toru Kitagawa and Aleksey Tetenov (2018) Who should be treated? empirical welfare maximization methods for treatment choice | 0.874 | 6 | 2 | 100% |
| 7 | Diego Olaya, Kristof Coussement, and Wouter Verbeke (2020) A survey and benchmarking study of multitreatment uplift modeling | 0.874 | 5 | 2 | 100% |
| 8 | Yingqi Zhao, Donglin Zeng, A John Rush, and Michael R Kosorok (2012) Estimating individualized treatment rules using outcome weighted learning | 0.811 | 4 | 2 | 100% |
| 9 | Adam N Elmachtoub and Paul Grigas (2021) Smart “predict, then optimize” | 0.737 | 3 | 2 | 100% |
| 10 | Charles F Manski (2004) Statistical treatment rules for heterogeneous populations | 0.737 | 3 | 2 | 100% |
Showing the top 10 of 50 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.
| Citing paper | Intensity | Mentions | Sections | |
|---|---|---|---|---|
| 1 | Unveiling the Potential of Robustness in Selecting Conditional Average Treatment Effect Estimators | 0.405 | 1 | 1 |