arXiv 3 Nov 2022 · Statistics — Methodology
arXiv:2211.01547 · PDF · DOI · OpenAlex · Extracted main text
To effectively optimize and personalize treatments, it is necessary to investigate the heterogeneity of treatment effects. With the wide range of users being treated over many online controlled experiments, the typical approach of manually investigating each dimension of heterogeneity becomes overly cumbersome and prone to subjective human biases. We need an efficient way to search through thousands of experiments with hundreds of target covariates and hundreds of breakdown dimensions. In this paper, we propose a systematic paradigm for detecting, surfacing and characterizing heterogeneous treatment effects. First, we detect if treatment effect variation is present in an experiment, prior to specifying any breakdowns. Second, we surface the most relevant dimensions for heterogeneity. Finally, we characterize the heterogeneity beyond just the conditional average treatment effects (CATE) by studying the conditional distributions of the estimated individual treatment effects. We show the effectiveness of our methods using simulated data and empirical studies.
appendix boundary found by appendix_command · 91% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Ding, P., A. Feller, and L. Miratrix (2019) Decomposing treatment effect variation | 0.874 | 6 | 2 | 100% |
| 2 | Ding, P., A. Feller, and L. Miratrix (2016) Randomization inference for treatment effect variation | 0.737 | 3 | 2 | 100% |
| 3 | Cox, D. R (1984) Interaction | 0.405 | 1 | 1 | 100% |
| 4 | Aronow, P. M., D. P. Green, and D. K. Lee (2014) Sharp bounds on the variance in randomized experiments | 0.405 | 1 | 1 | 100% |
| 5 | Benjamini, Y. and Y. Hochberg (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing | 0.405 | 1 | 1 | 100% |
| 6 | Chang, M., S. Lee, and Y.-J. Whang (2015) Nonparametric tests of conditional treatment effects with an application to single-sex schooling on academic achievements | 0.405 | 1 | 1 | 100% |
| 7 | Crump, R. K., V. J. Hotz, G. W. Imbens, and O. A. Mitnik (2008) Nonparametric tests for treatment effect heterogeneity | 0.405 | 1 | 1 | 100% |
| 8 | Fan, Y. and S. S. Park (2010) Sharp bounds on the distribution of treatment effects and their statistical inference | 0.405 | 1 | 1 | 100% |
| 9 | Heckman, J. J., J. Smith, and N. Clements (1997) Making the most out of programme evaluations and social experiments: Accounting for heterogeneity in programme impacts | 0.405 | 1 | 1 | 100% |
| 10 | Hilbe, J. M (2011) Negative binomial regression | 0.405 | 1 | 1 | 100% |
Showing the top 10 of 11 scored citations.