Zhengyuan Zhou, Susan Athey, Stefan Wager
arXiv 10 Oct 2018 · Statistics — Machine Learning
arXiv:1810.04778 · PDF · Extracted main text
In many settings, a decision-maker wishes to learn a rule, or policy, that maps from observable characteristics of an individual to an action. Examples include selecting offers, prices, advertisements, or emails to send to consumers, as well as the problem of determining which medication to prescribe to a patient. While there is a growing body of literature devoted to this problem, most existing results are focused on the case where data comes from a randomized experiment, and further, there are only two possible actions, such as giving a drug to a patient or not. In this paper, we study the offline multi-action policy learning problem with observational data and where the policy may need to respect budget constraints or belong to a restricted policy class such as decision trees. We build on the theory of efficient semi-parametric inference in order to propose and implement a policy learning algorithm that achieves asymptotically minimax-optimal regret. To the best of our knowledge, this is the first result of this type in the multi-action setup, and it provides a substantial performance improvement over the existing learning algorithms. We then consider additional computational challenges that arise in implementing our method for the case where the policy is restricted to take the form of a decision tree. We propose two different approaches, one using a mixed integer program formulation and the other using a tree-search based algorithm.
appendix boundary found by none_found · 100% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Swaminathan, Adith, Thorsten Joachims (2015) Batch learning from logged bandit feedback through counterfactual risk minimization | 1.000 | 12 | 4 | 100% |
| 2 | Athey, Susan, Stefan Wager (2017) Efficient policy learning self | 1.000 | 12 | 3 | 100% |
| 3 | Kitagawa, Toru, Aleksey Tetenov (2018) Who should be treated? empirical welfare maximization methods for treatment choice | 1.000 | 11 | 3 | 100% |
| 4 | Kallus, Nathan (2018) Balanced policy evaluation and learning | 1.000 | 9 | 3 | 100% |
| 5 | Zhang, Baqun, Anastasios A Tsiatis, Marie Davidian, Min Zhang, Eric… (2012) Estimating optimal treatment regimes from a classification perspective | 1.000 | 8 | 4 | 100% |
| 6 | Dudḱ, Miroslav, John Langford, Lihong Li (2011) Doubly robust policy evaluation and learning | 1.000 | 8 | 3 | 100% |
| 7 | Zhao, Yingqi, Donglin Zeng, A John Rush, Michael R Kosorok (2012) Estimating individualized treatment rules using outcome weighted learning | 1.000 | 7 | 3 | 100% |
| 8 | Zhao, Ying-Qi, Donglin Zeng, Eric B Laber, Rui Song, Ming Yuan, Mich… (2014) Doubly robust learning for estimating individualized treatment with censored data | 1.000 | 7 | 3 | 100% |
| 9 | Zhou, Xin, Nicole Mayer-Hamblett, Umer Khan, Michael R Kosorok (2017) Residual weighted learning for estimating individualized treatment rules | 1.000 | 6 | 3 | 100% |
| 10 | Bertsimas, Dimitris, Jack Dunn (2017) Optimal classification trees | 0.874 | 6 | 2 | 100% |
Showing the top 10 of 74 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.