Qizhao Chen, Morgane Austern, Vasilis Syrgkanis
arXiv 8 Mar 2023 · Econometrics
arXiv:2303.04416 · PDF · DOI · OpenAlex · Extracted main text
Estimating optimal dynamic policies from offline data is a fundamental problem in dynamic decision making. In the context of causal inference, the problem is known as estimating the optimal dynamic treatment regime. Even though there exists a plethora of methods for estimation, constructing confidence intervals for the value of the optimal regime and structural parameters associated with it is inherently harder, as it involves non-linear and non-differentiable functionals of unknown quantities that need to be estimated. Prior work resorted to sub-sample approaches that can deteriorate the quality of the estimate. We show that a simple soft-max approximation to the optimal treatment regime, for an appropriately fast growing temperature parameter, can achieve valid inference on the truly optimal regime. We illustrate our result for a two-period optimal dynamic regime, though our approach should directly extend to the finite horizon case. Our work combines techniques from semi-parametric inference and $g$-estimation, together with an appropriate triangular array central limit theorem, as well as a novel analysis of the asymptotic influence and asymptotic bias of softmax approximations.
appendix boundary found by appendix_command · 29% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | James M Robins (2004) Optimal structural nested models for optimal sequential decisions | 1.000 | 8 | 4 | 100% |
| 2 | Bibhas Chakraborty, Susan Murphy, and Victor Strecher (2010) Inference for non-regular parameters in optimal dynamic treatment regimes | 0.874 | 5 | 2 | 100% |
| 3 | Greg Lewis and Vasilis Syrgkanis (2020) Double/debiased machine learning for dynamic treatment effects via g-estimation self | 0.843 | 3 | 3 | 100% |
| 4 | Victor Chernozhukov, Whitney Newey, Rahul Singh, and Vasilis Syrgkanis (2020) Adversarial estimation of riesz representers self | 0.737 | 5 | 4 | 40% |
| 5 | Susan A Murphy (2003) Optimal dynamic treatment regimes | 0.737 | 3 | 2 | 100% |
| 6 | Kevin Gunn, Wenbin Lu, and Rui Song (2022) Adaptive semi-supervised inference for optimal treatment decisions with electronic medical record data | 0.644 | 2 | 2 | 100% |
| 7 | Chengchun Shi, Shikai Luo, Yuan Le, Hongtu Zhu, and Rui Song (2022) Statistically efficient advantage learning for offline reinforcement learning in infinite horizons | 0.644 | 2 | 2 | 100% |
| 8 | Rui Song, Shikai Luo, Donglin Zeng, Hao Helen Zhang, Wenbin Lu, and… (2017) Semiparametric single-index model for estimating optimal individualized treatment strategy | 0.644 | 2 | 2 | 100% |
| 9 | Peter J Bickel and Yaacov Ritov (1988) Estimating integrated squared density derivatives: Sharp best order of convergence estimates | 0.511 | 2 | 1 | 100% |
| 10 | Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo,… (2018) Double/debiased machine learning for treatment and structural parameters, 2018 | 0.511 | 2 | 1 | 100% |
Showing the top 10 of 99 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.