arXiv 16 Dec 2021 · Econometrics · 1 citations (OpenAlex)
arXiv:2112.09259 · PDF · DOI · OpenAlex · Extracted main text
This paper studies the robustness of estimated policy effects to changes in the distribution of covariates. Robustness to covariate shifts is important, for example, when evaluating the external validity of quasi-experimental results, which are often used as a benchmark for evidence-based policy-making. I propose a novel scalar robustness metric. This metric measures the magnitude of the smallest covariate shift needed to invalidate a claim on the policy effect (for example, $ATE \geq 0$) supported by the quasi-experimental evidence. My metric links the heterogeneity of policy effects and robustness in a flexible, nonparametric way and does not require functional form assumptions. I cast the estimation of the robustness metric as a de-biased GMM problem. This approach guarantees a parametric convergence rate for the robustness metric while allowing for machine learning-based estimators of policy effect heterogeneity (for example, lasso, random forest, boosting, neural nets). I apply my procedure to the Oregon Health Insurance experiment. I study the robustness of policy effects estimates of health-care utilization and financial strain outcomes, relative to a shift in the distribution of context-specific covariates. Such covariates are likely to differ across US states, making quantification of robustness an important exercise for adoption of the insurance policy in states other than Oregon. I find that the effect on outpatient visits is the most robust among the metrics of health-care utilization considered.
appendix boundary found by none_found · 100% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | V. Chernozhukov, J. C. Escanciano, H. Ichimura, W. K. Newey, and J.… (2020) Locally robust semiparametric estimation, 2020 | 1.000 | 16 | 3 | 100% |
| 2 | Y.-C. Hsu, T.-C. Lai, and R. P. Lieli (2020) Counterfactual treatment effects: Estimation and inference | 1.000 | 5 | 4 | 100% |
| 3 | I. Komunjer and G. Ragusa (2016) Existence and characterization of conditional density projections | 1.000 | 5 | 3 | 100% |
| 4 | P. Ho (2023) Global robust bayesian analysis in large models | 0.928 | 4 | 4 | 100% |
| 5 | T. Christensen and B. Connault (2023) Counterfactual sensitivity and robustness | 0.928 | 4 | 3 | 100% |
| 6 | A. Finkelstein, S. Taubman, B. Wright, M. Bernstein, J. Gruber, J. P… (2012) The oregon health insurance experiment: evidence from the first year | 0.874 | 15 | 2 | 100% |
| 7 | I. Csiszár (1984) Sanov property, generalized i-projection and a conditional limit theorem | 0.874 | 5 | 2 | 100% |
| 8 | V. Chernozhukov, D. Chetverikov, M. Demirer, E. Duflo, C. Hansen, W.… (2018) Double/debiased machine learning for treatment and structural parameters | 0.811 | 4 | 2 | 100% |
| 9 | A. Finkelstein (2013) Oregon health insurance experiment public use data, 2013 | 0.811 | 4 | 2 | 100% |
| 10 | E. H. Kennedy, S. Balakrishnan, M. G’Sell, et al (2020) Sharp instruments for classifying compliers and generalizing causal effects | 0.811 | 4 | 2 | 100% |
Showing the top 10 of 50 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.