David Arbour, Harsh Parikh, Bijan Niknam, Elizabeth Stuart, Kara Rudolph, Avi Feller
arXiv 21 Sep 2025 · Machine Learning
arXiv:2509.17180 · PDF · DOI · OpenAlex · Extracted main text
Many common estimators in machine learning and causal inference are linear smoothers, where the prediction is a weighted average of the training outcomes. Some estimators, such as ordinary least squares and kernel ridge regression, allow for arbitrarily negative weights, which improve feature imbalance but often at the cost of increased dependence on parametric modeling assumptions and higher variance. By contrast, estimators like importance weighting and random forests (sometimes implicitly) restrict weights to be non-negative, reducing dependence on parametric modeling and variance at the cost of worse imbalance. In this paper, we propose a unified framework that directly penalizes the level of extrapolation, replacing the current practice of a hard non-negativity constraint with a soft constraint and corresponding hyperparameter. We derive a worst-case extrapolation error bound and introduce a novel "bias-bias-variance" tradeoff, encompassing biases due to feature imbalance, model misspecification, and estimator variance; this tradeoff is especially pronounced in high dimensions, particularly when positivity is poor. We then develop an optimization procedure that regularizes this bound while minimizing imbalance and outline how to use this approach as a sensitivity analysis for dependence on parametric modeling assumptions. We demonstrate the effectiveness of our approach through synthetic experiments and a real-world application, involving the generalization of randomized controlled trial estimates to a target population of interest.
appendix boundary found by appendix_command · 66% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Ben-Michael, Eli and Feller, Avi and Rothstein, Jesse (2021) The augmented synthetic control method self | 1.000 | 6 | 3 | 100% |
| 2 | Chattopadhyay, Ambarish and Zubizarreta, José R (2023) On the implied weights of linear regression for causal inference | 0.928 | 4 | 3 | 100% |
| 3 | Bruns-Smith, David and Dukes, Oliver and Feller, Avi and Ogburn, Eli… (2023) Augmented balancing weights as linear regression self | 0.811 | 4 | 2 | 100% |
| 4 | Lin, Zhexiao and Han, Fang (2022) On regression-adjusted imputation estimators of the average treatment effect | 0.811 | 4 | 2 | 100% |
| 5 | Abadie, Alberto and Diamond, Alexis and Hainmueller, Jens (2010) Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program | 0.737 | 3 | 2 | 100% |
| 6 | Parikh, Harsh and Ross, Rachael and Stuart, Elizabeth and Rudolph, K… (2025) Who Are We Missing?: A Principled Approach to Characterizing the Underrepresented Population self | 0.737 | 3 | 2 | 100% |
| 7 | Ben-Michael, Eli and Feller, Avi and Hirshberg, David A and Zubizarr… (2021) The balancing act in causal inference self | 0.644 | 2 | 2 | 100% |
| 8 | Chernozhukov, Victor and Chetverikov, Denis and Demirer, Mert and Du… (2018) Double/debiased machine learning for treatment and structural parameters | 0.644 | 2 | 2 | 100% |
| 9 | Johansson, Fredrik D and Shalit, Uri and Kallus, Nathan and Sontag,… (2022) Generalization bounds and representation learning for estimation of potential outcomes and causal effects | 0.644 | 2 | 2 | 100% |
| 10 | Knaus, Michael C (2024) Treatment Effect Estimators as Weighted Outcomes | 0.644 | 2 | 2 | 100% |
Showing the top 10 of 40 scored citations.