Christian A. Scholbeck, Giuseppe Casalicchio, Christoph Molnar, Bernd Bischl, Christian Heumann
arXiv 21 Jan 2022 · Machine Learning · publishedData Mining and Knowledge Discovery (2024) · 8 citations (OpenAlex)
arXiv:2201.08837 · PDF · DOI · OpenAlex · Extracted main text
Beta coefficients for linear regression models represent the ideal form of an interpretable feature effect. However, for non-linear models and especially generalized linear models, the estimated coefficients cannot be interpreted as a direct feature effect on the predicted outcome. Hence, marginal effects are typically used as approximations for feature effects, either in the shape of derivatives of the prediction function or forward differences in prediction due to a change in a feature value. While marginal effects are commonly used in many scientific fields, they have not yet been adopted as a model-agnostic interpretation method for machine learning models. This may stem from their inflexibility as a univariate feature effect and their inability to deal with the non-linearities found in black box models. We introduce a new class of marginal effects termed forward marginal effects. We argue to abandon derivatives in favor of better-interpretable forward differences. Furthermore, we generalize marginal effects based on forward differences to multivariate changes in feature values. To account for the non-linearity of prediction functions, we introduce a non-linearity measure for marginal effects. We argue against summarizing feature effects of a non-linear prediction function in a single metric such as the average marginal effect. Instead, we propose to partition the feature space to compute conditional average marginal effects on feature subspaces, which serve as conditional feature effect estimates.
appendix boundary found by appendix_command · 83% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Williams R (2012) Using the margins command to estimate and interpret adjusted predictions and marginal effects | 1.000 | 6 | 3 | 100% |
| 2 | Apley DW, Zhu J (2020) Visualizing the effects of predictor variables in black box supervised learning models | 0.737 | 3 | 3 | 67% |
| 3 | Bartus T (2005) Estimation of marginal effects using margeff | 0.737 | 3 | 2 | 100% |
| 4 | Mize TD, Doan L, Long JS (2019) A general framework for comparing predictions and marginal effects across models | 0.737 | 3 | 2 | 100% |
| 5 | Ribeiro MT, Singh S, Guestrin C (2016) "Why should I trust you?": Explaining the predictions of any classifier | 0.737 | 3 | 2 | 100% |
| 6 | Hastie T, Tibshirani R, Friedman J (2001) The Elements of Statistical Learning | 0.644 | 3 | 2 | 67% |
| 7 | Leeper TJ (2018) margins: Marginal effects for model objects | 0.644 | 2 | 2 | 100% |
| 8 | Slack D, Hilgard S, Jia E, Singh S, Lakkaraju H (2019) Fooling LIME and SHAP: Adversarial attacks on post hoc explanation methods | 0.644 | 2 | 2 | 100% |
| 9 | Saltelli A, Ratto M, Andres T, Campolongo F, Cariboni J, Gatelli D,… (2008) Global Sensitivity Analysis: The Primer | 0.511 | 3 | 2 | 33% |
| 10 | Friedman JH (2001) Greedy function approximation: A gradient boosting machine | 0.511 | 2 | 2 | 50% |
Showing the top 10 of 59 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.
| Citing paper | Intensity | Mentions | Sections | |
|---|---|---|---|---|
| 1 | fmeffects: An R Package for Forward Marginal Effects | 1.000 | 13 | 5 |