Patrick Rehill, Nicholas Biddle
arXiv 2 Sep 2023 · Econometrics
arXiv:2309.00805 · PDF · DOI · OpenAlex · Extracted main text
Causal machine learning methods which flexibly generate heterogeneous treatment effect estimates could be very useful tools for governments trying to make and implement policy. However, as the critical artificial intelligence literature has shown, governments must be very careful of unintended consequences when using machine learning models. One way to try and protect against unintended bad outcomes is with AI Fairness methods which seek to create machine learning models where sensitive variables like race or gender do not influence outcomes. In this paper we argue that standard AI Fairness approaches developed for predictive machine learning are not suitable for all causal machine learning applications because causal machine learning generally (at least so far) uses modelling to inform a human who is the ultimate decision-maker while AI Fairness approaches assume a model that is making decisions directly. We define these scenarios as indirect and direct decision-making respectively and suggest that policy-making is best seen as a joint decision where the causal machine learning model usually only has indirect power. We lay out a definition of fairness for this scenario - a model that provides the information a decision-maker needs to accurately make a value judgement about just policy outcomes - and argue that the complexity of causal machine learning models can make this difficult to achieve. The solution here is not traditional AI Fairness adjustments, but careful modelling and awareness of some of the decision-making biases that these methods might encourage which we describe.
appendix boundary found by none_found · 100% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Mittelstadt, Allo, Taddeo, Wachter \ Floridi (2016) `The ethics of algorithms: Mapping the debate', Big Data & Society 3(2), 205395171667967 | 1.000 | 9 | 4 | 100% |
| 2 | Athey, Tibshirani \ Wager (2019) `Generalized random forests', The Annals of Statistics 47(2), 1148–1178 | 1.000 | 5 | 3 | 100% |
| 3 | Ireni Saban \ Sherman (2022) Ethical governance of artificial intelligence in the public sector, Routledge focus, Routledge, Taylor & Francis Group, London ;… | 1.000 | 5 | 3 | 100% |
| 4 | Busuioc (2021) `Accountable artificial intelligence: Holding algorithms to account', Public Administration Review 81(5), 825–836 | 0.928 | 4 | 3 | 100% |
| 5 | Mehrabi, Morstatter, Saxena, Lerman \ Galstyan (2019) `A survey on bias and fairness in machine learning', arXiv preprint arXiv:1908.09635 | 0.928 | 4 | 3 | 100% |
| 6 | Chernozhukov, Chetverikov, Demirer, Duflo, Hansen, Newey \ Robins (2018) `Double/debiased machine learning for treatment and structural parameters', The Econometrics Journal 21(1), C1–C68 | 0.843 | 3 | 3 | 100% |
| 7 | Citron (2007) `Technological due process', Wash | 0.843 | 3 | 3 | 100% |
| 8 | Hünermund, Louw \ Caspi (2023) `Double machine learning and automated confounder selection: A cautionary tale', Journal of Causal Inference 11(1), 20220078 | 0.737 | 3 | 2 | 100% |
| 9 | Breiman (2001) `Statistical modeling: The two cultures (with comments and a rejoinder by the author)', Statistical Science 16(3), 199–231 | 0.644 | 2 | 2 | 100% |
| 10 | Corbett-Davies \ Goel (2018) `The measure and mismeasure of fairness: A critical review of fair machine learning', arXiv:1808.00023 [cs] | 0.644 | 2 | 2 | 100% |
Showing the top 10 of 60 scored citations.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.
| Citing paper | Intensity | Mentions | Sections | |
|---|---|---|---|---|
| 1 | Transparency challenges in policy evaluation with causal machine learning –- improving usability and accountability | 1.000 | 5 | 3 |
| 2 | How do applied researchers use the Causal Forest? A methodological review | 0.405 | 1 | 1 |