Guillaume Saint-Jacques, Amir Sepehri, Nicole Li, Igor Perisic
arXiv 14 Feb 2020 · cs.SI · 5 citations (OpenAlex)
arXiv:2002.05819 · PDF · DOI · OpenAlex · Extracted main text
As technology continues to advance, there is increasing concern about individuals being left behind. Many businesses are striving to adopt responsible design practices and avoid any unintended consequences of their products and services, ranging from privacy vulnerabilities to algorithmic bias. We propose a novel approach to fairness and inclusiveness based on experimentation. We use experimentation because we want to assess not only the intrinsic properties of products and algorithms but also their impact on people. We do this by introducing an inequality approach to A/B testing, leveraging the Atkinson index from the economics literature. We show how to perform causal inference over this inequality measure. We also introduce the concept of site-wide inequality impact, which captures the inclusiveness impact of targeting specific subpopulations for experiments, and show how to conduct statistical inference on this impact. We provide real examples from LinkedIn, as well as an open-source, highly scalable implementation of the computation of the Atkinson index and its variance in Spark/Scala. We also provide over a year's worth of learnings -- gathered by deploying our method at scale and analyzing thousands of experiments -- on which areas and which kinds of product innovations seem to inherently foster fairness through inclusiveness.
appendix boundary found by appendix_command · 80% of the source is main text. Read the extracted text to check this.
The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.
| Reference | Intensity | Mentions | Sections | Main text | |
|---|---|---|---|---|---|
| 1 | Anthony Atkinson (1970) On the measurement of inequality | 0.511 | 2 | 1 | 100% |
| * | unmatched citation key * | 0.405 | 1 | 1 | 100% |
| 3 | Bennett Cyphers, Gennie Gebhart, and Adam Schwartz (2019) Data Privacy Scandals and Public Policy Picking Up Speed: 2018 in Review | Electronic Frontier Foundation, 2019 | 0.405 | 1 | 1 | 100% |
| 4 | Karen Hao (2019) Facebook's ad-serving algorithm discriminates by gender and race - MIT Technology Review | 0.405 | 1 | 1 | 100% |
| 5 | Molly Schuetz (2019) How the Algorithms Running Your Life Are Biased, 2019 | 0.405 | 1 | 1 | 100% |
| 6 | Alekh Agarwal, Alina Beygelzimer, Miroslav Dudḱ, John Langford, and… (2018) A reductions approach to fair classification | 0.405 | 1 | 1 | 100% |
| 7 | Anthony B Atkinson and Andrea Brandolini (2010) On analyzing the world distribution of income | 0.405 | 1 | 1 | 100% |
| 8 | Solon Barocas, Moritz Hardt, and Arvind Narayanan (2017) Fairness in machine learning | 0.405 | 1 | 1 | 100% |
| 9 | Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, an… (2016) Man is to computer programmer as woman is to homemaker? debiasing word embeddings | 0.405 | 1 | 1 | 100% |
| 10 | John Creedy (2016) Interpreting inequality measures and changes in inequality | 0.405 | 1 | 1 | 100% |
Showing the top 10 of 19 scored citations. 1 of these could not be matched to a bibliography entry, so only the citation key is shown.
arXiv econ.EM papers that cite this one, ranked by how heavily they lean on it.
| Citing paper | Intensity | Mentions | Sections | |
|---|---|---|---|---|
| 1 | An Outcome Test of Discrimination for Ranked Lists | 0.405 | 1 | 1 |