EconBase
← All papers

An Outcome Test of Discrimination for Ranked Lists

Jonathan Roth, Guillaume Saint-Jacques, YinYin Yu

arXiv 15 Nov 2021 · Econometrics · published2022 ACM Conference on Fairness, Accountability, and Transparency (2022) · 1 citations (OpenAlex)

arXiv:2111.07889 · PDF · DOI · OpenAlex · Extracted main text

Abstract

This paper extends Becker (1957)'s outcome test of discrimination to settings where a (human or algorithmic) decision-maker produces a ranked list of candidates. Ranked lists are particularly relevant in the context of online platforms that produce search results or feeds, and also arise when human decisionmakers express ordinal preferences over a list of candidates. We show that non-discrimination implies a system of moment inequalities, which intuitively impose that one cannot permute the position of a lower-ranked candidate from one group with a higher-ranked candidate from a second group and systematically improve the objective. Moreover, we show that that these moment inequalities are the only testable implications of non-discrimination when the auditor observes only outcomes and group membership by rank. We show how to statistically test the implied inequalities, and validate our approach in an application using data from LinkedIn.

Citation extraction

20
references
28
in-text mentions
20
distinct cited
1
self-citations
5,731
main-text words

appendix boundary found by none_found · 100% of the source is main text. Read the extracted text to check this.

Most heavily cited references

The works this paper leans on most, across its whole bibliography — not restricted to papers in our corpus. Ranked by composite intensity, which combines how often a work is mentioned, how many sections mention it, and how much of that falls in the main text rather than the appendix.

ReferenceIntensityMentionsSectionsMain text
1Becker (1957)0.92843100%
2Canay and Shaikh (2017) Practical and theoretical advances in inference for partially identified models0.64422100%
3Molinari (2020) Chapter 5 - Microeconometrics with partial identification0.64422100%
4Castillo and Petrie (2010) Discrimination in the lab: Does information trump appearance?0.58531100%
5Singh and Joachims (2018) Fairness of Exposure in Rankings0.51121100%
6Andrews and Soares (2010) Inference for Parameters Defined by Moment Inequalities Using Generalized Moment Selection0.40511100%
7Andrews and Shi (2013) Inference Based on Conditional Moment Inequalities0.40511100%
8Beutel, Chen, Doshi, Qian, Wei, Wu, Heldt, Zhao, Hong, Chi and Goodrow (2019) Fairness in Recommendation Ranking through Pairwise Comparisons0.40511100%
9Celis, Straszak and Vishnoi (2017) Ranking with Fairness Constraints0.40511100%
10Corbett-Davies and Goel (2018) The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning0.40511100%

Showing the top 10 of 20 scored citations.