Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification

Joy Buolamwini and Timnit Gebru ask whether commercial facial-analysis accuracy holds across groups that common benchmarks represent unequally. They contribute both a new dataset and an intersectional audit: measuring gender and skin type together reveals disparities that an overall score—or either category alone—can conceal.^[1]

Building a benchmark that exposes differences

The authors first annotate existing benchmarks. Lighter-skinned people comprise 79.6% of the 500 IJB-A subjects and 86.2% of the 2,194 Adience subjects whose images could be labeled. These are descriptions of the evaluated datasets, not the companies’ undisclosed training data.^[1]

Their Pilot Parliaments Benchmark (PPB) contains 1,270 unique parliamentarians from Rwanda, Senegal, South Africa, Iceland, Finland and Sweden. Countries were selected for parliamentary gender representation and contrasting skin-type distributions. Official portraits provide identifiable public subjects and relatively controlled pose and illumination. The resulting benchmark is 44.6% female-labeled and 46.4% darker-skinned; it seeks better subgroup representation rather than matching a particular deployment population.^[1]

Three annotators, including the authors, supply labels; a dermatologist supplies definitive Fitzpatrick skin-type labels. The analysis groups types I–III as lighter and IV–VI as darker, partly to reduce off-by-one labeling errors from estimating skin type in photographs. These categories measure one visible characteristic, rather than identifying race or fully representing skin diversity.^[1]

What the commercial audit finds

The April–May 2017 evaluation tests Microsoft, Face++ and IBM APIs. It predicts a binary gender label from a face; it does not match the face to an identity. The researchers use names, titles and appearance to label people perceived as women or men. They explicitly acknowledge that this reduction does not capture gender’s complexity or transgender identities, and that the vendors do not clearly specify whether they intend to classify gender identity or biological sex.^[1]

Table 4 reports the following error percentages on PPB:^[1]

Classifier	All subjects	Darker female	Lighter male
Microsoft	6.3%	20.8%	0.0%
Face++	10.0%	34.5%	0.8%
IBM	12.1%	34.7%	0.3%

The darker-female group constitutes 21.3% of the benchmark but accounts for 61.0–72.4% of classification errors across the systems. Face++’s lowest-error group is actually darker males, at 0.7%. The finding is an intersectional pattern, rather than a rule that every darker-skinned person is classified less accurately than every lighter-skinned person.^[1]

The authors also examine South Africa, whose portraits have image quality comparable to the European pictures and a wider skin-type range than the other African subsets. Disparities persist within that country. They argue that differences in portrait production do not fully explain the result, while acknowledging that skin type may correlate with facial geometry or gender presentation absent from training data. This comparison does not isolate the causal effect of skin color.^[1]

What an accuracy claim leaves open

Only IBM exposes confidence scores. Even darker-female misclassifications can have high confidence, so a confident label need not be correct. The APIs do not expose the probability distributions and adjustable thresholds needed to explore alternative error-rate tradeoffs. Proprietary training data and limited method documentation also constrain explanations for the disparities.^[1]

These findings describe the tested historical versions, portraits and labels. They supply neither current product performance nor a police identification error rate. Compare NIST’s identity-recognition evaluation and the Williams incident for different tasks and evidence. Social Alignment additionally asks which uses are justified; improved classification alone cannot answer that question.^[1]

Historical context

The proceedings record specifies February 23–24, 2018; chronology uses the conference month, while the measurements concern April–May 2017.^[1]

Follow-up evidence

The 2019 follow-up, by Inioluwa Deborah Raji and Buolamwini, revisits the benchmark and disclosure process. The original contribution remains a method of making an aggregate claim inspectable: identify who enters the test, how labels are assigned, and where errors concentrate.