Less Discriminatory Algorithms

Solon Barocas is a Principal Researcher in the New York City lab of Microsoft Research, where he is a member of the Fairness, Accountability, Transparency, and Ethics in AI (FATE) research group. He's also an Adjunct Assistant Professor in the Department of Information Science at Cornell University, where he’s part of the initiative on Artificial Intelligence, Policy, and Practice (AIPP). His research explores ethical and policy issues in artificial intelligence, particularly fairness in machine learning, methods for bringing accountability to automated decision-making, and the privacy implications of inference. He is co-author of the forthcoming textbook on "Fairness and Machine Learning: Limitations and Opportunities" and he co-founded the ACM conference on Fairness, Accountability, and Transparency (FAccT).

Talk: Less Discriminatory Algorithms

Attend this talk via Zoom

Abstract: Entities that use algorithmic systems in traditional civil rights domains like housing, employment, and credit should have a duty to search for and implement less discriminatory algorithms (LDAs). Why? Work in computer science has established that, contrary to conventional wisdom, for a given prediction problem there are almost always multiple possible models with equivalent performance—a phenomenon termed model multiplicity. Critically for our purposes, different models of equivalent performance can produce different predictions for the same individual, and, in aggregate, exhibit different levels of impacts across demographic groups. As a result, when an algorithmic system displays a disparate impact, model multiplicity suggests that developers may be able to discover an alternative model that performs equally well, but has less discriminatory impact. Indeed, the promise of model multiplicity is that an equally accurate, but less discriminatory alternative algorithm almost always exists. But without dedicated exploration, it is unlikely developers will discover potential LDAs.

Model multiplicity has profound ramifications for the legal response to discriminatory algorithms. Under disparate impact doctrine, it makes little sense to say that a given algorithmic system used by an employer, creditor, or housing provider is either “justified” or “necessary” if an equally accurate model that exhibits less disparate effect is available and possible to discover with reasonable effort. Indeed, the overarching purpose of our civil rights laws is to remove precisely these arbitrary barriers to full participation in the nation’s economic life, particularly for marginalized racial groups. As a result, the law should place a duty of a reasonable search for LDAs on entities that develop and deploy predictive models in covered civil rights domains. The law should recognize this duty in at least two specific ways. First, under disparate impact doctrine, a defendant’s burden of justifying a model with discriminatory effects should be recognized to include showing that it made a reasonable search for LDAs before implementing the model. Second, new regulatory frameworks for the governance of algorithms should include a requirement that entities search for and implement LDAs as part of the model building process.