FAI: End-To-End Fairness for Algorithm-in-the-Loop Decision Making in the Public Sector

The goal of this project is to develop methods and tools that assist public sector organizations with fair and equitable policy interventions. In areas such as housing and criminal justice, critical decisions that impact lives, families, and communities are made by a variety of actors, including city officials, police, and court judges. In these high-stakes contexts, human decision makers’ implicit biases can lead to disparities in outcomes across racial, gender, and socioeconomic lines.

While artificial intelligence (AI) offers great promise for identifying and potentially correcting these sorts of biases, a rapidly growing literature has shown that automated decision tools can also worsen existing disparities or create new biases. To help bridge this gap between the promise and practice of AI, the interdisciplinary team of investigators will develop an integrated framework and new methodological approaches to support fair and equitable decision-making.

This framework is motivated by three main ideas: (1) identifying and mitigating the impacts of biases on downstream decisions and their impacts, instead of simply measuring biases in data and in predictive models; (2) enabling the combination of an algorithmic decision support tool and a human decision-maker to make fairer and more equitable decisions than either human or algorithm alone; and (3) developing operational definitions of fairness and quantitative assessments of bias, guided by stakeholder discussions, that are directly relevant and applicable to the housing and criminal justice domains. The ultimate impact of this work is to advance social justice for those who live in cities, and who rely on city services or are involved with the justice system, by assessing and mitigating biases in decision-making processes and reducing disparities.

The project team will address both the risks and the benefits of algorithmic decision-making through transformative technical contributions. First, they will develop a new, pipelined conceptualization of fairness consisting of seven distinct stages: data, models, predictions, recommendations, decisions, impacts, and outcomes. This end-to-end fairness pipeline will account for multiple sources of bias, model how biases propagate through the pipeline to result in inequitable outcomes and assess sensitivity to unmeasured biases.

Second, they will build a general methodological framework for identifying and correcting biases at each stage of this pipeline, assessing intersectional and contextual biases across multiple data dimensions, and incorporating new ideas for model assessment and analysis of heterogeneous treatment effects. This generalized bias scan will provide essential information throughout the end-to-end fairness pipeline, informing not only what human and algorithmic biases exist, but what interventions are likely to mitigate these biases.

Third, the project addresses algorithm-in-the-loop decision processes, in which an algorithmic decision support tool provides recommendations to a human decision-maker. The investigators will develop approaches for modeling systematic biases in human decisions, identifying possible explanatory factors for those biases, and optimizing individualized algorithmic "nudges" to guide human decisions toward fairness.

Finally, the project team will create new metrics for measuring the presence and extent of bias. The outputs of the project will be designed for integration into the operational decision-making of city agencies responsible for making fair and equitable decisions in the criminal justice and housing domains. The investigators will assess the fairness of existing practices and create open-source tools for assessing and correcting biases, for users in each domain.

They will develop tools which can be used to (a) reduce incarceration by equitably providing supportive interventions to justice-involved populations; (b) prioritize housing inspections and repairs; (c) assess and improve the fairness of civil and criminal court proceedings; and (d) analyze the disparate health impacts of adverse environmental exposures, including poor-quality housing and aggressive, unfair policing practices. Operational deployments of the developed tools will be regularly and comprehensively evaluated to assess impacts and to avoid unintended consequences, both maximizing the benefits and minimizing potential harms from both algorithmic and human decisions.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.