Collaborative Research: SaTC: CORE: Small: Privacy and Fairness in Critical Decision Making

Many agencies or companies release statistics about groups of individuals that are then used as input to critical decision processes. For example, census data is used to allocate funds and distribute critical resources to states and jurisdictions. The resulting decisions can have significant societal and economic impacts for participating individuals.

In many cases, the released data contain sensitive information whose privacy is strictly regulated and Differential Privacy (DP) has become the paradigm of choice for protecting data privacy. However, while differential privacy provides strong privacy guarantees on the released data, it has become apparent recently that it may induce biases and fairness issues in downstream decision processes, including the allotment of federal funds, apportionment of congressional seats, and distribution of vaccines and therapeutics.

These biases and fairness issues may adversely affect the health, well-being, and sense of belonging of many individuals, and are poorly understood. This project addresses this knowledge gap at the intersection of privacy, fairness, bias, and decision processes. It will offer novel perspectives on differential privacy tools to address fairness and privacy jointly in critical decision processes.

It will quantify the disparate impact arising in these applications and contribute novel mechanisms and mitigation techniques to overcome some of these issues. These contributions will be embedded in modeling and software tools to make the technology widely available and applicable.

From a scientific standpoint, this project will develop a new generation of privacy-preserving tools that, by exploiting knowledge from differential privacy, optimization, and programming languages, will address biases and fairness issues in their designs, not as an afterthought. The project contributes new scientific knowledge along with five directions: (1) it identifies and understands the structure of downstream decision processes that may be subject to fairness issues when using DP data releases; (2) it identifies and understands the structure of DP mechanisms that may introduce biases; (3) it defines theoretical frameworks to characterize and reason about biases and fairness issues; (4) it designs mitigation measures that would remove or alleviate the biases and fairness issues, finding an appropriate tradeoff between privacy, accuracy, and fairness; (5) it develops modeling and software tools to automatically identify and explain biases and fairness issues, and derive mitigation measures from the specification of the decision process.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.