CIF: Small: Foundations and Applications of Blind Subgroup Robustness

Machine-learning algorithms may present discriminatory behavior across certain subgroups, meaning that segments of the overall population are measurably under-served by the model, rendering the decisions unfair. The most common approaches to address this challenge consider that the algorithm has access to a set of predefined protected subgroups during training, and the goal is to learn a model that satisfies a certain notion of fairness/robustness across these subgroups.

Perfect fairness can, in general, only be achieved by degrading the performance of the benefited subgroups without necessarily improving the disadvantaged and protected ones. This conflicts with ethical and legal notions of no-harm fairness, which are appropriate where quality of service is paramount, for example in health. To address this, this work considers notions of fairness and subgroup robustness that guarantee no unnecessary harm is done to any subgroup.

The project goes beyond this since it considers the case where the subgroups or demographics are not known a priori and might even change with time and algorithm deployment. The project brings these concepts of blind and no-harm subgroup robustness and fairness to the area of backwards compatibility, where the goal is to guarantee that new machine-learning algorithms are compatible with previous ones; and to the area of federated learning, where multiple sites share data for the sake of mutual benefit.

Lastly, potential connections of the proposed blind and no unnecessary-harm subgroup robustness with causal inference are investigated.

The project first formally studies blind and no-unnecessary-harm (Pareto optimal) subgroup robustness, where the machine-learning algorithm needs to be robust to all possible subgroups of the data (given a minimal subgroup size), without necessarily knowing in advance the subgroups' defining characteristics. This is formally studied, including the tradeoffs and costs of protecting unknown subgroups and the corresponding optimization algorithm; concepts of data and optimization uncertainty are also included to model potential sacrifices a subgroup can make in benefit of others.

Such formal study of blind subgroup robustness is an emerging field in the machine-learning community, and this project provides a fundamental and unifying view of it, combining theory with practice and critical information for policy makers. The project then extends the work to the area of backwards compatibility, with the goal to make all potential subgroups equally backwards compatible; and to federated learning, where the subgroup fairness and robustness is considered both across the silos/participants and inside each silo itself.

Finally, thanks to the close mathematical connection between invariant features and causality, the project further considers this proposed unifying framework of blind subgroup robustness to study connections between the automatically discovered critical subgroups, their features, and causality. Health applications provide a unique testbed for the frameworks developed here.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.