AI-DCL: EAGER: Explanations through Diverse, Feasible, and Interactive Counterfactuals

This award supports a research project that will help people to better understand decision algorithms that are developed using machine learning techniques. The research team will facilitate that understanding by making use of a promising class of explanations that use counterfactual scenarios. Such explanations provide understanding by showing how outcomes change when hypothetical changes are made in factors that together serve to determine the decision outcome.

As a concrete example, consider a person who applies for a loan from a financial company but is rejected by the loan distribution algorithm used by the company. To help the person understand why the decision algorithm rejected the application, the explanation algorithm would generate counterfactual scenarios in which the applicant's situation is hypothetically changed in viable ways (such as moving to a nearby city, or changing jobs) to see whether this affects the decision outcome.

If this approach is successful, it would be applicable to a variety of societally critical domains where machine learning holds promise for improving decision making including healthcare, criminal justice, finance, and hiring. The project will have other impacts as well. The research team will release a public web site to engage the public with human-centered machine learning approaches.

The PI will work with the University of Colorado Boulder's Science Discovery to present demos at events such as "Family Engineering Day" and "Boulder Computer Science Week". In addition to training graduate students, the PI will host high-school students as summer interns, integrate findings from the proposed work into educational activities at the University of Colorado Boulder, and make educational materials publicly available for use by instructors at other institutions.

This research project seeks to explain machine decisions by generating diverse and feasible counterfactuals and developing user-centered interactive processes. The results of this project will constitute an important step towards building machine-in-the-loop methods to empower users in understanding algorithmic decisions. Specific contributions include developing diversity and distance metrics for generating diverse counterfactuals, integrating causal graphs to generate feasible counterfactuals that align with real-world processes, developing novel user-centered designs to examine human interaction with counterfactuals, and advancing design principles for explaining algorithmic decisions.

The team will also develop human-centered designs that enable users to interact with counterfactual explanations. This will enable the researchers to conduct large-scale user studies to understand human preferences, which would in turn serve as an effective evaluation of their proposed method. The results of this research project will contribute to the emerging area of interpretable machine learning that emphasizes human-centered designs.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.