CRII: CHS: Harnessing Machine Learning to Improve Human Decision Making: A Case Study on Deceptive Detection

Humans are the final decision-makers in a wide variety of critical tasks that involve ethical and legal concerns, ranging from predicting criminal recidivism by the courts, to medical diagnosis, to identifying misleading information. These are challenging tasks for humans and for machines. However, for some closely-constrained tasks where vast amounts of training data are available, machine learning algorithms can outperform humans.

If the knowledge encoded in the machine learning models can be elucidated to humans, these implementations can support human decision making and even tutor humans to achieve better performance. Those are the goals of this project.

This project investigates human decision making with assistance from machine learning models for the task of detecting deception. It explores two domains routinely encountered on the Internet, online reviews and news articles. It develops two forms of assistance from machine learning models to improve human decision making while retaining human agency: 1) providing information based on machine learning models for real-time support of human decisions, and 2) automatically generating tutorials to help humans understand the nature of this task from the perspective of machine learning models (offline training).

This project develops novel algorithms that incorporate educational psychology to help teach humans the knowledge encoded in machine learning algorithms. The project evaluates the two forms of assistance by tracking human performance improvement in user studies. The project explores additional indicators, such as trust and time to complete tasks, to further understand collaboration between humans and machine learning algorithms.

The knowledge gained in the project will inform design principles for effective integration of artificial intelligence into human decision making.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.