SaTC: CORE: Medium: Collaborative: Using Machine Learning to Build More Resilient and Transparent Computer Systems

Machine learning algorithms are increasingly part of everyday life: they help power the ads that we see while browsing the web, self-driving aids in modern cars, and even weather prediction and critical infrastructure. We rely on these algorithms in part because they perform better than alternatives and they can be easy to customize to new applications.

Many machine learning algorithms also have a big weakness: it is difficult to understand how and why they compute the answers they provide. This opaqueness means that the answers we get from a machine learning algorithm could be subtly biased or even completely wrong, and yet we might not realize it. This project's goal is to make machine learning algorithms easier to understand, as well as to leverage some of the techniques used by attackers to trick machine learning algorithms into making mistakes to build computer systems that are more resistant to attack.

In addition to making fundamental contributions to how machine learning algorithms are designed and used, the project includes outreach efforts that will entice students to gain hands-on experience with machine learning tools.

This project focuses on deep neural networks (DNNs). A groundswell of research within the past five years has demonstrated the propensity of these models to being evaded by inputs created to fool them -- so called "adversarial examples." These types of attacks leverage DNNs' opacity: while DNNs can perform remarkably well on some classification tasks, they often defy simple explanations of how they do so, and indeed can leverage features for doing so that humans might find surprising.

This project leverages DNNs and the attacks against them to gain insights into how to build more resilient computer systems. Specifically, the project will use DNNs to model adversaries trying to attack computer systems and then "attack" these DNNs to learn how to improve these systems' resilience to attack. This modeling will be done using Generative Adversarial Nets (GANs), in which "generator" and "discriminator" models compete.

Central to this vision are the abilities to evade DNNs under constraints and to extract explanations from them about how they perform classification. Consequently, this project will make fundamental advances both in developing better methods to deceive DNNs and in improving this important machine-learning tool.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.