CAREER: New Frontiers in Bayesian Deep Learning

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).

Computers assist in safety-critical settings, with many moving parts and changing environments — driving vehicles in new traffic environments, classifying medical images acquired with different resolutions, manipulating robotic systems over rugged terrain, and designing life-saving pharmaceuticals. Such tasks require a careful representation of uncertainty so that we can protect ourselves against rare but costly mistakes and detect if we are operating outside of standard parameters.

We can further improve performance if we can incorporate prior knowledge into how we represent uncertainty. For example, we may believe that the label of an image should not change if the image is rotated or translated. Incorporating this knowledge enables efficient learning from a small amount of information, as well as greater accuracy and reliability, often making the difference between which problems can be solved and which cannot.

This research makes it possible to represent uncertainty in sophisticated models that learn from data, while specifying detailed prior knowledge, providing particular resilience to changing environments. It also significantly reduces the computations needed for a robust uncertainty representation, leading to computer models that can quantify uncertainty more quickly and reliably, at lower cost.

With systems that can encode uncertainty and efficiently transfer knowledge to new environments, we facilitate accurate medical diagnoses, reliable infrastructure, and life-saving scientific discoveries. The scientific innovations we develop through this work will be included in popular educational textbooks, and used as the basis for projects in outreach initiatives.

The goal of this research is to have Bayesian deep learning guide the research trajectory of the deep learning community at large, with principled approaches to foundational questions, such as how to manage the trade-off between inductive biases and flexibility. This goal has implications for essentially any predictive task, improving accuracy and robustness, while providing reliable uncertainty estimates for decision making.

There are three key parts to this research, which form a natural synergy. The first thrust pursues new priors that combine flexibility with useful inductive biases, and provide good out-of-distribution generalization. These priors provide a mechanism for encoding high level concepts into prior distributions, such as locality, independencies, and symmetries, without constraining model flexibility.

To realize the benefits of these priors, the next thrust focuses on accurate approximate inference procedures, inspired by Hamiltonian Monte Carlo and deep ensembling. The last thrust pursues applications in medical imaging, autonomous driving, and protein design. This award pursues initiatives that are of broad interest to society, including scientific and public policy applications, and outreach initiatives that include school teaching, open educational resources, major open source libraries, textbook writing, undergraduate internships, summer schools, symposia, and collaborations with local educational institutions.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.