Collaborative Research: Robust Deep Learning in Real Physical Space: Generalization, Scalability, and Credibility

The vulnerability of deep neural networks to small and imperceptible perturbations is a major challenge in machine learning today. For a variety of applications such as autonomous vehicles, security, and medical diagnosis, this weakness has severely limited the deployment of machine learning systems at scale. Existing theoretical studies, while laying a good foundation based on advanced statistical analyses, require various idealistic assumptions that are difficult to be validated in real physical environments.

Understanding the robustness of deep learning algorithms and its interactions with the real physical environment is therefore a critical step towards a better understanding of explainability, generalization, and trustworthiness. This project aims to close the gap by developing new theories and computer vision systems that can be realistically validated.

The outcomes of the research will create new technologies that can be translated into more secure and reliable commercial products, hence strengthening the global competitiveness of the United States; new trustworthy AI systems that can be deployed for surveillance and defense products to improve the national security of the United States; expand the next-generation workforce capacity by developing a complete training pipeline from K-12 outreach to undergraduate research, graduate mentoring, industry partnership, online learning modules, and curriculum development; broaden participation in STEM by leveraging the accessibility and intrigue of the foundational research concepts to conduct educational outreach that targets female participants from elementary up through graduate school; and promote the exchanges of ideas across disciplines in statistics, theoretical computer science, and image processing. 

Robust machine learning in real physical space requires co-modeling the deep neural networks and the environment in which the neural networks are operating. Research efforts focusing on one specific domain but not interacting with the other domain will unlikely solve the problem. The combination of skills in electrical engineering, statistics, and computer science possessed by the Purdue-UCSD team offers a unique opportunity to address the problem.

The technical approach the team will take is to reformulate the robust adversarial learning problem by incorporating the environmental factors. Four specific research objectives will be pursued: (1) Parametrizing the physical environment via a hierarchy of deterministic and generative approaches, so that the set of all possible distortions can be constrained. (2) Analyzing the generalization bounds of neural networks in the presence of the environmental factors and analyzing the credibility of such a system by studying the robustness and uncertainty quantification. (3) Developing computationally efficient algorithms to seek the equilibrium points of a proposed minimax optimization. (4) Building a computational photography testbed to implement the concepts and validate the theoretical results.

On the educational front, the project provides a suite of outreach activities to K-12 to improve their interest in STEM, and research opportunities to undergraduates.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.