CDS&E: Robust Symmetry-Preserving Machine Learning: Theory and Application

Deep neural networks (DNNs) have been a major driving force behind recent advances in data science and engineering. An emerging theme in DNN research is to exploit the intrinsic structure of the learning problems, such as symmetry, to improve the data-efficiency of DNNs in the small-data regime. Recent work on symmetry-preserving machine learning typically studies it in the ideal setting where the symmetry transformations are perfect, whereas in reality, however, they are usually “contaminated” by various sources of signal deformation.

The aim of this project is to rigorously measure and guarantee the deformation robustness of general symmetry-preserving DNNs, as well as quantifying their resulting performance gain. Results of the research are expected to advance understanding of robust geometric deep learning, with a diverse range of applications from computer vision to scientific computing with limited data.

The project will provide interdisciplinary training in applied mathematics, engineering, and data science to undergraduate and graduate students.

The overarching theme of the project is to leverage mathematical tools from differential geometry, applied harmonic analysis, and applied probability to improve the statistical-efficiency of machine learning models. Special emphasis has been placed on the rigorous analysis and promotion of robust symmetry-preservation that is broadly applicable to arbitrary Lie group representations on general feature fields.

In addition, the project aims to extend the idea of symmetry-preservation to deep distribution learning, and proposes a unified framework for data-efficient generation of distributions with intrinsic structures including—but not limited to—group symmetry; the improved statistical efficiency will be rigorously quantified through sample complexity analysis. The techniques to be developed in this project will be widely applicable across different disciplines, providing fundamental building blocks for the next generation of mathematical tools for the computational and geometric modeling of Big Data.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.