New Studies of Learning with Stochastic Convex Optimization

The paradigm of learning from data is playing an increasingly important role in science and engineering. The interplay between machine learning and mathematical optimization has been most fruitful, and one prominent area is stochastic convex optimization (SCO). However, there is relatively little work on the fundamental questions such as generalization and stability analysis, and the existing studies often focus on the standard classification and regression with smooth losses.

Furthermore, data collected and used for the learning often contains sensitive information such as financial records from fraud detection or genomic data from cancer diagnosis which presents an urgent need to develop privacy-preserving SCO algorithms with theoretical guarantees. These provide motivation for the project which aims to study the fundamental properties of machine learning inspired SCO algorithms including their stability, generalization, and differential privacy. Students will be involved and trained in interdisciplinary aspects.

The technical objectives of the proposed work are divided into three thrusts. The first thrust focuses on the study of stability and generalization of stochastic gradient methods (SGM) for solving SCO problems associated with non-smooth losses. The second thrust is to develop and study SGM algorithms for SCO problems which can prevent the privacy leakage using a well-accepted mathematical definition of privacy called differential privacy.

The third thrust is to study the stability, generalization, and differential privacy of SCO algorithms for pairwise learning which involves more complex losses than the standard classification and regression.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.