CAREER: Towards Reliable Machine Learning in Feedback Systems

Machine learning has made significant advancements in tasks like image recognition and click prediction, enabling applications such as self-driving cars and personalized social media feeds. However, when machine learning predictions are deployed to make decisions, small errors can have unintended consequences. In feedback loops, these errors can compound, leading to problems in safety, bias, and performance.

This project aims to address this challenge by developing algorithms for reliable decision-making in complex systems, ensuring that the benefits of machine learning are realized while minimizing its risks. By understanding how feedback loops work, we can unlock the full potential of machine learning to improve outcomes in areas like weather prediction, recommendation systems, and human-robot collaboration.

Our research program will develop innovative solutions that benefit society as a whole, with a focus on developing theory and algorithms that have direct impact on these applications. In addition, an integrated education plan will promote computer science education and inspire the next generation of innovators. Hands-on projects and interactive modules will introduce high school students to exciting topics like weather forecasting, balloon control, and robotics.

The project will leverage ideas and techniques from online optimization, control theory, system identification, and reinforcement learning to answer the following questions: How should decision algorithms make use of possibly unreliable machine learning predictions while ensuring good outcomes? How can we reliably predict the long-term impacts of decisions with models learned from temporally correlated data?

How do we adaptively make decisions while learning about initially unknown impacts? The algorithmic and theoretical frameworks will be developed in tandem with applications in weather prediction, autonomous aerial navigation, recommendation systems, and human-robot interaction, along the following three thrusts. First, consider decision-making with ML predictions.

Reliably leveraging unreliable predictions requires accounting for potential errors to guard against bad outcomes. The researchers will first develop algorithms that robustly guarantee performance and safety while benefiting from predictions when they are accurate and then use decision performance as a metric to evaluate prediction quality. Second, consider learning models of impacts.

Understanding the long-term impacts of decisions on individuals is crucial, and yet human activities are non-stationary, correlated, and partially observed. The researchers will develop reliable learning algorithms for data arising from such processes, with a particular focus on finite sample uncertainty quantification and bounded sample complexity.

Third, consider sample-efficient reinforcement learning: Adaptive decision-making requires simultaneously learning from data while making decisions. The researchers will develop model-based algorithms which can operate even in partially observed settings, such as the non-stationary user behaviors important for applications like recommendation and human-robot collaboration.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.