Collaborative Research: CIF: Small: Mathematical and Algorithmic Foundations of Multi-Task Learning

Reinforcement learning has emerged as one of the predominant frameworks for real-time decision making and control. It has been the driving force behind several recent high-profile successes of artificial intelligence, enjoying success in areas as diverse as robotic control, wireless communications, and protein structure prediction. While reinforcement learning provides a powerful and flexible framework for learning, data efficiency is a fundamental challenge: this framework is known to require significant computational resources and vast amount of data.

This challenge limits the applicability of reinforcement learning and keeps it from being applied in problems where training data and computational power are limited, including important applications such as wildfire monitoring and the search-and-rescue of lost people using unmanned aerial vehicles. This project addresses this challenge by developing new mathematical foundations of multi-task reinforcement learning and novel learning algorithms that require less data in the aggregate when multiple tasks are jointly learned.

The project integrates the research findings with rigorous educational and outreach activities, course development, and student training.

This project focuses on answering two fundamental questions: (1) Under what conditions does it take less data and computation to learn multiple tasks jointly than it would to learn each task individually? and (2) Can reinforcement learning algorithms learn something meaningful with only a limited amount of data and computation? Our approach to answering these questions draws on techniques from online learning, compressed sensing, and stochastic modeling.

In particular, this project covers both offline settings, where the similarity structure between tasks is learned from a given data set, and online settings, where this learned structure is used to efficiently adapt to a new task “on the fly”. The project also addresses the fundamental problem of catastrophic forgetting in multi-task learning, where the learned policy loses the ability to perform a previous task after training for a new task.

Over the course of this project, the proposed research activities will be evaluated systematically through a series of simulations of multi-robot navigation.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.