RI: Medium: Learning MDP abstractions for Autonomous Systems using Variational Methods and Symmetry Groups

Autonomous systems such as self-driving vehicles, hospital platforms, and household robots have great potential social and economic benefits, with the ability to transform the future of work, healthcare, and our daily routines. However, successful autonomy requires the robot to be able to make its own decisions and learn from its own experiences. This can be challenging because the real world is rich and complex and autonomous robotic systems can become confused by the details.

It is sometimes the case that an autonomous system will not generalize properly: it will perceive two very similar situations to be fundamentally different. This project aims to develop new methods for learning task- and domain-appropriate abstractions that will help autonomous systems generalize to new situations more effectively. Better abstraction will allow autonomous systems to make decisions more efficiently leading to improved learning and effective control.

This project will study the problem of abstraction within the decision-theoretic framework of Markov decision processes and reinforcement learning, which have been widely used as a framework for automated decision making. Recent advances in reinforcement learning have enabled autonomous agents and robots to accomplish challenging tasks, sometimes even surpassing human experts.

However, this comes at an extremely high cost, both in sample and computational complexity; millions of training steps and days of training time are typical, even in game-like environments. This project will develop approaches for making this process much more efficient, by explicitly encoding objectives for learning good abstractions into the agent's cost function.

Specifically, the PIs will study and develop approaches for compressing large continuous decision-making problems into small discrete ones, as well as approaches that incorporate explicit symmetry constraints that encode irrelevances in the problem. These methods will be evaluated on a variety of domains of varying complexity, including tasks on autonomous systems involving mobile navigation and robot manipulation.

The overall objective is to develop approaches that improve learning efficiency, abstraction quality, and generalization to new tasks and situations.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.