CAREER: Towards a Principled Framework for Resilient, Data Efficient and Scalable Reinforcement Learning for Control

The success of the traditional control system design depends crucially on the availability of tractable system models with known parameters, well-understood sources of uncertainty and clearly specified objectives. These assumptions are no longer true in the emerging paradigm of intelligent and autonomous large-scale engineering systems, such as next generation electricity systems.

A data-driven machine learning approach that takes advantage of large amounts of data coming from such systems can provide a promising path forward. However, unlike the remarkable successes of machine learning in classification problems such as image recognition, reinforcement learning (RL) that addresses the problem of “learning to control” has seen achievements limited to more structured or simulated environments, and its successes in real-world engineering systems are not as prominent.

There are three critical issues that significantly impede the success of RL in real-world engineering systems: lack of resiliency, data efficiency, and scalability. This CAREER proposal develops a principled approach for the RL-based design of control algorithms for large-scale real-world engineering systems, by overcoming the fundamental challenges of resiliency, data efficiency, and scalability.

The main application domain of interest is electricity systems, which guides the problem formulation and solution approaches, and lends credence to the algorithms using real-world examples.

The project has an innovative education plan that includes the ‘Aggie DeepRacer Project’ that follows an ‘experiential learning’ approach for integrating the research in RL into the educational curriculum. Hosting teachers from low socioeconomic status schools strengthens the project. Project outcomes include development of activity-based learning modules for high school students and presenting them at the Aggie STEM summer camp and Physics and Engineering Festival.

By developing a data-driven and learning-based approach for efficient control of power systems, this project also contributes to reducing the cost of fuel and operations, and hence significantly increasing the reliability of the overall energy system.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.