Synthetic Environment Design for Reinforcement Learning

Brief description of the context of the research including the potential impact

Reinforcement learning (RL) has made significant strides over the last decade, achieving superhuman performance on a range of RL tasks. However, its success has largely been limited to simulated environments where it is possible to generate task experience indefinitely, an unrealistic assumption for real-world tasks with limited available experience and poor simulation quality.

One approach to improving performance in this domain is to learn a simulation of the environment, before training an agent on synthetic data generated from this simulation. Doing so would enable RL agents to be trained on significantly less experience and without a manually-programmed simulation, transferring these advancements to real-world tasks.

Aims and Objectives

We aim to develop various methods for generating synthetic task experience and training RL agents on synthetically generated data. These methods should allow for improvements in sample efficiency on a range of existing RL benchmarks and may enable novel applications of RL in new domains. Novelty of the research methodology

Training RL agents on synthetic experience is a little-studied topic, however, some recent works have demonstrated success from it. We are proposing a novel approach to generating synthetic experience, in which we will use insights from Unsupervised Environment Design to tailor the experience to the current capabilities of the agent, thereby improving training efficiency and asymptotic performance.

Alignment to EPSRC's strategies and research areas

This project falls under the EPSRC's Artificial Intelligence Technologies research area, within the Engineering and ICT themes. Specifically, this work will enable the deployment of RL-based systems to real-world applications such as robotics, by allowing agents to train on minimal task experience and without any need for manually-programmed simulation.