REinforcement TWInning SysTems: from collaborative digital twins to model-based reinforcement learning

The concept of digital twins promises to revolutionize engineering by offering new avenues for optimization, control, and predictive maintenance.

Digital twins seek to virtually replicate systems using models that continuously ""learn"" from data, automatizing data collection, validation, and refinement and becoming ""self-learning"" models.

However, this concept is not yet established in engineering and requires significant developments in integrating machine learning with traditional ""domain-specific"" knowledge. The Re-Twist project tackles this challenge with two objectives.

The first objective is to develop a new framework that puts fundamental principles at the core of digital twinning and combines the training of a digital twin with the training of a controlling agent in ways that allow one to learn from the other.

The agent learns by trial and error, as in reinforcement learning, while interacting with the system and using the digital twin as a playground. I call this novel framework Reinforcement Twinning (RT). The second objective is to develop RT on lab-scale prototypes of systems at the centre of global challenges.

These are the optimal operation of wind turbines, drone propellers, sloshing tanks, and cryogenic liquid storage.

Wind turbines drive the fastest-growing renewable energy sector; drones have the potential to revolutionize monitoring, inspection, rescue missions, swift delivery of medical supplies and more.

The optimal management of cryogenic tanks, controlling the dynamics of sloshing and the thermodynamics of boil-off, will be essential to the economic viability of green fuels such as liquid hydrogen.

This project is ""high risk"" because it endeavours to establish a new discipline at the intersection of machine learning and energy engineering.

It promises ""high gains"" by aiming to experimentally validate twinning systems that could significantly impact society.