Safe Data-driven Control with Formal Guarantees

The theory of Mean Field Games (MFGs), originated in the research of Huang, Caines, and Malhamé on stochastic dynamic games and independently by Lasry and Lions as an extension of mean field theory in physics and statistical mechanics [1, 2], has opened new avenues in understanding complex multi-agent interactions. MFGs provide a framework for studying the asymptotic behaviour of large populations of agents where each agent's strategy depends on the average behaviour of others rather than individual interactions.

Recently, MFGs have received immense attention in the learning community for their ability to provide a solid theoretical underpinning to learning in multi-agent systems, and specifically in reinforcement learning [3-5].

The primary objective of this project is to develop an advanced theoretical framework for finding and analysing mean field equilibria in environments with a variety of constraints. Specifically, we aim to explore settings in which agent interactions are structured by dense graphs (graphon MFGs) or organized into coalitions, a context often referred to as mean field type games [6-8].

By investigating these structured interactions, we aim to extend the conventional MFG framework to more accurately reflect real-world systems where agents often interact within specific networks or groups rather than as a homogeneous population.

Our research will address several key challenges in this domain. Despite substantial progress, many open questions remain in MFG theory, such as generalizing the framework to continuous state spaces in discrete-time settings and extending it to incorporate deep learning techniques for scalability and flexibility [9]. Addressing these challenges requires not only a rigorous theoretical analysis but also the development of novel methods that can be implemented in practical scenarios.

For instance, establishing machine-checkable convergence proofs for various learning architectures, including deep learning models, will ensure the reliability and adaptability of MFG solutions in complex cooperative, competitive, and mixed interaction environments [10].

The motivation behind this research is driven by the need to develop reliable, scalable, and verifiable frameworks to capture the high-dimensional, dynamic, and potentially adversarial nature of complex systems with many agents. By advancing the theory of MFGs, particularly in constrained settings, our work will provide foundational insights applicable to a range of fields, including economics, finance, biology, robotics, and artificial intelligence.

Through our framework, we expect to enable more effective learning strategies in multi-agent environments, advancing both the theory and practical implementation of mean field equilibria solutions in a way that meets the demands of contemporary AI applications.

This project will undergo three main stages: (1) formalising the mean field equilibrium solutions for constrained interactions, particularly in the context of structured or unstructured settings; (2) extending these solutions to systems with time-dependent payoff structures; and (3) developing machine-checkable proofs of convergence for learning architectures.