Cooperation and Competition in Multi-Agent Reinforcement Learning

Context

Multi-agent reinforcement learning is a widespread framework for modelling multiple agents that learn to solve problems while interacting with each other, in a given environment. They are relevant in a variety of applications including autonomous driving, resource allocation, robotic swarms, resource allocation, energy management and multi-player games.

As AI agents increasingly work alongisde humans and other AI, it is essential for their safe deployment that we understand how they can effectively cooperate & find consensus despite competitive incentives & environments. Aims and Objectives

The primary aim of this research project is to understand the dynamics of cooperation and competition in multi-agent settings, which could lead to more robust, efficient and adaptable AI systems capable of negotiating complex environments, including decentralised systems. More specifically, our objectives are to:

1.Using tools from game theory, develop a mathematical understanding of how we can design agents that are capable of predicting, influencing, cooperating, and defending against other AI agents, especially adversarial or malicious ones.

2.On the flip-side, develop theoretically grounded methods to design games (environments) that incentivise cooperation in themselves, even in a decentralised setting where we lack control over other agents.

3.Building on this, develop novel algorithms that enable agents to dynamically switch between cooperative and competitive behaviors, as well as influence each other in order to incentivise consensus, depending on the behaviour of opponents.

4.Test in real-world scenarios by applying these algorithms to tasks such as resource allocation, multi-robot coordination, and negotiation games, to evaluate performance under real-world constraints.

5.Examine long-term behaviors such as trust-building, alliance formation, and betrayal in repeated interactions among agents.

6.Develop technical metrics to assess the safety of these AI systems - which is imperative for reducing AI risk and improving governance. Such metrics might take the form of physical requirements including the capacity to interrupt an AI agent, mechanistic interpretability of cooperative incentive, formal verification, or behavioural requirements based on stress-testing in closed and virtual environments, followed with open environments of increasing size.

Novelty of the Research Methodology

The novelty of our methodology lies in the game-theoretical exploration of cooperation and competition in a unified reinforcement learning framework, allowing agents to fluidly shift between cooperative and competitive modes based on their interactions and the evolving dynamics of the environment. The research methodology will leverage:

1.Game-theoretic concepts such as Nash equilibria, stable fixed points and Pareto optimality to inform the development of novel algorithms.

2.Meta-learning methods to allow agents to quickly adapt to new types of opponents or teammates, enhancing their ability to generalize across different tasks and environments.

3.Hierarchical reinforcement learning techniques to enable agents to plan and act at different levels of abstraction, facilitating more sophisticated strategic thinking in complex environments. This project falls within the EPSRC Artificial intelligence technologies research area.