Effective and Semantic Communication in Multi-Agent Reinforcement Learning

My project focuses on designing goal-oriented communication frameworks and systems using machine learning optimisation techniques. The communication problem can be divided into 3 levels: 1. Technical problem: How accurately can the symbols of communication be transmitted? 2. Semantic problem: How precisely do the transmitted symbols convey the desired meaning?

3. Effectiveness problem: How effectively does the received meaning affect conduct in the desired way?

The leading prevailing paradigm is the technical problem perspective. Consider streaming visual media or guiding a remote-controlled rover. Current approaches to communications consider a layered strategy.

First, the message (for instance video frame or a rover command) is mapped to a bit pattern to remove redundancy. Next, the compressed bit pattern is protected against channel distortion with an error-correcting code. Finally, the protected bits are mapped to channel symbols for transmission.

After the transmission, the process is reversed at the receiver. Each step of this process has been extensively researched in the last 80-years. This method has a significant advantage - it allows for much simpler analysis at each stage. The problem is broken into subproblems that are more manageable to tackle.

This approach is optimal for specific sources and long enough block codes - it cannot be beaten asymptotically. However, in general, this approach is not flawless. Communication is rarely the goal in itself.

Instead, it is used to achieve some other end. Thus, the success of communication should be measured in the context of the overall objective. That is the main focus of my project - semantic and effective communication problems.

The modular system fails to exploit the interactions, dependencies, and correlations between the steps. However, each of these stages is complex by itself. Thus, doing away with the modular approach is not feasible in a straightforward analytic manner.

That is why the current advancements in statistical methods such as machine learning are especially promising. The ability to learn inductively from examples allows for joining the modules of communication into a single system. However, what constitutes the desired behaviour is not always apparent. Thus, another framework is introduced.

Reinforcement learning is the set of algorithms that allows for learning complex behaviours through interactions with an environment. By introducing realistic communication channels, we can extend those methods to allow for learning of the communication schemes themselves jointly with the desired conduct. This framework can be extended to include multiple agents interacting and learning.

This study is relevant to remote control problems, drone swarm navigation, coordinated autonomous vehicle driving, distributed learning, or industrial internet of things where the number of independent actors need to coordinate to achieve a common goal.

Relevant EPSRC research areas: Artificial intelligence technologies, ICT networks and distributed systems, Digital signal processing, Statistics and applied probability.