Prediction and Coordination for Conversational AI

Being able to interact with machines through spoken language, in the same fluent and intuitive way we communicate with each other, has been a long-standing vision in both science fiction and research labs.

In recent years, Conversational AI has started to become a reality, partly thanks to the recent breakthroughs in deep learning.

However, a central limitation in current systems is that they process information and behave in a purely reactive manner.

Looking at human-human interaction, there is a large body of evidence which suggests that humans continuously predict (i.e. anticipate) each other’s actions, in order to achieve coordination.

The first objective in this project is to develop a novel computational model, based on deep learning, for continuous prediction of human communicative behaviour on multiple timescales in spoken interaction, including speech activity, prosody, attention and dialogue acts.

The second objective is to show how such a model can be analysed in order to get a better understanding of, and inform theories on, what signals humans use for prediction and coordination.

The third objective is to develop a model of coordination in dialogue, which allows conversational agents to anticipate human behaviour, simulate potential future scenarios, and adapt accordingly.

The project is of a highly interdisciplinary nature, and will break new scientific ground in computational linguistics, linguistics, speech technology, and artificial intelligence.