Beyond solving static datasets: Deep learning from streaming data

Data is key for modern AI solutions, especially deep learning.

Unfortunately, the data-driven nature of deep learning that makes it so powerful when dealing with complex and high-dimensional data, is also at the core of its main weakness: a model is only as good as the data it builds on.

In this project, we want to tackle some strong limitations inherent to the standard machine learning paradigm, which makes restrictive assumptions that are problematic in many real-world (“in the wild”) conditions.

By addressing these, we want to make a fundamental step towards more powerful deep learning systems that can learn continuously and know how to adapt as new data becomes available, in the context of computer vision.Traditional deep learning relies on the training data being representative for data encountered during system deployment.

This is perfect when working with stationary datasets. Yet in practice, data distributions are often non-stationary, i.e., they change over time. This can have a multitude of reasons – think of social trends, seasonal or geographic variations.

This calls for a new generation of deep learning methods, able to adapt to new conditions by continuously updating the models based on new training data becoming available. Learning from non-stationary streaming data is, however, still a major challenge requiring fundamental research.

In this project, we build on our earlier expertise in continual learning, to realize this ambitious goal.If successful, this will lead to machine learning systems that keep on learning over time, systematically improving their skills and never getting outdated.

It also may lower the threshold for applying machine learning, as it reduces the need for a skilled data scientist carefully preparing the data beforehand.

As a practical application, we plan to showcase our work’s feasibility, scalability and flexibility in the context of automatic generation of audio descriptions of videos for the visually impaired.