CASCADE – From Scene Flow to Data-Efficient Self-Supervised Dynamic Scene Understanding

Dynamic scene understanding is essential for the safe and efficient operation of autonomous systems, particularly in emerging applications such as autonomous vehicles.

In this project we advocate for the centrality of scene flow, the 3D motion field, in achieving dynamic scene understanding.

We also argue that it is important to address not yet one niche problem, such as scene flow, but rather take a system perspective and through this exploit the interdependencies between components which can be leveraged in a learning system.

We will use a self-supervised learning approach to be able to get access to data to train on without having to annotate it.

Insprired by how foundation models are trained we suggest a cascaded learning approach, where things are learned in stages, using the ouput from one stage as input for the next.

The goal is to go beyond the state-of-the-art in several components in a dynamic scene understanding system such as scene flow, object detection, object tracking, and ground segmentation. Given the complexity of the problem we plan to hire a postdoc who will work with the PI and his research group.