Structured Interactive Perception and Learning for Holistic Robotic Embodied Intelligence

Robot learning has made remarkable strides thanks to high-capacity neural models and extensive datasets.

However, there are persisting research questions concerning large-scale robot learning models: are massive architectures and data needed for achieving robotic embodied intelligence to solve tasks intuitive to humans?

And how can we make substantial progress toward robust and adaptive robot learning systems to operate in the dynamic real world?

I posit that these open problems stem from overlooking the underlying principles and structure that govern the intricate robot-environment interaction and evolution.SIREN addresses these pressing issues by proposing a unique systemic view of robot learning through the holistic representation of robot and environment as an integrated system.

To achieve this, we will unveil key properties of the action-perception cycle for developing embodied intelligence by studying the intertwined flow of information and energy within the components of the holistic system.

For that, we propose a framework that pioneers information-driven and physics-aware objectives that encompass the learning from embodied multisensorial streams of a modular graph representation of the robot-environment system and its dynamics, backed by the versatility of graph neural networks, allowing for modular uncertainty estimation to promote robustness.

Eventually, we will yield resilient dynamics for training uncertainty-aware, composable skills to adapt to new tasks.

SIREN's breakthroughs will enable robots, like humanoid mobile manipulators, to merge in unstructured, human-like settings and perform challenging tasks that require smooth and efficient perception-action coordination, balancing generalization and robustness in the face of inevitable real-world uncertainties.

Our paradigm shift opens avenues for future groundbreaking research rooted in SIREN's impacts toward continuous robot learning systems that are integrated and evolve with their environment.