Collaborative Research: Introspective Counterfactual Reasoning for Robust and Resilient Autonomy

The resilience and robustness of autonomous robotic systems in dynamic, unpredictable, and ever-changing environments are central concerns of the robotics community. To address these challenges, this research project introduces a novel "introspective counterfactual reasoning" capability to empower robots with lifelong autonomy. While counterfactual thinking—considering the implications of changes in the world that could have happened, but didn’t—is a foundational cognitive function in human beings, its application in robotics remains largely underexplored.

This project aims to bridge this knowledge gap by enabling robots to answer and learn from "what if" questions regarding both their surroundings and themselves, better preparing them for unforeseen events, potential hazards, and evolving contexts.

This project introduces two different yet interleaved forms of counterfactual reasoning: Contextual Physical Rehearsal and Introspection Adaptation. Contextual Physical Rehearsal allows the robot to model the physical world and forecast the outcomes of actions without actual execution. Introspection Adaptation focuses on predicting and enhancing the robot's capacity to perform tasks in unfamiliar environments and unexpected situations.

The strategy involves designing these capabilities, integrating them into diverse autonomy platforms as interconnected modules, and validating their efficacy in real-world tasks. The framework will be validated in a rigorous procedure from modular simulation testing to integration and deployment on real ground vehicles under challenging conditions. The project will create new interfaces that allow developing courses on field robotics and simulation and provide immersive and engaging programming activities for K-12 students.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.