CAREER: Formalizing Open World Safety for Interactive Robotics

This Faculty Early Career Development (CAREER) grant will support research that attempts to expand the understanding of how robots can make safe decisions. Ensuring robot safety is a complex challenge, as robots must understand how their actions could lead to harmful outcomes. Most robot safety efforts focus primarily on preventing collisions.

However, safety concerns in real-world environments, such as homes, city streets, and hospitals, are far more nuanced. For example, robots should avoid entering areas marked with caution tape, slow down when transporting hot cups of coffee to prevent spills, and seek clarification when uncertain about a task. This research aims to develop new methods that enable robots to better understand and respond to these nuanced safety challenges.

The project intends to advance scientific knowledge and contribute to national well-being by making robots more trustworthy and effective in a wide range of settings, from personal homes to public and professional spaces. Additionally, the project will support robotics education through the following initiatives: i) Developing a “Robotics Red-Teaming Challenge,” where college students will design and stress-test robot safety algorithms; ii) Engaging K-12 students in hands-on experiences with robot programming and troubleshooting; iii) Collaborating with government and industry stakeholders to establish safety benchmarks and promote the broader adoption of safe robotics practices.

This project will develop an algorithmic framework to align a robot’s understanding of safety with that of human stakeholders. The approach builds on foundational methods in safe control, which characterize safety constraints as arbitrary sets in state space. While this mathematical model is theoretically highly expressive, its practical application has been limited to controlled, collision-avoidance scenarios.

This award supports fundamental research to generalize this framework to more nuanced safety specifications by embedding learned patterns about the real world, inferred from robot and human data. Research conducted under this grant will strive to enable robots to: i) Automatically assess the trustworthiness of their learned models of human behavior and react accordingly; ii) Update their internal safety representations using online human feedback, such as language; iii) Generalize their safety policies to account for uncertainty and learned latent state spaces.

These advancements will be evaluated through hardware experiments, where robot arms and mobile manipulators will be deployed in unstructured environments, such as kitchen and atrium settings. In addition to these research objectives, the project includes a robust education and outreach plan to teach the next generation a nuanced yet practical perspective on robot safety.

This will emphasize the often subtle and unanticipated consequences of robot interactions in real-world contexts.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.