CAREER: Robust Perception and Customization for Long-Term Autonomous Mobile Service Robots

This Faculty Early Career Development (CAREER) award will enable mobile service robots capable of operating in real-world human environments over extended periods of time. Existing approaches in robot perception are very good at reasoning about the current state of the world but suffer from a marked limitation in reasoning about potential changes that inevitably occur over time.

Another common problem of robot perception is that when deployed in unforeseen environments, robots commonly experience perception failures due to unanticipated conditions and violations of design assumptions. Finally, end-use customization and enhancements during operational use is tedious and fragile. This project will overcome these challenges by developing robust algorithmic approaches to recognize and react to dynamic changes in environment, identify failures in perception and learn from them, and additionally learn new tasks while in operation.

The research will enable the development and long-term deployment of mobile service robots in homes, workplaces, disaster zones, hospitals, and myriad other environments. As part of the project, the education and outreach plan will include a longitudinal effort for the education and mentoring of undergraduate students throughout the academic year as well as computing workshops with fun robotic activities for middle to high school students.

This objective of this project is to develop robust algorithmic formulations and analytical and symbolic models to enable long-duration autonomous mobile operations of service robots in dynamic human environments. First, a reformulation of robot perception will be introduced that will explicitly reason about the relation between the current state of the world and possible changes over time, in terms of the geometric shapes, visual appearances, and types of motions that objects are likely to exhibit in the world.

Second, approaches will be developed for robots to autonomously build models of their perception competence by leveraging redundant sensing and discrepancies between perceptual predictions and actual outcomes, thus enabling them to avoid or overcome future situations that would lead to errors. Finally, techniques will be developed to address customizability and learning of novel tasks using physics-inspired symbolic programs.

The approaches developed will be rigorously tested at multiple levels of integration, including on a team of autonomous mobile service robots deployed indoors and outdoors, performing tasks including package delivery, guided tours, and environment monitoring.

This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE).

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.