CRII: FRR: Semantic Vector Fields for Robot Navigation and Exploration in Unstructured Environments

To achieve greater mobility and autonomy for robots, it is important to study robust perception and navigation algorithms that can operate well not just with paved surfaces but also under challenging terrain involving rocks, vegetation, and other hazards. In structured environments such as highways and warehouses, robots can navigate by following lane markings or pre-defined paths.

However, in unstructured environments, such as construction, agriculture, rural delivery and disaster sites, robots need to have a deeper understanding of the surrounding objects and terrain in order to navigate safely. Unfortunately, most mobile robot systems deployed for navigation and exploration use their sensors to mainly gather geometric information such as the shape and location of the obstacles and other objects in the environment.

Richer information is needed, such as type of terrains and how difficult they will be to traversed, to facilitate navigation and exploration. To address this fundamental research gap, this project aims to investigate a semantically-aware framework for field robots that complements the geometric information with semantic information about terrain properties in order to improve the navigation and exploration capabilities of robots.

This research represents an important step towards achieving robots with advanced artificial intelligence capabilities that can operate well in unstructured construction, mining, or agriculture environments. In particular, removing barriers of entry for automation technology in these traditionally labor-intensive industries is vital towards increased economic competitiveness of these industries in workplaces of the future.

This research will revisit the potential field navigation method using deep learning-based 3D semantic reasoning tools paired with self-supervised learning from motion feedback to provide a fresh perspective on active perception for robots. The semantic navigation method contains three main components: (i) a semantic vector field prediction network to map raw sensor data to semantic features and map semantic features to navigation signals; (ii) a pre-training scheme that transfers prior knowledge from an observation database and expert demonstrations to the navigation system; and (iii) a semantically-guided exploration scheme to enable the robot to take calculated risks while gathering information about the surroundings.

The method will initially be evaluated on the physics-based MSU Autonomous Vehicle Simulator (MAVS) which has the capability of simulating robot operation under challenging terrain and weather. Finally, field experiments will be conducted at the one-of-a-kind off-road proving ground at Mississippi State University.

This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE). This project is also jointly funded by the Established Program to Stimulate Competitive Research (EPSCoR).

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.