CAREER: Remote Control of Humanoid Robot Locomotion using Human Whole-body Movement and Mutual Adaptation

Agile teleoperation of legged robots through unstructured environments could enhance the capabilities of emergency responders such as firefighters. Consider the demanding physical task of manipulating an active hose while walking over uneven terrain. Delegating the task to a robot could be performed via direct teleoperation if the operator were to directly control the robot's entire body while experiencing the machine’s sense of balance and the interaction forces with the hose.

Due to lags in perception-action coupling, the task can only be performed reliably if the robot can predict and adapt to human motion intent. Teleoperation of coordinated locomotion and manipulation therefore requires bilateral motor and mind synergy between human and robot. The goal of this Faculty Early Career Development Program (CAREER) project is to advance a fundamental understanding of teleoperated locomotion of a humanoid robot firefighter using whole-body force feedback and eye gaze intent detection.

The project will advance the NSF mission to promote the progress of science and to advance national health, prosperity, and welfare by advancing a fundamental understanding of the complex dynamics that arise through the bidirectional physical interaction between human and robot agents during teleoperated locomotion. Activities planned to enhance the broader impacts of this research include efforts to enhance engineering education via active-learning activities, establish interactions with the general public, students, and professionals through outreach activities; and train the next generation of scientist leaders.

This CAREER project advances a fundamental understanding of the novel dynamics that arise during bilateral whole-body teleoperation of the locomotion of a humanoid robot traversing unstructured environments. Direct human-in-the-loop control of locomotion via teleoperation is one way to endow robotic assistants with human-level motor skills for physical interaction.

In this project, the human teleoperator uses their body to directly command the robot's locomotor while receiving force feedback through a novel torso-mounted interface. In contrast to conventional haptic interfaces that directly render to the operator the forces applied to the robot, the project's first research aim is to create a general mapping that renders also kinematic effect of perturbations.

The second aim is to understand human motor adaptation to robot locomotion dynamics. The third aim is to achieve navigation on uneven terrain via a shared-autonomy framework in which the robot adapts to human locomotion intent by modifying its foot placement on the fly. The efficacy of the approach is evaluated through human subjects experiment.

This work makes fundamental contributions to human whole-body haptics, biomechanics of locomotion, and control of legged robots.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.