Collaborative Research: DARE: A Personalized Assistive Robotic System that assesses Cognitive Fatigue in Persons with Paralysis

With the advancements in robotics and artificial intelligence, assistive robotic systems have the potential to provide support and care to people with Spinal Cord Injury (SCI). As robots become more widespread, like today’s mobile phones, assistive robots can play a significant role in assisting persons with disabilities at home, improving independence and everyday quality of life.

For example, a robot may assist an individual with motor impairments to perform a task, such as preparing lunch. Current research focuses on ensuring safe human-robot cooperation in industrial environments, which will not threaten or harm the physical health of the human teammate. However, there is limited research on understanding the cognitive or mental state of a human who cooperates with an assistive robot daily for performing activities of daily living.

This is important for developing effective, intuitive, and personalized human-robot cooperation. The objective of this project is to design and develop an end-to-end personalized assistive robotic system, called iRCSA (Intelligent Robotic Cooperation for Safe Assistance), to recognize, assess, and respond to a human’s cognitive fatigue during human-robot cooperation.

The focus of the system is on human-robot cooperative tasks where a human with SCI and a robot cooperate during daily tasks (e.g., cooking). Students who have experienced SCI will be involved in every stage of the project, to ensure the acceptability and usability of the proposed system. In addition to the significant impact of this research on the improvement of life independence for persons with disabilities, the project includes the development of new university courses for assistive technologies and summer school programs for K-12 students, so that students gain knowledge on robotics and assistive technologies for their prospective studies in Science, Technology, Engineering, and Math (STEM).

This project will develop an end-to-end framework for online cognitive fatigue assessment as part of the proposed Human-Robot Cooperation (HRC) system, which constitutes a breakthrough in the ability to effectively integrate the human component in the adaptation (personalization) of assistive robots and address key HRC challenges through these three thrusts: (a) The development of a novel human-technology system for online cognitive fatigue assessment using multimodal data; (b) the development of an adaptive robotic system for personalized interaction based on cognitive fatigue assessment; and (c) an experimental testbed of HRC scenarios to enable HRC, machine/deep learning, and robotic systems computational advances. In addition to detecting a user’s cognitive fatigue, the HRC system assesses the level of cognitive fatigue severity so that new methodologies can be designed to provide personalized robotic interactions based on the user’s state of cognitive fatigue.

A participatory action research approach will be followed involving students with motor impairments throughout the project. Evaluation of the system will identify system benefits and weaknesses to improve HRC for persons with paralysis, as well as assess system usability, reliability, and user experience. Beyond the assistive robotic applications, the project will advance research in multimodal machine learning for intelligent online assessment and control in several fields where teaming between robots and humans is required.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.