CAREER: Mobile Sensor-Based Adaptive Emotion Prediction and Feedback Delivery

Recent advances in wearable, mobile technologies and the Internet of Things enable us to collect moment-to-moment physiological, behavioral, social, and environmental data on mobility pattens, sleep, central and peripheral nervous system activity, and social interactions, all without disrupting daily routines. Such data show a potential for revolutionizing how we diagnose, ameliorate, and prevent health disorders.

This project designs, implements, and evaluates personalized, adaptive algorithms to detect and predict emotional states using multimodal sensor data, and to provide feedback to users to improve management of mental state. The emotion detection and feedback algorithms adapt to changing human physiology, behavior, context, and preferences. This research will result in emotion assistive technologies that enhance human performance, health, and wellbeing, thus improving quality of life.

The project will provide a platform for integrating mobile sensor data and providing feedback to subjects valuable to a broad range of populations for personalized medicine. The project will yield insights into ethical issues in the development, evaluation, and use of human data and artificial intelligence technology. The research activities will train graduate students, be integrated into classroom curricula in data science, and provide research opportunitites and summer internships for undergraduate and high school students.

This project will develop dynamic emotion modeling and feedback systems to harness multimodal human data for automatic human emotion recognition and prediction. The system will provide safe and personalized feedback delivery to help manage emotional states. This goal will be achieved by addressing three fundamental research challenges: (1) multi-modal, multi-timescale physiological and behavioral pattern interpretation and analysis; (2) adaptive emotion label sampling for effective emotion detection and prediction; and (3) safe and sustainable automatic personalized feedback delivery.

The resulting emotion detection and feedback systems will be integrated to help users in poor health manage emotion. The performance, usability, safety, and effectiveness of the technologies will be evaluated via human subject studies. 

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.