EAGER: III: Visualizing and Tracking Progress in Multimodal CR (Cardiac Rehabilitation) Data

While integration of multimodal information has been widely researched, the difficulty in carrying out this integration in a domain-agnostic, generic manner has been problematic. Past results have emphasized the need for addressing multimodal integration in a domain-specific manner. For instance, cardiac rehabilitation is a diverse range of practices for restoring individual's functioning.

Unfortunately, only 30% - 40% of patients report regular exercise at six months after discharge and 39%-45% of these patients suffer from at least one readmission within one year. These poor outcomes motivate the need for using technology for remote monitoring in this multi-modal system. The overarching goal of this proposal is to understand how multiple cardiac experts view multimodal data, decide on the progress, and the variations/biases among the experts' views and decisions.

This understanding would help us design and build a multimodal visualization and progress-tracking system that can provide meaningful information to stakeholders. Though the proposed algorithms for multimodal data analysis are tightly tied to the biomedical domain, the derived knowledge and understanding would be useful to other domains using multimodal sensing.

Results, metrics, and algorithms from this research will be published widely in high-quality academic journals and conference proceedings.

Integrating, visualizing and tracking progress using multimodal data is highly domain dependent. In this project, cardiac tele-rehabilitation deployed in-home is the target domain, generating multimodal data, with their diverse data characteristics and varied timeframes. Research challenges in domain-specific multimodal integration typically include the characteristics of the multimodal data as well as the domain-specific needs.

For multimodal integration for cardiac rehabilitation, the challenges include: (i) Integrating the multimodal data with diverse types of data with varying temporal characteristics to find relationships among potential adverse events; and, (ii) Possibilities for using mobile and wearable sensors to provide opportunities for personalization both in the rehabilitation and in the multimodal data integration. The proposed system, in the form of a mobile app, will democratize data acquisition.

This, in turn, could lead to a better understanding of bias among experts and possible strategies for mitigating the bias and provide appropriate feedback and nudges to patients.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.