SCH: Personalized Machine Learning for Repeat Adverse Health Events using Novel Multimodal Self-Supervised Pretraining Methods

This project involves the creation of artificial intelligence (AI) models that make predictions about health events like substance use and stress-related blood pressure spikes using data from smartwatches like FitBit, Apple Watch, and other wearables. The innovation of this project comes from training personalized models that learn exclusively from each person’s wearable data.

Recent advances in AI methods allow us to train these models to understand each user’s individual biosignals patterns related to heart rate, movement, and other signal recordings using large amounts of unlabeled data that are recorded when the user wears the device. This should, in theory, enable us to refine these models to learn to predict relatively complex recurring health outcomes like stress and blood pressure spikes using much fewer labeled examples than what would have previously been necessary.

We will test this paradigm in two user studies related to stress-related hypertension and substance use detection.

The status quo for machine learning consists of the development of a one-size-fits-all model which is usually trained on data coming from one group and tested on data from another disjoint group. However, the advent of self-supervised learning makes it possible to learn from vast unlabeled multimodal data streams recorded from a single individual, allowing for a pretrained model which learns feature representations which are specific to the baseline temporal dynamics of a single entity’s data streams.

This project seeks to understand how such personalized self-supervised learning on multimodal data streams can be used to overfit, in a positive manner, a machine learning model to the unique patterns of an individual’s sensor readings, thus enabling model personalization and prediction of traditionally difficult or subjective targets. To make these artificial intelligence (AI) innovations practically useful and because these personalized models still require on the order of tens of annotations to converge, we propose to develop novel human-computer interaction (HCI) techniques which are tightly integrated into the AI workflow to facilitate reliable and effective yet minimal annotations of the adverse health event of interest from the end user.

We will evaluate this paradigm on two separate health conditions with differing data types and nuances: (1) substance use and craving measurements and (2) stress-related hypertension, each predicted using multimodal passively collected consumer wearable and smartphone data streams.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.