SCH: Develop Clinical Time Series Foundation Models for Sepsis Early Detection

This project will advance national health and promote science and technology development by providing algorithms, software, and systems that can train foundation models on clinical time-series data for accurate and early detection of sepsis. Sepsis is a life-threatening condition that occurs when the body's response to an infection is out of control, leading to widespread inflammation, multiple organ failure, and eventually death.

Early prediction of sepsis is crucial for timely intervention and improved patient outcomes. The detection of sepsis hinges on the interpretation of clinical time series (CTS) data. These data are inherently complex, characterized by their high-dimensional nature and the inconsistent timing of measurements.

This project will build innovative technologies to develop robust foundation models, also known as large pretrained models, to process and learn from these intricate CTS data streams for acquiring a deep understanding of the temporal patterns leading to septic shock and enabling the detection of early warning signals that conventional methods might miss. The project will significantly improve the accuracy and timeliness of sepsis detection, which enables physicians to take early life-saving treatment interventions to prevent septic shock, organ damage, and death.

Furthermore, pretrained on a broad spectrum of CTS data, these foundation models will effectively accommodate the variability across different patient populations and clinical settings. In addition, these models do not require large amounts of labeled data, reducing the burden on healthcare systems to provide extensive annotated datasets.

To achieve the goal of developing foundation models on CTS data for sepsis early detection, this project will develop four thrusts of novel approaches, each pivotal to the lifecycle of developing foundation models. First, the project will develop CTSformer, a new Transformer-based model specifically for clinical time series (CTS) data. CTSformer is designed to adeptly cope with the complexities inherent in CTS datasets, such as irregular intervals and incomplete data entries.

Second, the project will develop a pretraining method capable of automatically discovering an optimal masking strategy for time points in CTS data, eliminating the need for extensive manual tuning. Third, the project will develop a new parameter-efficient finetuning approach that differentiably optimizes layer-specific ranks within low-rank adaptation, enhancing finetuning accuracy and computational efficiency.

Fourth, this project will develop bi-level optimization-based methods to enhance the interpretability of CTS foundation models. The proposed approaches are underpinned by a cohesive methodological foundation, which is bi-level and multi-level optimization, enabling end-to-end learning at task level. Together, these approaches converge to address the crucial medical challenge of early sepsis detection.

This project effectively addresses a fundamental knowledge gap that existing foundation models are inadequate in dealing with complex CTS data, incur high computational cost, require time-consuming labor-intensive manual tuning, and are difficult to interpret. It represents the first one developing effective, computationally efficient, and interpretable foundation models on CTS data for sepsis early detection.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.