Unified and fair multimodal representation learning for autoimmune diseases

ABSTRACT Autoimmune diseases affect 1 in 10 people. Commonly, patients needlessly suffer for years due to delays in diagnosis and referral delays to specialists. Systemic lupus erythematosus is a classic example due to its nonspecific symptoms and potential to mimic other diseases. It affects women 9 to 1 with average

diagnostic delays of over 5-years, increasing the chances of life-limiting end-organ damage. A diagnosis typically requires an experienced rheumatologist to carefully consider and integrate various data sources. This need for a specialist creates health equity concerns which are further compounded by the

disproportionately higher prevalence of lupus in Black and Hispanic women. This project will develop a unified multimodal representation learning technology that will allow 1) using many different datatypes (e.g., electronic health records, omics-data, full-body imaging, clinical measures, tabular data, and data from activity monitors); 2) adding data sources to the multimodal model

as needed; 3) supporting missing modalities by cross-modal generative learning; 4) providing inherent end-to-end interpretable results; and 5) patient-specific disease predictions and patient-personalized multimodal information acquisition plans. The project will use a holistic design approach where a model

will account for population imbalances during training and model design and will incorporate debiasing approaches directly into the modeling. Our approaches will be generally applicable to multimodal learning. They target significantly earlier diagnoses for autoimmune diseases, strategies to recommend suitable additional diagnostic tests, and the

ability to identify patients at greatest risk for the worst outcomes for which more aggressive treatments may be recommended.