Integrative Deep Learning Models for Multimodal Markers of Cancer Treatment Outcomes

PROJECT SUMMARY The most pressing challenge in oncology is the need for accurate biomarker-driven prognostic/predictive risk- stratification to identify patients who are unlikely to benefit from standard of care (SOC) chemotherapy early in their treatment, as they might be better candidates for alternative therapies (e.g., genome-targeted agents,

immunotherapy). Unfortunately, only ~31% of eligible cancer patients achieve partial/complete response to cytotoxic chemotherapy. For instance, over 40% of GB patients will inevitably recur within 6-8 months after chemotherapy, suggesting that they could have been better candidates for newer experimental therapies. A

significant challenge in management of these patients is thus, segregating GB patients based on their outcomes/response to treatment. Similarly, the aggressive chemoradiation protocol for rectal cancers results in up to 70% of patients achieving 3-year disease-free survival. However, reliably determining which rectal cancer

patients will not benefit from this protocol could allow for targeted adjuvant therapy to ensure optimal outcomes. Considering a “micro” to “macro” view of the tumor, comprehensive clinical evaluation for cancer involves acquiring multi-scale data, including radiology (e.g., CT, MRI) which provides macroscopic morphology and

structural tumor details, histology images containing rich phenotypic information at cellular level, molecular data (e.g., genome sequencing, gene expression, epigenomics, also known as multi-omics) which captures the underlying biological processes, and the clinical data (e.g., age, sex). Ability to comprehensively combine

disparate sources of information through computational approaches could enable discovery of new prognostic and predictive markers to reliably assess risks associated with response of chemotherapy and clinical outcomes. The F99 phase of this proposal continues my dissertation research on developing deep leaning (DL) multimodal

models (mmSurvNet) to build prognostic markers for clinical outcomes, by combining MRI and digital pathology, in rectal and GB tumors. My research for the F99 phase is driven by the hypothesis that DL models, using co- registered pathology and radiology images that capture spatially co-localized tumor biology, can yield robust and

reliable prognostic integrated-markers to predict clinical outcomes. Towards this, I will construct multimodal survival (mmSurvNet) models employing DL architectures that maximize spatial information across pathology and radiology. The attention maps for mmSurvNet will allow for establishing biological relevance, by spatially

correlating radiology images with corresponding pathology which will contain annotations of known prognostic tissue characteristics. My proposed K00 phase will involve building predictive DL models (mmPredictNet) through incorporation of genomic, clinical, longitudinal data together with radiology and pathology images to build

integrated markers predictive of response to chemotherapy, while also accounting for population health disparities. The current and future goals of my research are to develop comprehensive and reliable computational tools for clinically impactful treatment decision support in oncology.