Deep Learning of Mass Spectrometry Imaging

PROJECT SUMMARY Mass spectrometry imaging (MSI) is a rapidly developing technology which gives pathologists many new types of targets (e.g., metabolites and lipids) to assess for translational cancer research. However, the resulting data are even more complex than traditional images because they are highly-dimensional, and large (~100GB per

tissue section). Each “pixel” in the resulting data structure contains a 2-dimensional mass spectrum made of both measured ion mass and ion mobility (m/z, 1/K0), and each spectrum typically contains hundreds to thousands of individual ions (metabolites and lipids). Deep-learning methods (machine learning) have been

successfully applied to histopathology data by several laboratories including Dr. David Fenyo, Co-Investigator of the current proposal, with such models being able to discriminate between different cancer subtypes and grades for example. However, most machine learning models of image-data are designed around 3-data channels (Red,

Green, Blue) for analysis of digital images. Therefore, the n-dimensional data structure of mass spectrometry imaging datasets is not easily amenable to these proven machine learning workflows. We will make MSI data accessible to these approaches by expanding to n-dimension “color-channels”, with each unique metabolite or

lipid image serving as an individual data input. For the deep learning component, we will retain the same overall architecture and workflow of the Panoptes tool, published by Fenyo et. al., (Cell Reports, Medicine, 2021) but we will apply an n-dimensional approach and test the data structure on existing data which has parallel H&E

stain information annotated by pathologists. These challenges are addressed in Aim1 of the current proposal, while Aim 2 addresses a closely related challenge of detecting image correlations both within and between these data structures and other imaging modalities. Image correlations within such data are more trivial, but these

analyses are not well supported by existing academic or vendor software due to the amount of computation needed for hundreds of data dimensions. We further propose and test an approach for converting these multi- dimensional data into centroided single ion images, followed by linearization of the image to enable a simple

Pearson correlation metric, thereby making a complete correlation matrix accessible by a scaling factor of n2 to the number of detected ions. Secondly, to deal with spatial correlations between MSI datasets and images from other modalities, or adjacent tissue sections which may be different in size and shape, we propose to implement

a spatially aware elastic transform of the centroided image data prior to correlation analysis and machine learning.