ICF: NIRG: HistoMaps: Stain agnostic feature representations to identify clinically relevant traits in the tumour microenvironment

In early years of computational pathology, the algorithms were mainly focussed on segmentation and identification of objects such as nuclei, glands, ducts, vessels, and other patterns which are of interest to pathologists in every day clinical practice.

The concept was to assist pathologists in identifying patterns which are difficult to eyeball over the huge landscape of cancer tissue in a whole slide image (WSI).

The advent of modern CPath algorithms based on deep learning (DL) found that there are hidden features which humans usually ignore due to inattentional blindness.

Therefore, CPath has moved beyond identification and classification of individual patterns within a whole slide image (WSI) towards WSI-level or case-level diagnosis, mutation and therapeutic response prediction discovering new morphological patterns, tissue phenotypes, even surpassing pathologist performance in some cases.

On the other hand, DL algorithms are usually considered to be a black box due to lack of interpretability of the learnt features which makes it difficult to understand the biology of different diseases.

One of the reasons is our inability to analyse huge landscapes in the tumour microenvironment (TME) where the WSIs are divided into small patches before analysis due to hardware limitations and complex DL architectures required for the analysis of images from different stains and modalities.

The challenge is the gigapixel size of the WSIs containing the landscape of cancer which on one hand compels exploration but at the same time faces technological challenges. Due to tumour heterogeneity these small patches are usually not representative of the WSIs.

Therefore, we need to develop techniques which can analyse WSIs without dividing them into smaller patches keeping the spatial information intact.

This not only allows to overcome tumour heterogeneity limitations but helps in identifying heterogenous regions and embedded spatial relationships linked to patient outcome and other clinical variables.

These techniques should be able to overcome the practical limitations of the hardware, invariant to the input stains and should be able to help with interpretability and biological understanding of the TME.

The algorithmic limitations are currently being tackled by WSI-level weakly supervised labels or compressed representations.

These approaches have some major drawbacks e.g., these approaches discard the essential spatial information required to incorporate cell-to-cell interactions in clinically significant regions during compression and are focussed mostly on identification or classification of disease into sub-categories where the DL model is treated as a black box.

Analysis of TME at the cellular level is important to understand mechanisms in cancer where tumour heterogeneity plays a significant role.

Multiplexed Immunofluorescence (MxIF) images provide additional data to subtype individual cells on the same tissue section which is not currently possible with existing brightfield approaches.

There have been recent advances in whole slide image fluorescence imaging which allow scanning of WSIs with multiple markers.

Therefore, we need stain and modality agnostic approaches which can analyse WSIs without losing spatial information at the cellular level so the rich data can be mined for better understanding of cancer.

We propose to build on existing technology and utilise the extracted information to understand TME interactions at the whole slide image level.

In this project, we will develop stain agnostic techniques to analyse and identify patterns in whole slide images (WSIs) by creating HistoMaps which can be directly related to biologically meaningful and clinically relevant parameters i.e., mutations, survival and response to therapy linking histology landscapes to clinical variables for better understanding of cancer helping oncologists to make informed decisions on therapeutic interventions and assisting pharma to develop new targets.