SCH: Blazing Data Trails: Digital Pathology and Specialist Attention

When looking for cancer in clinical slides, pathologists move the focus of their attention around the slides in complex ways. These skilled shifts of attention are critical to how pathologists make expert diagnoses. This research program seeks to understand these shifts in attention in order to build an artificial intelligence (AI) system that that will be able to look at a slide the way a human pathologist would.

Building an “AI expert pathologist,” however, requires a lot of data for it to learn, just like a pathologist needs years of training to become an expert. In order to provide the model with many examples of expert attention behavior, essential for it to make good predictions, the investigators will collect a large dataset of attention behavior from human pathologists.

The human pathologists’ behavior will also serve as feedback to the AI model, enabling the AI system to model and reproduce how the human pathologists expertly sample the slides by moving their focus of attention. The investigators will also build AI-fueled tools that can predict where an expert would have focused their attention in a slide, thereby giving human pathologists feedback from the AI pathologist.

The aim is to improve human accuracy of cancer diagnoses, which is paramount to improving the healthcare infrastructure of the country. The work also has the potential to improve histopathology training in medical personnel and to lead to next-generation AI models for cancer classification. The AI scientists trained through this project will be experts in building AI-tools that understand human expert performance and synergistically enhance it.

A large database will be created of pathologist’s cursor-based movements during cancer interpretations, referred to as attention trajectories. These will be collected online from pathologists searching for metastatic cancer in Whole Slide Images (WSIs) of lymph nodes that were excised as part of cancer surgeries. For each WSI, one of four “diagnoses” will also be collected: negative, small, medium, or large metastases.

Using a family of AI methods called imitation learning, the investigators will generate personalized as well as group prediction models of pathologist attention trajectories, applying Active Imitation Learning to real human behavior. Techniques for batch processing and pathologist-in-the-loop learning of attention trajectories will also be developed.

An improvement in the efficiency and accuracy of pathology classification algorithms is expected through use of a multi-resolution approach that only processes small parts of WSIs by combining computational and human attention priors. Lastly, attention-based diagnostic aids that suggest areas to examine at higher magnification will be developed for human pathologists to use during slide interpretation

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.