Interactively trained ‘human-in-the-loop’ deep learning approach to improve cardiac CT and MRI assessment for accurate therapy response and mortality prediction

Background: A comprehensive assessment of all four chambers of the heart is critically important to inform the diagnosis and treatment of individuals with cardiac and pulmonary disorders.

We have recently developed a deep learning-based method trained on scans with high variability that has achieved fully automated and accurate segmentation of the left ventricle on cardiovascular magnetic resonance imaging (CMRI).

There is an urgent need to improve existing deep learning models to automate contouring of all four cardiac chambers to derive a comprehensive cardiac assessment.

Aims and objectives: Aim: Develop an interactive human-in-the-loop multi-stage deep learning tool to measure the health of the heart.

Objectives: Extract automatic physiological parameters by interactive human-in-the-loop deep learning using a multistep approach.

Assess the prognostic significance of the approach Develop a prototype for trials Project plan and methods used: Perform multiple interactive steps of manual editing and deep learning on four chamber contours and landmarks using 600 CMRI / 600 CT scans to achieve ideal deep learnt segmentation. Derive automatic physiological parameters and assess the scan-scan repeatability.

Assess the prognostic significance of automatic physiological parameters and deep learnt shape and/or motion analysis in well phenotyped cohorts such as the ASPIRE registry, Sheffield 3D lab and National Pulmonary Hypertension Cohort (>14K CT and MRI). Success shall be judged at receiver operating characteristic analysis 0.80=ideal.

We will develop the user interface/prototype, seek feedback from end users/individuals and engage with an industrial partner.

Timelines for delivery: In work package 1 (WP1) we develop the automatic physiological parameter extraction approach (M12), in WP2 we will assess the prognostic significance (M18) and in WP3 we will develop the prototype (M24). Anticipated Impact and Dissemination: Precise, rapid and repeatable prognostic and therapy response assessment.

Dissemination will be achieved through professional bodies, journal publications, guidelines, and patient associations.