Promoting algorithmic equity in in-hospital mortality prediction

Project Summary/Abstract In-hospital mortality prediction models (MPMs) are widely used in clinical research and practice; but existing MPMs suffer from algorithmic bias, or systematic differences in performance by group. We and others showed that MPMs for hospitalized patients – the Sequential Organ Failure Assessment score (SOFA) and Laboratory-

based Acute Physiology Score, version 2 (LAPS2) – overestimate mortality for Black patients with acute respiratory failure (ARF) or sepsis, and underestimate mortality for white patients. Biased MPMs may thus produce healthcare inequities and flawed inferences about contributions of sociodemographics to clinical

outcomes. Therefore, we seek to develop, validate, and demonstrate the impact of a novel MPM that optimizes fairness (i.e., defined by ‘groupwise optimality,’ optimizing subgroup performance without sacrificing predictive accuracy) across key subgroups defined by race, ethnicity, sex, age, primary language, insurance status, and

social vulnerability without sacrificing accuracy. We will address key causes of bias in model development: differential missing data and calibration biases. We will study hospitalized ARF and sepsis patients because they face high risks of biased predictions due to diagnostic uncertainty and high mortality risk, and these

syndromes pose increased mortality risks for racial and ethnic minorities. In Aim 1, we will develop a fairness- informed, in-hospital MPM. We will identify predictive features using those in common MPMs and structured data within 24 hours of presentation. We will assess missing data bias by comparing feature proportions by

subgroup, excluding biased features, using a 2018-2023 cohort of ~220,000 encounters across 28 hospitals in the University of Pennsylvania and Kaiser Permanente Northern California health systems. We will select features using elastic net regression, and develop and internally validate a set of novel MPMs for use at

admission, building logistic and elastic net regression, and machine learning models. We will implement model bias audits and mitigation strategies (i.e., multicalibration, optimizing calibration across subgroups without sacrificing predictive accuracy) to develop a set of optimized MPMs. We will evaluate performance overall and

by subgroup, and compare performance to SOFA, LAPS2, and the Epic Deterioration Index. In Aim 2, we will conduct focus groups among key stakeholders to present blinded results of the novel MPMs, varying subgroup performance tradeoffs and decision thresholds, to select the model and thresholds that best promote equity

and accuracy. In Aim 3, we will test the external validity of this MPM among patients admitted to MedStar Health, a health system serving primarily racial and ethnic minority patients, using a different electronic health record. In Aim 4, we will quantify the impact of the novel MPM on key use cases, by (1) re-analyzing our team’s

pragmatic trials to assess the impact of risk adjustment on effect estimates overall and by subgroup; and (2) performing microsimulation informed by intensive care unit (ICU) demand in our health systems during peak COVID surges to compare ICU bed allocation across subgroups, compared to SOFA and LAPS2.