ML-ROVER: Machine Learning to Reduce Laboratory Test Overutilization

More than 20% of laboratory tests in the intensive care unit are medically unnecessary. Laboratory overutilization contributes to reduced care quality and adverse complications including iatrogenic anemia. Multiple medical societies have highlighted the urgent need to address this problem, but the vast majority of

interventions in the literature are quality-improvement initiatives with poor generalizability. The literature is even sparser in the highly vulnerable pediatric population, who are more susceptible to iatrogenic anemia given their smaller blood volumes than adult patients. With the rapid expansion of electronic health records (EHRs) and

the development of massive clinical databases, machine learning (ML) has become a promising tool that can address the problem of laboratory overutilization. While few models have been developed to predict laboratory results in adults, among pediatric patients model development is extremely limited. Furthermore, very few

predictive ML models in any clinical domain are translated into usable clinical decision support (CDS). Despite strong recommendations from nearly all best practice informatics resources, most CDS implementations rely on “out-of-box” deployment rather than employing user-centered design principles. This can leave

burdensome, sometimes harmful systems in place without demonstrated effectiveness. This staged-award proposal will leverage the PICU Data Collaborative (PDC), a multicenter collaboration of pediatric intensive care units (PICUs) across the United States, to develop ML-based CDS to predict future laboratory values for the purpose of reducing laboratory overutilization. In the R21 phase, ML

models will be trained on 188,000+ unique PICU patient encounters in the PDC database to forecast future laboratory values (Aim 1). Concurrently, guided by the Practical Robust Implementation and Sustainability Model (PRISM) framework, contextual factors will be identified to inform an ML-based CDS implementation

within the PICU sociotechnical environment. In the R33 phase, new a priori identified features will be incorporated to enhance the ML models developed in Aim 1. These retrained models will be silently evaluated on prospective data from selected PDC sites. In Aim 4, an EHR-embedded CDS tool will be designed for a

selected PDC site, incorporating user-centered design principles informed by the results of Aim 2. The final ML model from Aim 3 will then be deployed in a pilot study at the single site, where we will measure the implementation outcomes of reach and adoption. The result of this work will establish a useful, usable CDS tool to reduce laboratory overutilization based

on multicenter data and framed in the PICU contextual environment. Our pilot deployment will establish the groundwork for a future effectiveness-implementation hybrid multicenter trial. These processes will be generalizable and serve as a blueprint for developing data-driven translational decision support tools that can

be adapted to a variety of diverse healthcare settings and subsequently reduce disparities.