RCT of a Measurement Feedback App to Improve Data Quality, Supervision & Outcomes in Behavioral Health

PROJECT SUMMARY/ABSTRACT Decades of treatment studies demonstrate that youth with significant behavioral health needs make more progress when their treatment planning is informed by ongoing quantitative data collection (e.g., by changing treatment strategies, increasing therapy hours, and adding services), however, aides often do not collect high

quality data consistent with evidence-based practice. Measurement feedback systems (MFS), originally developed to support data collection and inform treatment decisions in outpatient therapy as part of measurement based care, may be an ideal starting point from which to improve aides' data collection; however,

MFS have not been applied and tested in this setting. Our application, Footsteps, comprises a low-cost MFS and implementation strategy to support electronic data collection and target implementation mechanisms – aides'

intentions, attitudes, norms, and self-efficacy – associated with data collection to optimize clinical care. Footsteps was developed in partnership with community behavioral health agencies, guided by behavioral economics principles, user-centered design, and a conceptual model that integrates the science of behavior change with

organizational theory. Footsteps integrates digital data collection in a customizable, server-based, native app with tools for supervisors to review data and provide feedback, and behavioral-economics informed features, such as gamification, leaderboards, employee of the week emails, targeted reminders, celebratory/encouraging

messages, and in-app data collection tutorials, to increase motivation to collect data. As part of our Penn ALACRITY Center (P50 MH127511), we conducted a pilot RCT in which we compared Footsteps with a data- collection-only app in a pilot trial. We found that Footsteps was acceptable to aides and feasible to use, and that

it engaged our target mechanisms of attitudes, norms, self-efficacy, and motivation. We now are ready to test the app in a fully powered trial. Our formative work also raised three key questions: a) which behavioral strategies are most effective for increasing data collection; b) does Footsteps alter supervision processes; and c) does

Footsteps ultimately improve youth outcomes? We propose a randomized, hybrid type 2 effectiveness- implementation pragmatic mixed-methods trial in which we enroll 150 aides and 30 supervisors. Specifically, we propose to: (Aim 1) examine whether Footsteps improves data collection quality and youth outcomes in an RCT

on Footsteps vs. a “data collection only” application; (Aim 2) explore mediators of data collection quality, specifically changes in aides' intentions, norms, attitudes, and self-efficacy regarding data collection via biweekly surveys and interviews; (Aim 3) examine the impact of Footsteps on supervisory processes, team

communication, and changes to child treatment plans via interviews and biweekly surveys with 30 supervisors and a subsample of 30 aides. The proposed study would be one of the first studies to examine the effect of theory-informed behavior change mechanisms on data collection in behavioral health care and will advance the

science of digital health as a support to practitioners in implementing behavioral health programs.