Auditing Social Media Algorithmic Pathways to Measure Prevalence of Online Misinformation Related to Opioid Misuse

Abstract: Opioid misuse has become a public health epidemic in the United States with more than 70% of indi- viduals with an opioid use disorder (OUD) never receiving any sort of treatment. Even fewer receive medications for addiction treatment (MAT)—the gold standard for treatment and a safe, cost-effective way to reduce the risk of

overdose while improving the likelihood of sustained recovery. Due to the stigma surrounding opioid misuse, in- dividuals often seek non-conventional ways to recover, such as using online resources, specifically social media, and in particular microblogging sites like Twitter. However, social media platforms are often rife with MAT misin-

formation (MATM), posing a serious barrier to recovery. Moreover, the harmful effects of online misinformation are further exacerbated by the design of the algorithms that drive content curation or recommendation on social media sites. Yet, research on understanding algorithmic pathways to health-misinformation is rare and that re-

lated to opioid misuse is practically non-existent. This R21 proposal will address this gap by conducting formative research through the use of robust audit methodologies coupled with rigorously validated machine learning (ML) techniques, to lay bare an unexplored phenomena in the OUD medication and treatment domain—algorithmically

curated MATM in online social media systems, specifically Twitter—one of the most widely used social media platforms for sharing and seeking OUD information. The work advances this research agenda by leveraging the team’s pioneering research in addressing two of the key technical challenges driving this proposal: a) building

computational approaches to audit black-box platform algorithms that curate, recommend, or filter information viewed by end users; and 2) developing ML techniques that detect pre-existing or emergent online misinforma- tion. Drawing from advances in algorithmic audit work and PI’s own successful audit study designs, Aim 1 will

build tools and methodologies to audit search and recommendation algorithms for MATM on Twitter across vari- ous individual user characteristics and algorithmic inputs. The developed methodologies will be generic enough to be adaptable across other social media platforms. In Aim 2, we will leverage these methodologies to conduct

an exhaustive set of carefully controlled audit experiments on Twitter to investigate it’s search and recommenda- tion algorithms’ tendency to surface MATM. We will also develop and evaluate ML methods that can automatically determine whether the collected social media posts contain MATM. Finally, in Aim 3 we will develop a mixed-

methods approach to quantitatively and qualitatively validate our audit results with participants on Twitter who misuse opioids. The project brings together a multidisciplinary team of computer scientists and a clinical psychol- ogist, with expertise in social media analytics and recruitment, online algorithmic audits, substance use disorders,

machine learning, and natural language processing. The knowledge we produce will set the stage for future re- search in early detection of risky OUD behaviors, understanding the role of the online information environment in exacerbating or preventing OUD risks and launching evidence-based interventions to mitigate such risks.