The Effect of Large Language Models (LLM) on Engagement in Online Peer Production Communities

This project explores the impact of generative AI technologies, specifically large language models (LLMs) on online peer production content communities, such as Wikipedia and GitHub.

These communities depend heavily on the existence of a vibrant community and its members wiliness to contribute voluntarily and the introduction of LLM applications is expected to have transformative effect on the engagement dynamics within those production communities.

The proposal is structured around three primary aims: (1) the effects of LLM applications on individuals' willingness to engage and contribute to online communities, focusing on the quantity, type and quality of contributions; (2) the impact of LLM applications on communities openness, inclusiveness and trans-community mobility; and (3) the effects of LLM applications on users' demand for community-produced content, analyzing overall demand shifts and changes in content consumption patterns.

The project is based on an empirical analysis of publicly available data collected via APIs, as well as field experiments conducted in collaboration with existing online platforms.

We are currently witnessing an explosion of new LLM applications over time, which are already disrupting various content communities and will contribute to do so during the lifetime of this project.

We will collect longitudinal data on content communities in different domains and use the introduction of different LLM applications as quasi-experiments.

We also present preliminary results, based on a quasi-experiment that leverages the selective rollout of GitHub Copilot, a ground-breaking coding-focused LLM, in October 2021.

This project is pivotal in gaining in-depth understanding of the evolving landscape of digital collaboration and content creation.

It aims to provide actionable insights into the role of advanced LLM in shaping online communities, guiding future technological developments, policy-making, and community management strategies.