The diffusion of development: Extending RCTs with agent-based modelling to understand spillovers of development interventions

Randomised controlled trials (RCTs) are widely considered the gold-standard to assess the impact of development interventions. However, RCTs can only go so far in helping development organisations answer the key question that interests them: Do their interventions improve people's livelihoods? After all, experimental impact evaluations can only tell us how treated individuals fare in comparison to control individuals.

Meanwhile, development organisations implement interventions in the hope that benefits extend beyond their direct beneficiaries. Because it is impossible to provide aid, training, or other assistance to everyone who needs it, such organisations often hope for spillover effects of their interventions that have long-lasting and self-sustainable effects on the whole community despite only targeting a few individuals.

Moreover, it is plausible that some interventions require spillovers as individuals need support from their communities to make life-changing decisions.

We propose to transform the ways that researchers and development organisations assess the impact of development interventions by combining RCTs with agent-based computational modelling (ABM). We hope to apply agent-based modelling in a novel way, by using them to enhance RCTs and providing a cost-effective measure of the widespread effects of interventions beyond the direct beneficiaries.

This methodology greatly improves upon existing experimental designs that are adapted to study spillovers - not only because it is more flexible and more cost-effective, but because it can reveal much more about spillover effects. First, existing experimental designs only allow us to observe spillovers on strictly defined units. Our method will allow us to observe population-wide spillovers.

Moreover, because we can observe population-wide spillovers, we can see how changes to interventions can shape outcomes at the macro-level. And finally, while we know that the interventions will often work or work best when a sufficient number of individuals have adopted the desired behaviour, it is very difficult to observe where these thresholds lie using existing experimental designs.

Our method will allow us to observe tipping points at which the treatment has affected sufficient numbers of people that subsequent change is readily accepted.

We will apply our proposed methodology to a development intervention - funded by DfID and implemented by the International Organisation for Migration (IOM) -- aimed at improving prospects for self-employment in the Gambia and Senegal. Our ultimate aim is to provide transparent and thorough guidelines on appropriate experimental designs that can capture community effects; programming packages to implement the methodology in NetLogo, a free and easy-to-use ABM platform; and a user-friendly app to conduct preliminary tests and power calculations.