Evaluation of Bolton’s Trauma Informed Schools Pilot (PHIRST LiLaC)

Introduction

Trauma-informed practice (TIP) is based on the knowledge that exposure to adversity can have significant adverse effects on life opportunities for groups facing multiple disadvantages (Whitaker et al, 2019). This can include the ability to develop positive relationships as well as influencing physical and mental health outcomes (Department for Levelling Up, Housing and Communities, 2023).

In recent years, interest in concepts such as Adverse Childhood Experiences (ACEs) has grown, with the introduction of trauma informed initiatives rolled out in non-clinical settings. While research is similarly evolving, evidence is still needed about how TIP functions in different contexts.

This evaluation aims to contribute to this evidence base centering on a pilot initiative in Bolton initially funded to work with one secondary school with a view to embedding a trauma-informed approach in the college’s culture, policy and practice. The longer-term aim is to inform wider rollout of TIP across schools aligned with the borough’s Health & Wellbeing Strategy.

The protocol describes the approach and methods to be used for PHIRST LiLaC’s evaluation during a first phase of the study. About the intervention

Bolton’s public health team commissioned the pilot as part of a wider programme to prevent the negative effects of ACEs in the borough following research undertaken by Public Health Wales and Bangor University. The organisation commissioned to deliver the pilot is called KCA Training (https://kca.training).

A secondary school in Bolton has been the primary site for the implementation of a package of support for TIP.

Bolton Learning Partnership (BLP) is a collaborative partnership of member schools and colleges with Bolton Council that aims to support the personal development, attainment and achievement of students across Bolton. A light touch rollout of the pilot will take place through the BLP in 2024 and beyond.

Overview of the Bolton trauma informed schools pilot Strand 1: Delivery in college setting

This main pilot phase has sought to take a whole school and community approach providing training across different staff groups in the college setting, additional targeted support to the college’s SEN team, as well as providing support to embed trauma informed approaches in the college’s policies and practices (e.g. development of a relationship policy). Engagement with the wider community including parents, and Voluntary, Community, Faith and Social Enterprise (VCFSE) organisations has also been attempted.

KCA’s remaining support within the college in 2024 will shift to a focus on embedding TIP for the longer term. Strand 2: Rollout via Bolton Learning Partnership (BLP)

Wider rollout of TIP to other schools in the borough is being supported. Rather than replicating the more resource intensive model in the college above, activities will be delivered through a Learning Partnership in Bolton, a collaborative partnership of over 20 schools and colleges with Bolton Council.

The following activities are planned:

Trauma Informed Practice Hub sessions. BLP already convene a series of topic-based learning hubs and a new Hub will support TIP locally. Training materials will be provided alongside the hub sessions.

Interested practitioners (champions) will be trained using a Resource the Trainer model enabling these in-house trainers to deliver TIP training/awareness raising activities in their schools.

A final element will involve engagement with Virtual Schools, children and adolescent mental health services and educational psychology services. Local evaluation

KCA has conducted initial inhouse monitoring and evaluation activity. They have provided access to their performance management portal and initial discussions have taken place about the possibility of accessing data pertaining to the pilot, subject to necessary data sharing agreements being in place.

Principally, KCA’s inhouse evaluation comprises:

Wave 1 baseline cross-sectional survey of staff (approx. 135/174 staff completed) and survey of pupils to gauge awareness and knowledge; with a follow up survey conducted (wave 2). The staff survey covered topics such as confidence/understanding of trauma/TIP as well as perceptions of the wider school culture;

Short feedback survey of staff to understand training experiences; A small number of interviews/focus groups with students and staff;

Accessing routine/service data for the college (e.g. behavioural reports, permanent and temporary exclusions for students). Evaluation aims and objectives

The overarching aim is to contribute knowledge and understanding about the implementation of TIP within school settings in order to inform future local programme development and implementation.

The first phase of the study will develop a detailed understanding of the TIP schools initiative in Bolton. More specifically, phase one will identify: (i) the change mechanisms embedded in the initiative’s design including how the pilot is expected to lead to anticipated or observed outcomes (ii) the processes of implementation; (iii) what features of the context are perceived to influence pathways to change (iv) the perspectives of different stakeholders on TIP in school settings and the ways in which it is understood to address local health inequalities.

In the second phase, we will conduct an assessment of the pilot’s rollout to other schools and its shorter-intermediate term delivery and impacts, including the extent that TIP is becoming embedded in school policies and practices?.

An initial logic model developed to guide the evaluation has been informed by a recent outcomes framework for TIP (Home Office, January 2024) and the wider evidence base as well as discussions with local authority and education practitioners involved with the pilot. This outcomes framework sets out potential outcomes associated with TIP within organisations as well as for key groups/beneficiaries (e.g. staff/practitioners and students) as well as outcomes across the short, intermediate and longer term.

The logic model also provides illustrative examples of change mechanisms theorised to underlie the intervention. Study methods Phase 1

Two main components are planned in phase 1: an ongoing evidence synthesis and a set of realist interviews with programme stakeholders. This phase will also produce the research plan for phase 2 of the evaluation. Evidence synthesis to support theory building

Review work commenced in the evaluation planning stage to identify the state of evidence on TIP more generally; we plan to continue an evidence synthesis component in the first phase of the evaluation delivery phase to support the process of theory building. Rather than undertaking a more formal systematic review, the synthesis will follow principles of realist synthesis (Rycroft-Malone et al, 2012), to extract evidence on change mechanisms, outcomes and contexts across studies of TIP in other school settings.

Searches and data extraction will be purposeful to elaborate on emergent programme theories identified in our primary fieldwork. For example, relationship building and trauma awareness development are already identified as mechanisms within the pilot therefore our searches/synthesis would purposely focus on critiquing evidence related to this component.

Interviews with TIP schools pilot stakeholders Primary data collection in phase one will involve a set of realist interviews with a range of core stakeholders. Sampling:

The sample will be practitioner stakeholders linked to the pilot who have had involvement or contact with the TIP pilot in the college and Bolton Learning Partnership (BLP). These may include: commissioners, members of the senior leadership team responsible for developing school policy at the college and other schools in the Bolton Learning Partnership; practitioners in receipt of intense training delivered to the college and the resource the trainer scheme, trainees trained by the new in-house trainers; specialist staff with a pastoral, SEN, behaviour or attendance remit.

dissemination and outputs

Locally, the evaluation delivery phase will embed regular opportunities to discuss the findings via our evaluation project group. We will also share all outputs with local partners and invite them to provide feedback prior to any outputs being finalised. After completion of the evaluation, we will also follow up with the Local Authority partner (around three months after the evaluation completes) to understand how recommendations are being taken forward and to scope opportunities for future research and/or scaling up their implementation.

Networks and organisations with interests in TIP will be a focus for wider dissemination. LiLaC routinely uses a range of formats to disseminate research findings and achieve impact. This includes media and social media; videos; infographics; participants’ and professional newsletters; as well as presentations to community groups and service providers.

Phase 2: The aims of the phase 2 evaluation are to: Determine the early impacts of the TIA initiative Explore the processes of implementation of TIAs within and across schools Understand change mechanisms underlying the TIA initiative Identify options for longer term evaluation of educational outcomes

Understand what is needed to sustain change over time. Public facing outputs (after phase 2 is completed) will include Research briefing/report for local authority audiences Peer reviewed paper in an academic journal Public output for wider school stakeholders (students, staff) Presentations to practitioner/academic networks/conferences