Supporting teachers and children in schools: the effectiveness and cost-effectiveness of the incredible years teacher classroom management programme in primary school children: a cluster randomised controlled trial, with parallel economic and process evaluations

Existing research

Childhood psychopathology is common and the prevalence of antisocial behaviour in particular has increased in recent years [1, 2]. Poor socio-emotional adjustment in early childhood increases the risks of psychiatric disorder, risk taking behaviour, educational failure, and involvement in crime in both childhood and adulthood [1, 2]. Affected children incur substantial costs to both society and their families [3]. The impairment and societal costs of antisocial behaviour, however, are evident throughout the population distribution rather than just among those with the highest level of problems [3]. Children in poorly managed classrooms observe that disruptive behaviour commands staff attention while good behaviour is rarely acknowledged, which may contribute to later disruptive behaviour and / or disaffection and disengagement from secondary school with the attendant risks to health and educational attainment. Teachers often complain of lack of training to manage disruptive behaviour, which is associated with higher stress levels and burn out [4–6]. An intervention that supports teachers to manage disruptive behaviour and promote socio-emotional competence could potentially benefit every child subsequently taught by that teacher as well as the teacher themselves, and could therefore be substantially more cost-effective than direct intervention with subsequent cohorts of children.

Despite a multitude of programmes targeting children [7], a recent systematic review identified that only two interventions focusing on enhancing teachers’ skills that had been studied more than once; [8] one of these was the Incredible Years (IY) Teacher Classroom Management (TCM) course. TCM was also the programme with the most robust evidence, despite the fact that there have been only two trials of TCM in isolation from other interventions [9, 10]. In a small observational study in Wales, 23 teachers reported high levels of satisfaction with the TCM course. Direct observation revealed that teachers who had accessed TCM gave clearer instructions, allowed more time for compliance and their pupils were more compliant [9]. A subsequent RCT involved 12 classes (16 teachers) from 11 primary schools and 107 children aged 4–5 years [9]. The results indicated weak evidence that TCM produced changes in teachers’ behaviour and their pupil’s behaviour [9]. A second trial in Limerick involved 11 schools, 22 teachers and 207 children aged 4–7 years and detected changes in teacher’s behaviour on both observation and self-report (. There was little evidence of change in teacher-rated child well-being [10]. They estimated the intervention to cost approximately €2000 per teacher or €100 per child in schools with a class size of 20. Neither trial has been published in peer reviewed journals. Follow-up into a second year in Wales has been completed but results are not yet available, while follow-up in the Irish study is ongoing. Both studies suggest that TCM is potentially sufficiently intense to change teachers’ behaviour. Other studies involving TCM have either added additional coaching for teachers or children, and/or studied the parallel parent and child programmes with or without TCM. All suggest that TCM is potentially effective but given the additional interventions, it is impossible to estimate the impact of TCM alone as a public health intervention [11–14]. We are unaware of any other planned or ongoing trials of TCM in isolation from other interventions.

The intervention: the teacher classroom management programme

The intervention being evaluated is the IY Teacher Classroom Management (TCM) programme [14]. TCM draws on cognitive social learning theory, particularly Patterson’s theories [15] about how coercive cycles of interaction between adults and children reinforce unwanted behaviour patterns, Bandura’s ideas [16] about the importance of modelling and self-efficacy, and Piaget’s developmental interactive learning methods [17]. In addition, it also incorporates strategies for challenging angry, negative, and depressive internal dialogue in adults whilst interacting with children, drawn from cognitive behavioural approaches. TCM is delivered to groups of ten teachers, and involves six whole-day sessions spread over a period of six months. It is delivered in a collaborative style with group leaders encouraging all to share their experience and expertise and to value that of others. The explicit goals are to: a) enhance teacher classroom management skills and to improve teacher-student relationships; b) assist teachers to develop effective group and individual behaviour plans to enable proactive rather than reactive classroom management; c) encourage teachers to adopt and promote social and emotional regulation skills; and d) encourage teachers to strengthen positive teacher-parent relationships with all parents.

TCM uses goal setting, reflective learning, video modelling, role play, rehearsal of novel management strategies, group discussion, support and problem solving, and cognitive and emotional self-regulation training. Teachers are encouraged to experiment with novel strategies between sessions and to discuss their resulting experiences. The intervention will be delivered by suitably trained and supervised ‘group leaders’. We envisage the same process of delivery should the trial support the implementation of TCM. The Incredible Years (IY) Foundation is collaborating to ensure that our group leaders deliver the intervention to the highest possible level and with fidelity to model. The comparator is teaching as usual (TAU). Plymouth, Torbay and Devon Local Authorities have a range of behavioural support and educational psychology services, and teachers in both arms will be able to access whatever other resources are available to them. Precisely what is accessed and how TCM might complement or supplement what is already available will be a major focus of the qualitative aspect of the process evaluation.

Aims/objectives

Setting

The setting is primary schools within Devon, Torbay and Plymouth.

Exclusion criteria

Recruiting teachers and children from classes containing single year groups is vital to preserve the original allocation group status for follow up. Provided that we start with single year groups, the original control group children will then be with a new teacher in subsequent academic years allowing the control teachers to access TCM. It does not affect the trial design for children to graduate into mixed year group classes after the first year of participation, so schools only need to have one single year group class to be eligible to participate.

Design

Data collection

The data for the RCT and economic evaluation will be captured at specified time points. Process evaluation data will be collected at different times in parallel to the main study. Data collection will be via questionnaires (self-report and proxy) completed by teachers, children and parents; classroom observation by independent researchers; academic assessment by teachers and researchers; focus groups/telephone interviews with teachers, headteachers and Special Educational Needs Coordinators (SENCos), and telephone interviews with sub-sets of parents. Additional interviews with parents and children will be subject to a further funding bid if sufficient parents are willing to participate. Teachers will use a web-based electronic data capture system to complete all questionnaire measures on themselves and the children.

Randomisation and concealment

Schools will be randomised to intervention (TCM course) or control (Teaching As Usual, TAU) through a password protected trial website that will be set up and maintained by the UKCRC accredited Peninsula Clinical Trials Unit. Randomisation will be stratified by the following: level of deprivation at school level (below or above 19% of pupil’s eligible for free school meals); city/non city location (Plymouth/Exeter/Torbay versus other addresses); and school year (Key Stage 1 or Foundation and Years 1–2 versus Key Stage 2 or Years 3 and 4). To ensure concealment, all schools within each cohort of recruitment will be randomised simultaneously after the baseline measures have been completed.

We will be unable to blind the staff in schools as to which group they are allocated to. Researchers undertaking the observations of teachers will be external to the core research team and will be kept blind to group allocation at all times, although it is possible that teachers may inadvertently disclose this information to them. We will ask these researchers to guess which groups the teachers that they observed were in at the end of their follow up observations to check if blinding was maintained and to tell us if a teacher informed them about their allocation. Baseline measures will be completed before randomisation and therefore all parties will be blind to allocation at this point. Parents and children are unlikely to be aware of whether their child’s teacher has completed the TCM course and the follow-up measures, with the exception of the service-related interviews completed by parents on a sub-sample and the child measures, are questionnaires that are completed independently and thus difficult for the core team of researchers to influence should they become aware in their liaisons with school about allocation. In addition, the teacher-completed follow-up measures in the second and third year of each school’s participation in the study (T2 and T3) will be completed by a teacher that did not access the intervention, although they are likely to know whether their colleague did or not.

Assessment and follow up

The trial will start two weeks into the new school year in September. Parents will have two weeks to opt themselves and their child out of the trial. Baseline assessments (T0) will be completed by the October half term holiday. The measures will be completed by the child (on themselves), their parent (on themselves, their child and their child’s use of services) and the class teacher (on the children and themselves). Direct observations in the classroom will take place with a subsample during this time. Following randomisation, the intervention group teachers will then start the TCM course in early November. Literacy/numeracy assessments with a subsample of children will take place in the February/March.

The first follow-up assessment, identical to T0, will be completed in May/June of that first academic year (T1 or nine months post baseline), with the classroom observations completed by researchers blind to allocation.

Each school participates in the trial for three academic years. Only the measures relating to the children (including SDQs completed by their subsequent teachers in the two follow up years) contribute to trial follow-up data at T2 (18 months post baseline) and T3 (30 months post baseline). The teachers will complete the measures relating to themselves to assess the impact of TCM on professional functioning in the longer term, but at T2 and T3 these will no longer contribute to trial outcome data as described earlier. As the recruitment is rolled out over three years, the assessment of different cohorts will be carried out simultaneously within the same school year.

Trial outcome measures

The primary outcome is child well-being and mental health measured by the teacher-completed Strengths and Difficulties Questionnaire, but this will be supplemented with a number of other measures of child mental health and behaviour. The secondary outcomes are: child attainment and teachers stress, burn out and professional self-efficacy.

Economic resource-use data

Process evaluation data

The process evaluation will include quantitative and qualitative methodologies; data collection and analysis will run parallel to the pilot and main phases.

Other data

Proposed sample size

Statistical analysis

Recruitment

There are 67 primary schools in Plymouth, 31 in Torbay and 314 in Devon (total 412), which provide plenty of opportunity to recruit the 80 schools required for the trial. The recruitment of schools will be staggered over a three year period, with 15 schools recruited in the first cohort, 30 schools in the second cohort and 35 schools in the third cohort. A recruitment strategy will be devised, and recruitment targets and rates continually monitored to ensure adherence to the trial plan. The Devon, Plymouth and Torbay Associations of Primary Headteachers are supporting the study and are taking an active role in publicising the study to their members.

Retention

Our calculations indicate that even if as many as six schools (clusters) from each arm withdrew from the study (i.e. we retained 34 in each arm) then the study would still have 80% power at the 5% level of significance to detect our specified effect of 0.3 standard deviations on the total SDQ difficulties score. In order to maximise retention, we will update participating schools, families and other key stakeholders about the study’s progress through termly newsletters as well as feeding into the newsletters of relevant organisations such as the Devon, Plymouth and Torbay Associations of Primary Head Teachers. Parents will be offered a voucher in recognition of their time spent on the completion of questionnaires and interviews and control teachers will be offered TCM if they provide data at T0 and T1. Teachers will also be offered incentives for questionnaire completion at each data collection point. As a result of feedback from our first Feasibility Study, we will recommend that head teachers do not nominate teachers who have additional management roles that may compete with TCM for time away from the classroom. The upper age limit of recruitment in Year 4 classes means that the follow-ups can be completed while even the older children remain at their primary school. In order to make their participation in the study as straightforward as possible and resolve any issues quickly, each school will have a named researcher to build a collaborative relationship.

Data management

All confidential data will be held in accordance with the Data Protection Act. Each school and participant included in the trial will be assigned a unique identifier and all data will be stored without identifying details. Data will be held on a secure database on a password-protected computer at the Peninsula Medical School. Access to data will be restricted to the research team. All trial documentation will be retained for ten years. The Peninsula Clinical Trials Unit (PenCTU) will develop the study database in addition to carrying out randomisation. It is envisaged that the teachers will complete the questionnaire measures on themselves and the children using a web-based tool, which will also be designed and maintained by the PenCTU.

Project timetable and milestones

The timing of recruitment, assessment of participants and delivery of the intervention is dictated by the school academic year between September and July. Milestones for the trial have been set according to this criteria and a rolling programme of recruitment will take place over three academic years. Three cohorts of schools will be recruited over the first three years of the study. Follow-up of these cohorts will overlap with recruitment and will take place between the second and fifth years of the study. Data entry and process evaluation will be ongoing, and economic evaluation and analysis will take place between the second and fifth years of the study. The trial data analysis and report writing will take place during the fifth year.

Trial management group

The senior management team (TF, SL, SB, OU, SS, BN, VE) will meet on a bi-monthly basis to review progress and set targets.

Trial steering committee (TSC)

The TSC will be chaired by Professor Paul Stallard, a clinical psychologist who has led several trials of interventions in schools, and will also include members of our User Advisory Group with representatives from specialist educational professionals, teachers and parents. The TSC will meet at the beginning of the trial and annually thereafter to oversee its conduct.

Data monitoring and ethics committee (DMEC)

An independent DMEC will consist of an independent statistician (Chair), a mental health and/or educational practitioner and an academic researcher. The DMEC will meet annually and if necessary in response to any serious untoward incidents to ensure that recruitment and retention is sufficient for the trial to continue and to review the analytic plan. The DMEC will report to the TSC in the first instance.

The University of Exeter will act as the Sponsor for the study. STARS will be hosted in the Child Health Research Group, which has experience in the successful delivery of community based paediatric trials.

Consent to participate

The nested qualitative study is likely to elicit sensitive and confidential data, and attention to ethics and participant confidence in and acceptance of researchers is crucial. Information about the process evaluation will be included in trial information sheets to teachers (focus groups) and headteachers and SENCos (telephone interviews). Written consent for participation in focus groups or individual telephone interviews will be sought at the same time as consent to take part in the trial. A reminder and additional information on focus groups will be given to teachers towards the end of the TCM course. Head teachers and SENCos will be sent a reminder and information about the telephone interview process in advance of the interviews. Consent will be checked verbally at the beginning of interviews and will be recorded.

Assessment of harms and adverse effects

The main indicators of harm will be the questionnaires completed by the children, parents and teachers at all four time points. Questionnaires will be screened for signs of severe distress. This will be defined in relation to teachers as a report of the most negative views of the future and self, and being completely unable to enjoy life in the Everyday Feeling Questionnaire. A child will be defined as severely distressed if they report feeling sad in response to all seven questions of the How I Feel About My School [24] measure. Parent and Teacher SDQs at follow-up will be screened for reports of their child’s problems being “much worse”. Responses meeting the above criteria will be reported to TF or a nominated deputy on the same working day; TF is a clinical mental health practitioner and will contact the family, school or local safeguarding practitioners as necessary.

Any concerns detected in this way will be recorded on a standardised pro forma, a copy of which will be sent to the Data Monitoring and Ethics Committee (DMEC) who will assess how likely it is that the adverse event is related to the conduct of the trial or TCM and advise if they consider additional action should be taken. The same process will be activated in response to any concerns raised by participants at other times, either spontaneously or during the focus groups and interviews; the latter will actively seek evidence of adverse effects.

Publication plan

Liaison with parents, practitioners and policy makers

Dissemination of our findings to all those working in relation to child mental health and education is essential. We have detailed plans for reaching the child mental health and the children and young people’s services communities and educational practitioners, academics and policy makers. Reports from all components of the project will be made available to NHS commissioning bodies and we will publish in peer reviewed academic journals to span both education and mental health publications as described above.

Key professional groups in relation to education include, but are not limited to, teachers (including head teachers, SENCo’s, learning mentors, behavioural support teachers and teaching assistants), educational psychologists and behavioural support teachers. In order to engage with these groups, we will offer presentations of our findings at key national and regional meetings, including those of the Association of Educational Psychologists and the British Psychological Society, the British Educational Research Association conference in the UK and the American Academy of Child and Adolescent Psychiatry conference in the US and the National Association of Special Educational Needs and the Social Emotional and Behavioural Difficulties Association conferences in the UK. In addition, we will seek to present through the national network of the Association for Child and Adolescent Mental Health, whose branches are multidisciplinary and include both health and education. We will offer presentations of the findings to voluntary agencies and support groups involved with child mental health, such as Young Minds, Action for Children, and the Mental Health Foundation.

We also plan to feed our findings back to the Teaching Agency within the Department for Education, which has a number of initiatives to improve the specialist training of teachers following the Lamb Enquiry Report [62]. One strand of these initiatives focuses on socio-emotional and behavioural difficulties. Continuing professional development may offer opportunities to engage with education-based practitioners. In most areas there are local academic networks for different educational professional groups, such as special educational needs coordinators, lead behaviour professionals and head teachers that link with regional and national networks and would allow further dissemination of information through meetings, bulletin boards and newsletters. Should the trial lead to clear recommendations that this training should be provided to teachers, we will seek to link with the head of the Schools Inspection Service in order to explore how our findings could lead to the incorporation of appropriate standards into the inspection regime.