Aims/objectives
-
To evaluate whether TCM improves socio-emotional well-being among children as measured by the teacher completed Strengths and Difficulties Questionnaire (SDQ) cross-validated with direct observation, parental SDQ and child report on How I Feel About My School where available.
-
To evaluate whether TCM improves academic attainment as measured by teacher assessment of pupil progress (APP) cross-validated with standardised assessments and SATS where available.
-
To evaluate whether any improvements in well-being and attainment are sustained over the next two academic years.
-
To evaluate whether TCM reduces ‘burn out’ and improves self-efficacy and well-being among teachers using the Maslach Burn Out Inventory, the Teacher Self-Efficacy Questionnaire and the Everyday Feelings Questionnaire.
-
To evaluate whether TCM improves teacher’s classroom management skills using the behavioural management strategies reported by teachers in the Teacher Satisfaction Questionnaire.
-
To use qualitative methods to investigate how teachers apply the strategies suggested by TCM in the classroom and any factors that may influence this process, including: year group taught, school climate and additional support and advice available to them.
-
To evaluate both the utility of TCM to teachers in their practice one year after attending the course and how TCM is related to additional sources of behavioural support and school context.
-
To evaluate the cost and cost-effectiveness of TCM compared to teaching as usual at final follow-up.
-
To extrapolate the results from the randomised controlled trial (RCT) into young adulthood using decision analytic modelling and published data to explore the longer-term cost and cost-effectiveness implications of TCM compared to teaching as usual and to model potential cost savings in the longer term.
Setting
The setting is primary schools within Devon, Torbay and Plymouth.
Inclusion criteria
-
Teachers, parents and children in primary, state run, mainstream schools in Devon, Torbay or Plymouth with at least one single year group class of 15 or more pupils in Reception or Years 1–4. This will provide a sample of children aged 4–9?years at recruitment.
-
To be eligible, the nominated teacher must have classroom responsibility for a single year group class for a minimum of four days per week.
Exclusion criteria
-
Schools that have only mixed year group classes, all classes have fewer than 15 children, are under “special measures”, are privately funded or are without a substantive head teacher.
-
Teachers on contracts of less than three years.
-
Children with so little use of spoken English that they are unable to complete the measures, even with support.
-
Children whose parent(s) do not have a sufficient use of English to enable them to give consent for their child to participate or answer questionnaires, even with assistance.
Recruiting teachers and children from classes containing single year groups is vital to preserve the original allocation group status for follow up. Provided that we start with single year groups, the original control group children will then be with a new teacher in subsequent academic years allowing the control teachers to access TCM. It does not affect the trial design for children to graduate into mixed year group classes after the first year of participation, so schools only need to have one single year group class to be eligible to participate.
Design
Cluster randomised controlled trial
The core of the study will be a pragmatic cluster randomised controlled trial with one teacher and their pupils per primary school (cluster) allocated to TCM training or teaching as usual (TAU). The recruitment plan is dictated by the school academic year and the duration of the intervention (six months), which means that the TCM course can only run once per academic year. Eighty schools will be recruited over three year; 15 in Cohort 1, 30 in Cohort 2 and 35 in Cohort 3.
Each school will participate in the trial for three academic years. Child and teacher outcomes will be assessed at the beginning and end of the first academic year (T0 and T1). At the end of the first academic year, teachers and children will separate; the study children will have new teachers in each follow up year (T2 and T3), who will complete the child well-being measures even if a child moves school. The study teachers will also be working with a new class of children, which allows us to offer the control teachers TCM in the second year of participation (i.e. after recruitment) as an incentive for participation. Child outcomes will be measured in both follow-up years (T2 and T3) as part of the follow up trial design. Schools sometimes move teachers around year groups for school improvement purposes, so the situation may arise when a teacher and children participating in the trial may come back into contact with each other during the follow up period. Control teachers will be offered access to TCM in the second year of participation in the trial, so contact between control children and control teachers would serve to reduce any differences between the arms of the trial, while contact in the second and third years of participation between TCM children and TCM teachers may increase the differences between arms by effectively giving these children a double dose of the intervention. We will explain verbally and in writing to head teachers at recruitment why it is important to avoid a reunion between the teacher and their baseline class during follow-up and we will monitor schools closely to record if this does happen. To prevent contamination and to reflect how the intervention might be rolled out in ‘real life’, the unit of allocation is the school; one teacher per school will participate.
Embedded teacher cohort
Control teachers will have accessed the course by T2 and T3. As control teachers access the TCM course in the second year of participation, data relating to the teachers (self efficacy, burn out and mental health and well-being) will contribute to the trial outcomes at T1 only when there is a control group. We plan to combine data from teachers in the trial with the same data collected already from the four TCM groups that we have run as part of our Feasibility Studies. Consequently, we will be able to create an uncontrolled cohort of approximately 120 teachers (eight trial groups plus four Feasibility Study groups, each of 10 teachers) to provide additional data using their baseline and national norms for comparison. It is planned that teacher self-report outcomes will be recorded once a year for up to ten years using web-based data collection. This additional study will be the subject of an alternative funding bid.
Economic evaluation
The economic evaluation will take a broad public sector perspective, including the use of all health, education and social care services, plus criminal justice sector resources and criminal activity. Data will be presented by sector to allow alternative perspectives to be considered separately. Economic data will be collected at T0, T1, T2 and T3; at baseline, information will cover the previous 6 months; at follow-up service use since the previous time-point will be recorded. The cost and cost-effectiveness of TCM compared to TAU will be analysed at the final follow-up point (T3) in terms of the child’s socio-emotional well-being.
Process evaluation
The process evaluation will combine quantitative and qualitative methodologies, and data collection and analysis will run parallel to the pilot and main phases. Quantitative data relating to the administration and delivery of the TCM course will be collected to record how many sessions were attended by each teacher and to reflect course experience and the adoption of strategies. These will use routine methods of capturing data developed by the Incredible Years Team. Qualitative data will be collected to inform the trial processes and to assess translation of TCM strategies into practice and any impact of TCM on the use of services within schools.
Data collection
The data for the RCT and economic evaluation will be captured at specified time points. Process evaluation data will be collected at different times in parallel to the main study. Data collection will be via questionnaires (self-report and proxy) completed by teachers, children and parents; classroom observation by independent researchers; academic assessment by teachers and researchers; focus groups/telephone interviews with teachers, headteachers and Special Educational Needs Coordinators (SENCos), and telephone interviews with sub-sets of parents. Additional interviews with parents and children will be subject to a further funding bid if sufficient parents are willing to participate. Teachers will use a web-based electronic data capture system to complete all questionnaire measures on themselves and the children.
Randomisation and concealment
Schools will be randomised to intervention (TCM course) or control (Teaching As Usual, TAU) through a password protected trial website that will be set up and maintained by the UKCRC accredited Peninsula Clinical Trials Unit. Randomisation will be stratified by the following: level of deprivation at school level (below or above 19% of pupil’s eligible for free school meals); city/non city location (Plymouth/Exeter/Torbay versus other addresses); and school year (Key Stage 1 or Foundation and Years 1–2 versus Key Stage 2 or Years 3 and 4). To ensure concealment, all schools within each cohort of recruitment will be randomised simultaneously after the baseline measures have been completed.
We will be unable to blind the staff in schools as to which group they are allocated to. Researchers undertaking the observations of teachers will be external to the core research team and will be kept blind to group allocation at all times, although it is possible that teachers may inadvertently disclose this information to them. We will ask these researchers to guess which groups the teachers that they observed were in at the end of their follow up observations to check if blinding was maintained and to tell us if a teacher informed them about their allocation. Baseline measures will be completed before randomisation and therefore all parties will be blind to allocation at this point. Parents and children are unlikely to be aware of whether their child’s teacher has completed the TCM course and the follow-up measures, with the exception of the service-related interviews completed by parents on a sub-sample and the child measures, are questionnaires that are completed independently and thus difficult for the core team of researchers to influence should they become aware in their liaisons with school about allocation. In addition, the teacher-completed follow-up measures in the second and third year of each school’s participation in the study (T2 and T3) will be completed by a teacher that did not access the intervention, although they are likely to know whether their colleague did or not.
Assessment and follow up
The trial will start two weeks into the new school year in September. Parents will have two weeks to opt themselves and their child out of the trial. Baseline assessments (T0) will be completed by the October half term holiday. The measures will be completed by the child (on themselves), their parent (on themselves, their child and their child’s use of services) and the class teacher (on the children and themselves). Direct observations in the classroom will take place with a subsample during this time. Following randomisation, the intervention group teachers will then start the TCM course in early November. Literacy/numeracy assessments with a subsample of children will take place in the February/March.
The first follow-up assessment, identical to T0, will be completed in May/June of that first academic year (T1 or nine months post baseline), with the classroom observations completed by researchers blind to allocation.
Each school participates in the trial for three academic years. Only the measures relating to the children (including SDQs completed by their subsequent teachers in the two follow up years) contribute to trial follow-up data at T2 (18 months post baseline) and T3 (30 months post baseline). The teachers will complete the measures relating to themselves to assess the impact of TCM on professional functioning in the longer term, but at T2 and T3 these will no longer contribute to trial outcome data as described earlier. As the recruitment is rolled out over three years, the assessment of different cohorts will be carried out simultaneously within the same school year.
Trial outcome measures
The primary outcome is child well-being and mental health measured by the teacher-completed Strengths and Difficulties Questionnaire, but this will be supplemented with a number of other measures of child mental health and behaviour. The secondary outcomes are: child attainment and teachers stress, burn out and professional self-efficacy.
Teacher completed measures on each child
Strengths and difficulties questionnaire (SDQ [18])
The teacher-rated version of the SDQ is the primary outcome measure. The SDQ is a brief, valid and reliable measure of socio-emotional competence that is widely used to assess mental health in childhood. It will be completed by teachers and by parents at all four time-points. The subscales, behaviour, emotions, overactivity/concentration, peer relationships and prosocial behaviour, will allow the examination of particular aspects of well-being in isolation. Responses to the first four add to give a Total Difficulties score. Ratings of child distress and impact of difficulties on home life, friendships, classroom learning, and leisure activities combine to form the Impact Scale. Teacher SDQs will be cross-validated with parental SDQs, direct observation and the child view of school measure described below.
Assessments of pupil progress (APP [19])
APP will be used as the measure of child academic attainment. The APP is completed by all teachers routinely in accordance with detailed guidelines related to the National Curriculum [19] and is a structured approach to periodically assessing children’s level of attainment in mathematics, science, reading, writing and speaking and listening. It enables teachers to track pupils' progress from Year 1 through to the end of year 6. Levels range from eight P levels (working towards Level 1) to Level 5 (above average expectation for a child at the end of year 6); levels 1–5 have three sub-levels (a-c). Using the APP allows us to gather data on academic attainment on all participating children without additional work for teachers and researchers, while the APP approach has proven to be robust, manageable and reliable in practice [19]. APP scores will be supplemented by SATs, which are scored using the same classification as APP, where SATS results are available. Both will result in ordinal data and children are expected to make two points difference on the 12 points of the scale that the age-group under study will reach. Reliability will be further assessed using detailed psychometric tests (WIAT II [20]) in a subsample (see below).
Adapted pupil behaviour questionnaire (PBQ)
The PBQ was developed for and used extensively in school effectiveness studies, and is based on findings of the Elton Report [21]. It measures the types of classroom-based disruptive behaviours of particular concern to school staff. Teachers will complete the PBQ for all children in their class. The adapted version contains six items scoring 0 = never, 1 = occasionally, 2 = frequently. Items are summed with a higher total score indicting more disruptive behaviour.
Teacher completed measures on themselves
Teachers’ sense of efficacy scale [22]
A 12 item measure assesses the teacher’s perception of their sense of effectiveness as a teacher on three subscales (each with 4 items): Student Engagement, Instructional Practice and Classroom Management. Response is on a nine point scale for each item with anchors at 1 = nothing, 3 = very little, 5 = some influence, 7 = quite a bit and 9 = a great deal. Mean scores with a range of 4–36 are calculated for each scale with a higher score indicating a greater sense of efficacy.
Maslach burnout inventory- general survey [23]
A 16 item measure assesses aspects of ‘burnout syndrome’ which are recorded on three separate subscales: Exhaustion, Cynicism and Professional Efficacy. Respondents choose from seven options ranging from 0 = never, 1 = sporadic, 2 = now and then, 3 = regular, 4 = often, 5 = very often, 6 = daily. Mean scores are calculated for each subscale. A high degree of burnout is reflected in high scores on Exhaustion and Cynicism and low scores on Professional Efficacy.
Everyday feeling questionnaire (EFQ) [24]
A 10 item measure which records well-being over the previous four weeks. Half of the items focus on well-being and half on distress. Items are scored 0–4 for items with distress content and 4–0 for items with wellbeing content, with a maximum score of 40, with a higher score indicating increased distress.
Child completed measures
How I feel about my school [25]
Our group has developed and tested a measure of children’s attitude towards school. We recruited 268 pupils aged 4–7 years from three schools, who completed the seven-item How I Feel About My School[25] questionnaire on two occasions, two weeks apart. Internal consistency was satisfactory (Cronbachs alpha =0.62 at Time 1, 0.67 at Time 2), with good test-retest reliability (intraclass correlation coefficient (ICC) = 0.63), and there were small but statistically significant correlations with parental reports on a parallel measure (Pearson correlation coefficient = 0.22 at Time 1 and 0.20 at Time 2). Children select one of the following responses for each item: sad (0), OK [1], happy [2], with a higher score indicating greater happiness at school. The potential range of the total score is 0–14 with a higher score indicating great enjoyment of school.
Parent completed measures
Strengths and difficulties questionnaire (SDQ [18])
Parents will also complete the parent rated version of the SDQ about their child at the four time points.
Observer completed measures
A random sample of schools will be chosen to complete these measures in order to validate the findings using the briefer questionnaire measures; they are not primary or secondary outcome measures for the trial. However, there will be some practical considerations that will restrain which schools we can enter for observations; some schools may refuse, and others may not have additional rooms available for the individual child assessments to take place. We will compare the schools that we visit with those that we cannot in detail to search for any potential biases, but as we are comparing one source of data with another on the same children, we do not anticipate that selection bias will have a major influence on our results. The observations will be completed by researchers who are independent to the core research team.
Wechsler individual achievement test (WIAT II-UK) [20]
The WIAT-II (Wechsler Individual Achievement Test – Second edition) is a psychometric assessment which measures reading, numerical attainment and language attainment in children from the age of 4. It is a psychometric assessment that is administered individually; it takes between 60–90 minutes depending on the child’s age and ability. It allows an assessment of the child’s functioning in these areas to be compared to national norms to determine the child’s achievement and ability in relation to other children their age.’ We will use the WIAT data to supplement the data gathered by the APP among a sub-sample of 50 children. Sub-tests of the WIAT II have been chosen to map onto the APP [18] for reading, spelling and maths and include Word reading, Reading comprehension and Spelling for literacy and numerical operations and mathematical reasoning for numeracy.
Teacher-pupil observation tool (T-POT [26])
Teacher-child interactions will be directly observed in a sub-sample of 20 classrooms (25% of teachers) using the TPOT. This is a structured real-time frequency count of defined teacher behaviours and types of teacher-child interaction that will be carried out by observers blind to allocation. Inter-rater reliability with two or three observers rating 21 primary school teachers was high (ICC = 0.78) [25]. The focus of the observation is the class teacher.
The T-POT uses continuous coding to look for nine different teacher behaviours and seven different behaviours from the children in the class. It measures behaviours that the TCM intervention specifically targets for change and therefore will be able to assess whether the teachers’ and children’s behaviour changes between T0 and T1. ‘Teacher negatives’ include: physical behaviours such as restraining/moving the child; verbal behaviours such as reprimanding the child; and not being explicit to a child about the behaviour that is expected. TCM aims to provide teachers with strategies to enable them to use more positive approaches and therefore reduce the need for these negative behaviours/verbalisations. This therefore will be measurable pre- and post-intervention at T0 and T1. The T-POT also looks at ‘ teacher positives’, which include praise, positive physical contact, positive facial expressions and verbalisations (e.g. laughing).
All teachers have their own unique style and therefore there are no ‘cut-off’ points to indicate good or bad practice, instead the T-POT encourages comparing change between two observations, particularly in relation to ratios of positive to negative behaviours. Scores on the T-POT will be compared to the relevant items on the classroom management and instructional practice subscales of the Teacher sense of efficacy scale, and the teachers report of the TCM strategies adopted at the end of the course for those in the intervention arm.
T-POT can also assesses a range of ‘child positives’ and ‘child negatives’ to assess whether there is any potential impact of TCM on child behaviour in the classroom, but in STARS, due to time and financial restraints, the focus of observations is on the teacher behaviour.
Economic resource-use data
Parent completed measures
Child and adolescent service Use schedule (CA-SUS) [27–29]
Resource-use information will be collected using the Child and Adolescent Service Use Schedule (CA-SUS), developed by one applicant (SB) in previous economic evaluations involving child and adolescent mental health populations [27–29]. Two versions of the CA-SUS will be used. Firstly, a brief self-report version to collect data on a limited set of key resource items (high cost and/or high use) from all parents at all four time points. Second, the full standard interview version of the CA-SUS will be used with a random sample of 50 parents in interview at T2 and T3 in order to validate and supplement the briefer self-complete version at all time points.
Data from educational records
Parent data on service use will be supplemented with data on educational service use at pupil level collected from schools.
Process evaluation data
The process evaluation will include quantitative and qualitative methodologies; data collection and analysis will run parallel to the pilot and main phases.
Quantitative data
Data will be routinely collected relating to the administration of the TCM course. Group leaders complete standard checklists after each session that indicate which parts of the expected curriculum were covered. Standardised session evaluations and self-monitoring checklists are completed by teachers after each session to assist group leaders in planning, with a satisfaction questionnaire after the final session that collects data on the teachers’ application of the techniques covered in the course. TCM sessions will be filmed for supervision with the TCM programme developers, which allows the research team to analyse the videos for fidelity to model and contextual factors in each group. There will be eight TCM groups in total, each of 10 teachers, by the end of the trial with an additional four groups of 10 teachers from the two feasibility studies. This routinely collected data will be supplemented with data on recruitment, attendance and engagement with TCM, and with the qualitative data to provide contextual information on which to base recommendations about how TCM should be implemented successfully.
Qualitative data
Qualitative data will be collected using focus groups and semi-structured telephone interviews at different times throughout the study.
Focus groups will be used to collect data on the learning, uptake and use of TCM techniques in the classroom and informal transference to other staff members. In addition, a follow-up focus group with teachers, one year after completion of the TCM course will explore the maintenance of TCM techniques. We will undertake telephone interviews with Head teachers and SENCos in the second and third year of participation to collect data about differential use of support services, attribution to the teacher being TCM trained and perception of the place of TCM among other available sources of support.
Topic guides will be developed for both the focus groups and semi-structured (telephone) interviews. Interviews will be audio-recorded and transcribed.
Should time permit, we will undertake exploratory semi-structured interviews and/or focus groups with parents to explore their hopes and priorities for change in relation to their views and experiences of teacher classroom behaviour management and the promotion of mental health and well-being at school. This will, however, comprise a separate study with its own protocol and is mentioned in this protocol only because we will ask parents to indicate if they would be interested in participating as part of the feasibility work for the additional study.
Other data
Parent reported
Parents will provide basic socio-demographic information about themselves and their child at baseline, and will include the following demographic details: child’s eligibility for free school meals, post code to link to the index of multiple deprivation, the number of children living in the household, housing tenure (rented or not), and the highest level of qualification of the parent(s) or carer(s).
School reported
We will gather school level data on the percentage of children eligible for school meals at recruitment and the index of multiple deprivation at lower super output area as a proxy for the school catchment area according to the school’s postcode [30]. We will also obtain information from schools about the type and level of emotional enrichment programmes (e.g. Socio-emotional aspects of learning, Thrive) being delivered in school, and how much other outside behavioural support they receive.
Proposed sample size
Randomised controlled trial
Forty schools (clusters) will be randomised to each of the intervention and control arms, using one class from each school. Assuming that each class contains 30 pupils and that the recruitment rate is 70% (achieved among parents in the Helping Children Achieve trial [31] using the SDQ) we anticipate that 21 (i.e. 30*0.7) children from each class and a total of 840 (i.e. 21*40) children in each trial arm will participate in the study. Assuming 10% attrition for the children, we expect 19 of them to be followed-up at T3 in each class: a total of 760 (i.e. 19*40) children followed-up at T3 in each trial arm. As clusters are randomised the sample size calculation takes account of the correlation between participants’ responses within clusters. The intra-cluster correlation coefficient (ICC) for the primary outcome measure (SDQ total difficulties score) was estimated to be 0.15 using data from [Sayal et al 32]. Using the formula VIF = 1 + (n - 1)* ICC presented in [Donner and Klar 33], the variance inflation factor (VIF) is 3.7 (i.e. 1 + [1]*0.15). The study will therefore be equivalent to a trial in which 205 (=760/3.7) participating pupils were individually randomised and provides 85% power at the 5% level of significance to detect a difference in the mean SDQ score between trial arms equivalent to an effect size of 0.3 of a standard deviation or a difference of 2 points on the raw SDQ scale. This would reduce the percentage of children classified in the borderline/abnormal range from 20% to 14% (http://www.sdqinfo.org/UKNorm.html) where borderline/abnormal is defined as those scoring 12 and above out of 40. Data from Goodman & Goodman [34] suggest that the odds of psychiatric disorder decrease by 33% for each 2 point decrease in the teacher SDQ and by 40% for each 2 point decrease in the parent SDQ.
Nested qualitative study within process evaluation
Sampling within the process evaluation will be purposive [35, 36] to facilitate data collection of the views and experiences of participants, who can comment on the delivery, uptake and use of TCM strategies, appropriate to each phase of the trial. All intervention teachers in the trial will be invited to take part in focus groups. Sampling of Head-teachers and SENCos will reflect the aims of each trial phase.
Course experience and research processes
The aims of the process evaluation for the first cohort are to elicit a fuller understanding of the experiences of the course, course delivery and research process. All intervention group teachers (n = 10) will be invited to join a focus group after the course finishes. All head teachers from intervention group schools will be invited to take part in a telephone interviews (n = 10). Head teachers from control group schools (n = 5) will also be invited to take part in a telephone interview to elicit their views on the research processes.
Teacher learning and use of TCM strategies
In cohorts 2 and 3, all teachers in the intervention groups (15 in each year) will be invited to join a focus group aiming to elicit views and experiences of the learning, uptake and use of TCM strategies in the classroom. For teachers in their follow up year (i.e. second and third years of participation) all teachers from the previous year’s course will be invited to join a focus group to elicit views on maintenance of the use of TCM techniques in the classroom.
Impacts of course
In each of second and third years of participation, we plan to conduct interviews with up to 15 head teachers and 15 SENCos from the intervention schools In this phase we will aim to achieve a diversity of head-teachers and SENCos from a range of schools [36].
Although not the only issue affecting sample size in qualitative research, a guiding principle includes the concept of saturation [37, 38]. Our sample size takes account of current guidance on optimising sample size in qualitative research.
Statistical analysis
Analysis of effectiveness
All comparisons between trial arms will use the intention to treat principle where schools and participating pupils are analysed according to the arm to which they were randomised. Random effects linear regression models [39] will be fitted to compare means for continuous outcomes (including the primary outcome or SDQ total difficulties score) between the trial arms allowing for the correlation between outcomes of children from the same school specifying school effects as random. The method of marginal models using Generalised Estimating Equations with information sandwich (“robust”) estimates of variance and assuming an exchangeable correlation structure within school clusters [40] will be used to compare binary outcomes (e.g. borderline/abnormal versus normal status on the SDQ) between the trial arms, also allowing for clustering. A test of interaction will be implemented for each outcome to investigate whether the intervention effect differs across the three time points (T1, T2, T3).
Unadjusted analyses and analyses adjusted for important prognostic factors at the pupil level (e.g. child gender, year group and baseline SDQ score), cohort of recruitment and the school level (level of deprivation, urban versus rural status and whether involved in other emotional enrichment programmes) will be implemented. In a secondary analysis, interaction terms will be included to investigate possible differences in intervention effect (on the primary outcome SDQ score only) between pre-defined subgroups based on school and individual deprivation, low versus high baseline SDQ scores, length of teacher experience and year group. These sub-group analyses have been selected for a number of reasons. First, children experiencing socio-economic deprivation may benefit more than their more privileged peers. Second, previous work suggests that children with higher levels of difficulties experience the most benefit [10]. Third, teachers in our Feasibility Study suggest that newly-qualified teachers would gain the most from the course, and finally, there is a common belief that these interventions will have the biggest impact on younger children. The latter belief has focused research on very young children and this would be the only study to investigate TCM in children aged over seven years. P-values of 0.01 and less will be interpreted as providing evidence for interaction effects. Although the power to detect moderate subgroup interactions will be low, we are primarily interested in investigating the possibility of large quantitative interactions and not qualitative interactions where the direction of intervention effect differs between sub-groups. Demographic and baseline characteristics at the school and pupil level will be summarised using means and standard deviations (or medians and inter-quartile ranges) for quantitative characteristics and percentages for categorical characteristics.
Cost and cost effectiveness
TCM costs will be calculated using a standard micro-costing (bottom-up) approach [41], and will be based on teacher and trainer salaries plus on-costs (employers national insurance and superannuation contributions) and appropriate capital, administrative and managerial overheads. Costs for NHS hospital contacts will be taken from NHS reference costs [42]. Nationally applicable unit costs will be applied to all community health and social care contacts [43], medications [44], crimes and criminal justice resources [45, 46]. The costs of schooling and school based services will be taken from various sources including Ofsted reports (the UK inspectorate and regulatory body for schools in England; http://www.ofsted.gov.uk) and published documents [47, 48].
Despite the often skewed nature of costs, mean costs will be compared using standard parametric tests and the robustness of the results confirmed using bootstrapping [49]. The advantage of this approach, as opposed to logarithmic transformation or non-parametric tests, is the ability to make inferences about the arithmetic mean, which is more meaningful from a budgetary perspective [50].
The primary economic evaluation will explore cost-effectiveness at the T3 follow-up. Cost-effectiveness will be measured initially in terms of the primary outcome measure (SDQ). Cost-effectiveness will be assessed using the net benefit approach [51]. Uncertainty around the cost and effectiveness estimates will be represented by cost-effectiveness acceptability curves [52, 53]. A joint distribution of incremental mean costs and effects for the two groups will be generated using non-parametric bootstrapping to explore the probability that each of the treatments is the optimal choice, subject to a range of possible maximum values (ceiling ratio) that a decision-maker might be willing to pay for an additional unit of outcome gained. Cost-effectiveness acceptability curves will be presented by plotting these probabilities for a range of possible values of the ceiling ratio [54]. These curves are a recommended decision-making approach to dealing with the uncertainty that exists around the estimates of expected costs and expected effects associated with the interventions under investigation and uncertainty regarding the maximum cost-effectiveness ratio that a decision-maker would consider acceptable [53, 55]. To explore the longer-term implications of TCM, data from the RCT will be extrapolated and supplemented with data from the literature using decision analytic modelling techniques [56], in line with methods used to model the long-term impacts of parenting interventions for the prevention of persistent conduct disorders in children [57].
Qualitative analysis
All audio-taped qualitative data will be transcribed verbatim and anonymised. Data will be stored using Nvivo software http://www.qsrinternational.com/products_nvivo.aspx and will be password protected. Analysis will be guided by a realist perspective, to identify experiences as the lived ‘reality’ of participants, but we are also interested in the ways in which participants account for their experiences within the context of the trial and their own schools based experiences [58]. Thematic analysis of interview and focus group data will be both theoretically driven by the research questions and allow for more inductive analysis whereby emergent themes are also identified. This mixed approach will help explicate patterns of experience and views of teachers, head-teachers and SENCos. As highlighted previously, analytical interests in the study vary across the different trial data collection periods. During the first year, analysis will focus on the research processes (teachers and head-teachers), while during the last two years, the focus will on the TCM intervention, use in the classroom including identification of key contexts, influences and transference (teachers, head-teachers and SENCos). Analytical focus within the year following TCM course completion will be on maintenance of TCM skills in the classroom and differentials in use of services.
In our analysis, ‘keyness’ of themes does not relate to incidence of occurrence but to whether a theme captures information relevant to the research questions, in this case relating to a range of trial processes [59]. The Framework Approach [36] will be used to manage data and aid systematic analysis (description and summary of key themes, patterns and links in the data), allowing the researcher to move between levels of abstraction during analysis and between a theory driven and more inductive approach, while also displaying the relevant data sources. This approach will help maintain a focus on the process evaluation objectives for the different phases of the study.
Summary and illustrative data will be available, relevant to the aims of the qualitative research for each phase, to facilitate further interpretation and discussion of which processes worked well or not so well within the main trial. A number of methods will be adopted to enhance rigour during analysis including: checks for thematic saturation and consistency [37]; shared analysis; analytical discussions will be recorded; self-reflective memos will also be kept by researchers [60, 61] and we will undertake a deviant case analysis, which involves a re-interrogation of data searching for new themes not covered in the initial data analysis [61].