School and participant characteristics
School characteristics
Information will be collected for each school including the Index of Community Socio-Educational Advantage (ICSEA) [54] metropolitan, regional, rural or remote location as determined by DET, number of enrolments and region (NW or SW).
Participant demographic characteristics and professional experience
Basic demographic information will be collected via survey, including participant age, school postcode, highest level of education, languages spoken at home, the number of years teaching and/or working within primary schools (school staff only), previous participation in child mental health and wellbeing related training courses and professional development (school staff only).
Primary outcome and measure
Teacher confidence in supporting mental health
The School Mental Health Self-Efficacy Teacher Survey (SMH-SETS) [44] will be completed by classroom teachers. The SMH-SETS is a psychometrically sound 15 item measure that utilises a 6-point Likert scale that addresses teacher’s confidence in their ability to successfully support students’ mental health with excellent internal consistency (Cronbach’s alpha = .91), item reliability and validity [55].
Secondary outcomes and measures
Perceived level of support for student mental health
To evaluate the impact of the MHWC model on perceived levels of support for student mental health, study designed questions will be developed to capture (a) how much support staff expect in managing student mental health and wellbeing from within their school and DET, and (b) how supported staff have felt in the past month from within their school and DET.
Mental health literacy
To evaluate the perceived level of knowledge and confidence in identifying child mental health and wellbeing issues, study designed items will be developed and rated on a 6-point Likert scale from strongly agree to strongly disagree. Study devised multiple choice items requiring respondents to choose the correct answer will also be developed to capture knowledge about internalising and externalising mental health issues in students.
To evaluate skills and confidence related to mental health and wellbeing of students, school staff will complete a skills assessment at T1, T5 and T6 (see Table 2). The skills assessment will comprise case study vignettes covering a range of potential scenarios school staff may be presented with, and respondents will be asked about key behavioural issues, contributing factors, strategies and further support required, as well as confidence in supporting the child described in the vignette.
To evaluate parents/carers child mental health literacy, items will be drawn from the Child Health Poll – child mental health edition [45]. The Child Health Poll includes items that address parent confidence in recognising and managing child mental health issues (measured on a 3-point Likert scale: confident, somewhat confident, not confident) and items that address knowledge about child mental health (measured on a 6-point Likert scale: strongly agree to strongly disagree).
Child mental health stigma
To evaluate changes in attitudes and stigma the Attitudes about Child Mental Health Questionnaire (ACMHQ [46];) will be administered. It consists of 30 items with four subscales: child dangerousness/incompetence, general stereotypes, community devaluation/discrimination and personal attitudes. Each item is rated on a 6-point Likert response scale that ranges from 1 (strongly disagree) to 6 (strongly agree). The ACMHQ subscales have good to excellent internal consistency (α =0.78–0.96) [46].
Unmet mental health and wellbeing need
To evaluate changes in the level of unmet need for mental health and wellbeing support within school classrooms, study devised questions will be used to capture (a) the proportion of students within each teacher’s classroom who have unmet needs for school-based mental health and wellbeing support and (b) the barriers to obtaining mental health and wellbeing support for these students.
Child mental health and wellbeing
To evaluate the impact of the MHWC model on child mental health, the Strength and Difficulties Questionnaire (SDQ) will be used [47]. The 25-item measure is a brief emotional and behavioural screening questionnaire for children aged 4–10 years old. There are 5 subscales with 5 items in each; emotional symptoms, conduct problems, hyperactivity/inattention, peer relationships and prosocial behaviour. The SDQ has good concurrent and predictive validity, and satisfactory internal consistency [47].
Prioritisation of student mental health
To evaluate the level of prioritisation of child mental health and wellbeing, school leaders will answer questions exploring the level of priority given to wellbeing and mental health provision for students, whether wellbeing and mental health provision for students is part of the school strategic plan or annual implementation plan, or if the school has a policy related to child mental health and wellbeing.
Engagement with mental health support
School staff engagement with school-based and external child mental health services will be measured using study designed questions to capture the perceived level of engagement required, the actual level of engagement and the types of child mental health support and services provided to staff.
Student engagement with mental health support and perceived availability of mental health and wellbeing support will be measured using subscales from the Attitudes to School Survey (AToSS) years 4–6 version [48]. The AToSS is a DET administered survey that will be completed by students and consists of 60 items across 6 domains with 3–5 factors/subscales within each domain. The following subscales will be used: Teacher concern; School connectedness; Advocate at school; Managing bullying; Peer relationships; Perceptions of school; Social outcomes; Subjective physical & mental health; Experience of COVID-19. The survey items use a five-point Likert scale from 1 (strongly disagree) to 5 (strongly agree).
Parent engagement with school-based and external child mental health and wellbeing support will be measured using study designed items and items from the DET Parent Opinion Survey (POS) [49]. The following subscales from the POS will be used: Student agency and voice; Confidence & resiliency skills; Managing bullying; Promoting positive behaviour; School connectedness; Positive transitions; Remote & Flexible learning items (2020). The items use a five-point Likert scale from 1 (strongly disagree) to 5 (strongly agree).
Implementation measures
Readiness to implement
To effectively evaluate the implementation of the MHWC model we will also seek to understand individual and contextual characteristics (across all participant groups) that are present prior to implementation which may also influence program outcomes. We will use the Readiness to Implement Scale [51] which looks at three key areas: feasibility, fit and staff support. It is a self-report measure, consisting of 20 items and has an overall reliability of Cronbach’s alpha [α] = .92 [51].
Costs of the intervention
Costs of delivering the intervention will be estimated to inform wider implementation. This will be based on budgets for each role, the log of activities kept by MHWCs tracking time and resources used and from records of replacement teachers filling in for classroom teachers to attend professional development and training activities run by the MHWCs. The costs will be from the perspective of the school.
MHWC referral activities log and job analysis
To evaluate the tasks, responsibilities, time, and resources required to achieve all aspects of the MHWC role and successful implementation of the MHWC model, MHWC activity will be recorded in a study developed database. This information will include number of students impacted, number of interactions with classroom teachers, time spent on tasks, type of mental health and wellbeing activities the MHWC is assisting with, number of referrals made, uptake/outcome of referral, waitlist times, number, and type of interactions with regional staff. This information will be collected for two working weeks during the academic school year (Semester 2).
Feasibility, appropriateness, and acceptability of the MHWC model
To evaluate the feasibility, appropriateness, and acceptability of the MHWC model we will use the Feasibility of Intervention Measure (FIM), Intervention Appropriateness Measure (IAM) and the Acceptability of Intervention Measure (AIM) [50] adapted as required. These measures will assess the extent to which a new treatment or innovation can be successfully used or carried out within a setting (FIM), the perceived fit, relevance and compatibility of the intervention for a given practice setting, provider, or consumer and/or perceived fit of the innovation to address a particular issue or problem (IAM) and the perception among implementation stakeholders that an intervention is agreeable or satisfactory (AIM). These measures consist of 4-items each targeting perceived intervention acceptability. Items are measured on a 5-point Likert scale (completely disagree-completely agree) and the score is a calculated mean. Each measure has excellent structural validity (FIM α = 0.89, IAM α = 0.91, AIM α = 0.85) and test-retest reliability (FIM α = 0.88, IAM α = 0.87, AIM α = 0.83) [50].
Engagement of school staff with MHWC model
To evaluate the level of engagement that school staff have with the MHWC model, questions will be developed to capture the perceived level of engagement required, the actual level of engagement and the types of support provided to staff by MHWCs.
Focus groups
To further explore and capture data regarding the feasibility, appropriateness, and acceptability of the model, purposive sampling will be used to recruit a subset of school and regional staff to participate in qualitative focus groups at T4 and T5, using a semi-structured interview guide. Focus groups will consist of between 3 and 8 participants and will include MHWCs and other trainees from participating intervention schools as well as a cross-section of other school staff (including leadership and education support staff, wellbeing staff, classroom teachers) and regional support staff. Interviews will be facilitated by an experienced qualitative researcher, audio-recorded with permission from the participants and transcribed verbatim using an external transcription service.
Moderators
Exploration of potential moderators to the relationship between the intervention and the primary outcome will include planned FTE allocation of MHWC, socio-economic status (ICSEA value), school geographic location (metro/rural/regional), as well as process moderators; training dosage (hours attended of training) and local adjustments to MHWC FTE by the school during implementation (average hours worked in MHWC role).
Sample size calculation
We based the sample size calculation on the primary study objective of detecting the impact of the MHWC model on classroom teachers’ self-reported confidence to support student mental health and wellbeing which is assessed by the School Mental Health Self-Efficacy Teacher Survey (SMH-SETS [55]). We assumed a mean SMH-SETS score of 67.7 (standard deviation = 9.5) in the BAU arm based on unpublished data provided by the measure authors [56]. Participation of 12 teachers per school with the intra-cluster (intra-school) correlation coefficient (ICC) set to 0.1 (expected ICC is unknown); the sample of 32 schools (16 intervention schools and 16 BAU schools) provides 80% power at the (2-sided) 5% level of significance to detect a mean difference between the trial arms of 3.9 on the teacher-reported SMH-SETs score.
Planned statistical analysis
All analyses will be based on observed data only; i.e. we will assume data are Missing Completely At Random. We will make every attempt to ensure that data are not missing from surveys at the point of completion. If there are missing responses to surveys completed by school or regional staff, we will follow up via phone or email up to three times to obtain the missing responses. If missing data are > 10% for specific outcomes, then predictors of missing data will be explored and sensitivity analyses making further adjustments (i.e. valid under Missing At Random) will be used such as imputation. The details of how this will be conducted will be outlined in the statistical analysis plan.
The primary outcome (mean difference in SMH-SETS total score) will be analysed using mixed effects linear regression fitted at the teacher level, including a fixed effect for arm (MHWC vs. BAU) and a random effect for school, adjusting for baseline values of SMH-SETS and school matching criteria (ICSEA, number of enrolments and metro/regional/rural location).
According to the nature of the secondary outcomes to be analysed (binary, continuous or ordinal) the appropriate mixed effects model will be used to estimate the impact of the MHWC model on the outcome of interest compared to the BAU schools. These models will be fitted at the participant level, including a fixed effect for arm (MHWC vs. BAU) and a random effect for school adjusting for school matching criteria (ICSEA, number of enrolments and metro/regional/rural location) and baseline value of the outcome where available. Analyses will be done using Stata Statistical Software MP.
Cost analysis
Variation in costs across participating schools will be evaluated to identify factors that may impact on the costs of delivering the intervention according to school characteristics such as number of student enrolments, location, and SES. Based on these estimated costs, budget impact of delivering the intervention state-wide will be calculated, with associated resource implications, particularly workforce.
Qualitative analysis
Focus group transcripts will be deidentified and imported into the computer software package QSR Nvivo 12 [57]. Data analysis will follow the guidelines for reflexive thematic analysis [58]. An experienced qualitative researcher will independently code the data and emergent categories and themes will be cross-checked with the broader team for accuracy and to ensure the data is well represented. Study findings will be reported in line with the COREQ checklist for reporting qualitative research [59].