- Study protocol
- Open Access
The effectiveness of school-based skills-training programs promoting mental health in adolescents: a study protocol for a randomized controlled study
BMC Public Health volume 19, Article number: 712 (2019)
Adolescence is a period of elevated stress sensitivity, which places adolescents at increased risk of developing mental health problems such as burnout, depression, anxiety, and externalizing problems. Early intervention of psychological needs and low-threshold care addressing such needs may prevent this dysfunctional development. Schools may provide an important environment to identify and address psychological needs. The aim of this protocol is to describe the design of a study aiming to evaluate the effectiveness of low-threshold school-based skills-training programs promoting the mental health of adolescents and to examine moderators of the effectiveness.
A Randomized Controlled Trial will be conducted to examine the effectiveness of two school-based skills-training programs aiming to promote mental health by improving either skills to deal with performance anxiety or social skills. A multi-informant (i.e., students, parents, and trainers) and multi-method (i.e., questionnaires and physiological measurements) approach will be used to assess program targets (skills to deal with performance anxiety or social skills), direct program outcomes (performance or social anxiety) and mental health outcomes (i.e., stress, internalizing and externalizing problems, self-esteem and well-being), as well as specific moderators (i.e., student, parent and program characteristics, social support, perfectionism, stressful life events, perceived parental pressure, positive parenting behavior, treatment alliance and program integrity).
The current study will provide information on the effectiveness of school-based skills-training programs. It is of crucial importance that the school environment can provide students with effective, low-threshold intervention programs to promote adolescents’ daily functioning and well-being and prevent the emergence of mental health problems that negatively affect school performance.
Dutch Trial Register number NL7438. Registered 12 December 2018.
Adolescence is a phase of rapid growth and development in physical and psychological domains . During adolescence many changes occur simultaneously, including puberty and the transition to high school. At the same time, adolescence is a period of increased stress sensitivity [2,3,4], which contributes to adolescents’ increased risk for mental health problems, such as burnout, depression, anxiety and externalizing problems [5,6,7,8] and which may negatively affect the well-being of adolescents [9,10,11] and later developmental outcomes . Stress also has a negative effect on academic performance [13,14,15] and can result in school absenteeism or dropout . Addressing psychological needs at an early stage, for instance to deal with stress and stress-inducing factors, is of crucial importance to prevent the development of mental health problems, school dropout and dysfunction later in life. An environment particularly suitable to help vulnerable adolescents is the school environment. Intervening in the school-context may be particularly beneficial to schools as well-being and mental health have been positively associated with academic functioning .
A promising avenue to reach adolescents with psychological needs is by focusing on stress. As experiencing stress is part of normative development during adolescence , interventions focusing on the reduction of stress may be experienced by adolescents as a low-threshold and appealing way to address their psychological needs. Stress has been defined as the condition or feeling that results when individuals perceive that the demands of a situation exceed the individual’s personal, psychological, or social resources . During adolescence, performance pressure as well as social situations at school may trigger feelings of stress, also referred to as academic and social stress .
Academic stressors that are frequently reported by adolescents are related to tests, grades, homework, expectations about school, expectations about their career and future life plans . Academic stress is related to performance anxiety, where individuals experience fear of failure, the fear to be unable to meet certain expectations of themselves or others, or test anxiety. Improving skills to deal with academic stress through intervention programs can reduce stress in the school environment [21, 22]. Social stressors originate from an individual’s social environment and are caused by factors that disrupt the relationship with others, such as social rejection, isolation, disagreements or bullying . Dysfunctional social interactions can trigger stress , hence, improving social skills by teaching adolescents to better communicate with others, might reduce perceived social stress.
The ability to cope with stress is very important and requires cognitive and behavioral efforts to control or reduce stressful experiences . One way to thus counter the negative effects of adolescents’ stress is addressing either academic or social stress by offering preventative skills-training programs that provide adolescents with tools to effectively cope with stress and regulate emotions . In order to effectively help adolescents who perceive academic or social stress and to promote their mental health, this protocol describes a study to examine the effectiveness of two school-based skills-training programs targeting skills to deal with performance anxiety or social skills.
Several studies implemented skills-training programs targeting performance anxiety, mainly focusing on test anxiety. Recent studies in secondary school students reported decreased test anxiety compared to controls after interventions targeting a combination of coping skills, relaxation techniques and study skills [27,28,29,30,31]. A reduction was also found in physiological stress [27, 31], internalizing problems  and behavioral problems , as well as increased self-esteem .
Recent studies demonstrated increased positive social behavior and improved social skills in secondary school students after mindfulness or social and emotional skills based interventions compared to controls [32, 33]. A reduction was found for perceived stress , problem behaviors [33, 34] and internalizing problems , as well as increased self-esteem [33,34,35].
In sum, previous studies targeting skills to deal with performance anxiety and social skills in secondary school students demonstrated promising results, for academic and social stress as well as for mental health outcomes. However, previous research shows several limitations. First, only few studies performed a randomized controlled trial (RCT) [30, 35], which means that in non-randomized studies there might be confounding factors or bias. Second, only few studies used a targeted small-group intervention [28, 30, 35], which means that in studies with universal interventions there might be minimal or no effect for at-risk students because their greatest effect is on low-risk students, who are not the desired target . Moreover, these studies had relatively small sample sizes (N = 74 and N = 56) [30, 35] which precludes clear conclusions about the effects, or targeted youth that experienced a traumatic event , precluding conclusions about non-clinical community samples. Third, only a few studies investigated mixed-samples of students from diverse ethnic backgrounds [30, 31, 33], which is important to be able to draw conclusions that generalize to the whole population . Hence, there is still insufficient evidence for the effectiveness of skills-training programs, especially for students in mixed-ethnicity community samples. Additionally, little is known about potential moderators of the effectiveness of school-based skills training programs, likely due to power issues. Which students benefit from such training programs and which factors contribute to their effectiveness?
Moderators are student, parent and program characteristics that are likely to affect the effectiveness of the training programs. Student characteristics that may affect the effectiveness of the skills-training programs include demographic characteristics (i.e., age, gender, ethnicity, educational level and socioeconomic status (SES)) and social support, perfectionism, stressful life experiences, severity of problems, perceived parental pressure and basal stress levels.
For instance, high levels of social support have been positively associated with well-being and mental health in children and adolescents [38, 39]. Moreover, high levels of social support have been associated with a more beneficial psychological treatment outcome in clinical samples [40, 41]. High occurrence of stressful life events has been associated with stress-related psychopathology and negative mental health outcomes, such as anxiety, depression and risk behavior [42,43,44] and predicts adverse treatment outcome . On the one hand, high social support and low occurrence of stressful life events are associated with more beneficial outcomes. On the other hand, several studies showed larger program effects for adolescents with high initial problem severity [46, 47]. Following this line of reasoning, it may also be possible that students with low social support or with high occurrence of stressful life events profit most from the skills-training programs, because the sessions provide them with tools that are not provided by their social network.
Perceived stress positively correlates with perfectionism  and perceived parental pressure , which have been associated with mental health problems such as anxiety and depression [50,51,52]. Previous studies showed that perfectionism and maternal rejection (i.e., less emotional warmth), the latter associated with parental pressure , are related to less beneficial treatment outcomes in children and adolescents with anxiety or depression [53,54,55,56]. It is therefore expected that students with higher levels of perfectionism or students who perceive more parental pressure may benefit less from the skills-training programs.
Parent characteristics that may affect the effectiveness of the skills-training program include demographic characteristics (i.e., educational level) and positive parenting behavior. Positive parenting behavior increases adolescents’ social competence reflecting positive functioning at school including peer competence and attachment to school , and improves the relationship between parent and adolescent . Because of this warm relationship, parents may be more involved in the life of their child and students may feel more confident to talk about the program at home. It is therefore expected that the skills-training programs are more effective for students with parents that show more positive parenting behavior.
Additionally, program characteristics, i.e., treatment alliance, program integrity and trainer characteristics such as ethnicity, experience and perceived competence, may moderate the effectiveness of the skills-training programs. Treatment alliance is the perceived bond between the participant of the skills-training program and the group leader (i.e., trainer), which is demonstrated to be positively associated with treatment outcome in depressive patients  and in children and adolescents with anxiety disorders [59, 60]. Hence, it is expected that high participant-trainer alliance contributes to a more beneficial outcome. Program integrity refers to the extent to which a program is implemented as originally planned, and is reported by very few studies . Finding non-significant effects may not be caused by an ineffective program, but because a program is not carried out as intended . Therefore, it is important to examine program integrity in order to correctly draw conclusions on the effectiveness of the skills-training programs.
In sum, the current protocol describes a study that will be conducted to examine the effectiveness of two types of targeted school-based skills-training programs addressing skills to deal with performance anxiety (program 1) and social skills (program 2). In order to overcome some of the limitations of previous studies, small-group skills-training programs are evaluated in a mixed-ethnicity community sample of students with different educational levels, where students self-select to one of the skill-training programs. For each program, a RCT is performed to examine if these low-threshold interventions reduce performance or social anxiety and promote students’ mental health. Both skills-training programs will be compared to a control waitlist condition. As illustrated in Fig. 1, it is expected that the performance anxiety skills-training program will improve skills to reduce performance anxiety (i.e., coping skills, including negative thought restructuring and managing emotions), resulting in a reduction in students’ performance anxiety (i.e., fear of failure and test anxiety) and in improved mental health (i.e., reduced stress and internalizing and externalizing problems, and increased well-being and self-esteem). Additionally, we expect that the performance anxiety skills-training program will directly reduce performance anxiety and improve mental health. It is expected that the social skills-training will improve students’ social skills and thereby reduce their social anxiety and improve their mental health (i.e., reduced stress and internalizing and externalizing problems, and increased well-being and self-esteem). In addition, we expect that the social skills training program will directly reduce social anxiety and improve mental health (see Fig. 1). The second aim is to investigate the moderators of the effectiveness of both skills-training programs on all outcomes. This is important because it is likely that the skills-training programs are not equally effective for all students. Student, parent and program characteristics will be examined as potential moderators. The final aim of this study is to evaluate if the school-based skills-training programs are experienced as sufficiently accessible, meaningful and helpful by the students, their parents and the trainers. Figure 1 shows a conceptual model of the research design.
A RCT will be conducted for both school-based skills-training programs targeting: 1) skills to deal with performance anxiety and 2) social skills. For each skills-training program, students who have indicated interest in attending a training at their school, will be randomized by the first author (stratified for education level) into the experimental group in which the training starts immediately or into a waitlist control group that receives the training approximately eight weeks later (see Fig. 2), using computerized randomization in a 1:1 ratio. We will use a multi-method (questionnaires and physiological data) and a multi-informant (students, parents and trainers) design and will recruit a mixed-ethnicity sample of students from different educational levels.
Questionnaires assessing program targets (i.e., skills to deal with performance anxiety or social skills), specific goals of the skills-training programs (i.e., performance or social anxiety) and mental health (i.e., stress, internalizing and externalizing behavior, well-being and self-esteem) will be completed by students and parents prior to the start (T1) and after completion of the interventions in the experimental group (T2). Trainers complete questionnaires about program content (to assess program integrity) after each training session. Additionally, in a subsample of students (approximately N = 40 students per skills-training program), physiological parameters will be measured during a resting period to assess the basal stress levels of students (i.e., heart rate, heart rate variability and skin conductance measurements with wearables).
Student and parent characteristics (i.e., demographics including gender, age, ethnicity and education level, and expectations about the program) are assessed prior to the interventions, as well as trainer characteristics (i.e., demographics including gender, age, ethnicity, education level, and level of experience and perceived competence). Lastly, after completion of the intervention, students, parents and trainers will evaluate the skills-training program.
The design of this study has been approved by the Ethical Committee Psychology of Leiden University (CEP18–1105/419) and is registered in the Dutch Trial Register (number NL7438). To maintain participant confidentiality, all records that contain names or other personal identifiers will be stored separately from the collected data identified by code numbers.
For each school-based skills-training program we aim to include N = 130 students to ensure there is enough power for the analyses (N = 65 for both the experimental and waitlist control group). A total of N = 260 students will be included. This sample size is sufficient to investigate the effectiveness of the interventions and potential moderators, with a power of .80, an alpha of .05 and a medium effect size of .25 .
Participants are mixed-ethnicity students in the first, second, and third year of secondary schools (7th, 8th and 9th Grade students) of at least three schools in the Netherlands that offer education at various levels (from vocational education level to preparatory university education level). The students will be between 11 and 16 years old.
So far, three Dutch urban secondary schools have been recruited. Possibly, additional schools will be recruited via the researchers’ contacts and networks. Schools that show interest will receive information about the study and will be asked to participate.
This study will be performed in the context of a Response to Intervention model (RTI) that aims to identify vulnerable students and provide them with appropriate interventions . Before entering the current study, focusing on the effectiveness of school-based skills-training programs, classes of students receive educative information about stress and are asked about their own stress levels (i.e., Tier 1 of the RTI model: universal intervention targeting all students). After these three educative lessons, students will be asked to indicate if they would like to learn more about dealing with academic or social stress by following a skills-training program. Students will be asked to self-select to one of the school-based skills-training programs (i.e., skills to deal with performance anxiety or social skills), if needed assisted by parents or teachers (i.e., Tier 2 of the RTI model: targeted intervention directed at self-selected at-risk students). Students who self-select to a Tier 2 intervention are asked to participate in the current study.
The skills-training programs are offered by schools via their own care system or by external youth care organizations. The interventions are implemented at school by trained teachers or professionals. Students and parents receive written information about the different skills-training programs that are offered and receive an information letter about the corresponding research study. Trainers also receive an information letter about the study. Students and parents will be asked to provide active informed consent for the student’s participation in the study. If students or parents do not give consent to participate, the student will not receive the skills-training program offered by the study but will receive help via the schools’ own care system. Active informed consent will also be obtained from parents and trainers for their participation. After receiving consent, students will be randomized into the experimental condition or the waitlist condition. Figure 2 shows the flow chart of the design of the study.
Students in the experimental condition of each skills-training program will participate in seven 45-min small-group sessions during consecutive weeks. Weekly sessions will take place at the school of the students and the sessions will be delivered by an experienced trainer or trained teacher in small groups. The performance anxiety skills-training program consists of cognitive coping strategies (e.g., negative thought restructuring and managing emotions), relaxation techniques and dealing with pressure. The social skills training program consists of social skill building (e.g., identifying personal qualities, giving own opinions, setting boundaries, and standing up for yourself) and cognitive coping strategies (e.g., managing emotions).
The waitlist group will not receive any training during the implementation of the intervention in the experimental group and will only complete the pre-and post-intervention measurements. The waitlist group will receive the intervention immediately after the post-intervention measurements, approximately 8 weeks later than the experimental group.
Table 1 presents an overview of the measurements used at each assessment point for the skills-training programs.
Program targets, direct program outcomes and mental health outcomes will be assessed before the start of the skills-training program and immediately after the completion of the program.
Skills to deal with performance anxiety will be measured with the Dutch version of the Cognitive Emotional Regulation Questionnaire – short form (CERQ-short) , completed by students. This instrument is a 18-item self-report measuring cognitive related coping (e.g., “I think I can learn something from the situation” or “I keep thinking about how terrible it is what I have experienced”). It consists of nine subscales: Self-blame, acceptance, rumination, positive refocusing, refocus on planning, positive reappraisal, putting into perspective, catastrophizing and other-blame. The authors made a distinction between maladaptive coping (self-blame, other-blame, rumination and catastrophizing) and adaptive coping (acceptance, refocus on planning, positive refocusing, positive reappraisal and putting into perspective) . The internal consistency of the subscales is between .68 and .81 .
Social skills will be measured with the Scale for Interpersonal Behavior of Adolescents (SIG-A) [67, 68], completed by students and parents. For parents, an adapted version will be used where “I” is replaced with “my child”. The self-report version for students consists of 47 situations that are evaluated on two dimensions (i.e., how much anxiety students experience during these situations and how often they experience these specific situations). In this study we only use the performance dimension (i.e., frequency) to assess social skills. For example, the item “Starting a conversation with someone you haven’t met before” is scored on a scale from “never” to “always” for the performance dimension. The instrument consists of four subscales that refer to specific social situations: 1) display negative feelings (14 items, e.g., “If someone interrupts you, saying you find that annoying”) 2) express personal limitations (13 items, e.g., “Asking for an explanation about something that you didn’t understood”), 3) initiate assertiveness (9 items, e.g., “Starting a conversation with someone you haven’t met before”) and 4) display positive feelings (8 items, e.g., “Agreeing when someone makes a compliment about your appearance”). This instrument has sufficient psychometric properties with a Cronbach’s alpha above .80 for all subscales .
Direct program outcomes
Performance anxiety will be measured by two instruments measuring different domains, i.e., fear of failure and test anxiety, completed by students and parents. For parents, adapted versions are used where “I” is replaced by “my child”. The short form of the Performance Failure Appraisal Inventory (PFAI)  will be used, translated into Dutch. The PFAI is a 5-item self-report instrument measuring fear of failure (e.g., “When I am failing, I am afraid that I might not have enough talent”) with good reliability (internal consistency between .72 and .82) [69, 70]. The Dutch short version of the widely used Spielberger Test Anxiety Inventory (TAI) [71, 72] will be used to assess anxiety in school testing situations. This instrument consists of 20 items (e.g., “I feel confident and relaxed while taking tests”) and has demonstrated adequate internal consistency (between .92 and .96) .
Social anxiety will be measured with the social phobia scale of the Dutch version of the Revised Child Anxiety and Depression Scale (RCADS) , completed by students and parents. For parents, an adapted version will be used where “I” is replaced with “my child”. This instrument consists of 9 items (e.g., “I worry what other people think of me”) and has good internal consistency (between .78 and .81) [74, 75].
Mental health outcomes
Stress levels of students will be measured with the Chronic Stress Questionnaire for Children and Adolescents (CSQ-CA)  and will be completed by students and parents (adapted version). The CSQ-CA is a 19-item self-report questionnaire (e.g., “I often get upset about things that are not important”) that demonstrated good psychometric properties with an internal consistency of .87 . In addition, physiological measurements are performed to assess basal stress levels in a subsample of students. Heart rate, heart rate variability and skin conductance are measured via a wearable (Shimmer3 GSR+; ) during a 5-min resting period where students watch a relaxing aquatic video . Prior to the physiological assessment, students will complete Visual Analogue Mood Scales (VAMS)  about their current mood and stress level.
Internalizing and externalizing problem behavior will be assessed with the Youth Outcome Questionnaire (Y-OQ-30.1) , translated into Dutch and completed by students. This instrument consists of 30 items (e.g., “My emotions are strong and change quickly”) and measures change in psychological symptoms and social functioning. It includes six subscales: somatic complaints (3 items), social isolation (2 items), aggression (3 items), conduct problems (6 items), hyperactivity/distractibility (3 items) and depression/anxiety (6 items). Internalizing problem behavior will be assessed with the subscale depression/anxiety and externalizing problem behavior with the subscales aggression and conduct problems. The internal consistency of the total scale is .92 and between .55 and .85 for the different subscales .
Well-being of students will be assessed with the Dutch version of the WHO-Five Well-Being Index , which consists of 5 items (e.g., “My daily life has been filled with things that interest me”). This instrument has good internal consistency for an adolescent sample (between .82 and .85) [81, 82].
Self-esteem is reported by students by completing the Dutch version of the Rosenberg Self-Esteem Scale (RSES) [83, 84]. The instrument consists of 10 items (e.g., “I take a positive attitude toward myself”) and has sufficient internal consistency (between .77 and .88) for high school students .
Student, parent and trainer characteristics, including age, gender, ethnicity and education level will be collected at baseline. Parents will also report family characteristics and information about work and education level (as indicator of SES), and trainers will report information about their level of experience, perceived competence and educational background. Finally, students and parents will report their expectations for the skills-training program (at baseline) and will evaluate the program (post-intervention measurement).
Positive parenting behavior will be measured with the Dutch Abbreviated Scale of Parenting Behavior (VSOG) , completed by parents. This instrument is a 25-item self-report for parents with five subscales. In this study only the subscale positive parenting behavior will be used, which consists of 8 items (e.g., “I make time for my child, when he/she wants to tell me something”). The subscale has a Cronbach’s alpha of .83 for mothers and .87 for fathers .
Social support will be measured with the Social Support List – Interactions (SSL-I) , adapted for use in adolescents. It measures the extent of received social support by social interactions in an individual’s social network. The instrument consists of 12 items (e.g., Does it ever happen to you that people: “are interested in you” or “ask you for help or advice”?) and three subscales (i.e., everyday social support, social support in problem situations and esteem support) and has acceptable internal consistency for all subscales (.70 or above) [86, 87].
Perfectionism will be measured with the Child and Adolescent Perfectionism Scale (CAPS-14) , translated into Dutch. This instrument is a 14-item self-report measuring perfectionism and consists of three subscales: self-oriented perfectionism-striving (3 items, e.g., “I try to be the best at everything I do”), socially prescribed perfectionism (7 items, e.g., “Other people always expect me to be perfect”) and self-oriented perfectionism-critical (4 items, e.g., “I get mad at myself when I make a mistake”). The internal consistency of the subscales is between .72 and .86 .
Stressful life events will be measured with the Negative Life Events Inventory [89, 90], translated into Dutch. This instrument is a 20-item checklist of negative life events (e.g., “Somebody in my family had a serious illness”). Students are asked to indicate whether an event had occurred during the previous year and includes events that occurred to family members and directly to themselves. The internal consistency is between .67 and .71 [89, 90]. Students will complete this instrument after completion of the skills-training program (post-intervention measurement).
Perceived parental pressure will be measured with the subscale parental pressure of the Multidimensional Inventory of Perfectionism in Sport (MIPS) , translated into Dutch. The perceived parental pressure subscale consists of 8 items (e.g., “My parents set extremely high standards for me”). The internal consistency is good (above .92) [91, 92]. Parental pressure has a high temporal stability  and students will therefore complete this instrument at post-measurement.
Treatment alliance will be measured with the Therapy Alliance Scale for Children (TASC) , translated into Dutch and altered for use in group skills-training programs. This is a 12-item instrument measuring the bond-aspect of alliance (6 items, e.g., “I like my trainer”) and the tasks of the program (6 items, e.g., “I work with my trainer on solving my problems”). This instrument has good internal consistency (above .70) [95, 96] and is completed by students after completion of the skills-training program (post-intervention measurement).
Program integrity will be measured with a questionnaire specifically developed for this study. After each session, trainers will be asked to evaluate the session by registering if they carried out the session as intended (e.g., is the content of the session sufficiently treated, are other components discussed, and what was their overall impression of the session).
To examine the effectiveness of the skills-training programs, the effects of all outcome measures will be investigated by conducting analyses of covariance (ANCOVAs). The dependent variables are the outcome measures at post-test (program targets, direct program outcomes and mental health outcomes), the independent variables are the conditions and the covariates are the pre-test (baseline) measurements of the outcome measures. The effect of potential categorical and continuous moderators on the effectiveness of the skills-training programs on the specific goals of the programs and mental health of students will be examined by adding the moderators to the ANCOVAs. Data will be imported from an online server software (i.e., Qualtrics) and will be securely stored at the server of Utrecht University where data back-ups will be performed regularly.
This study protocol introduces the design of two RCTs investigating the effectiveness of school-based skills-training programs targeting skills to deal with performance anxiety or social skills to promote the mental health of adolescents. Further empirical tests of the utility of the RTI model will be published separately. Previous studies demonstrated reduced stress levels, reduced internalizing and externalizing behavior, increased self-esteem and improved well-being in adolescents receiving skills-training interventions. To overcome the limitations of previous studies, including study design (e.g., non-randomization, small sample sizes and the absence of moderator analyses) and use of mostly universal interventions, we will investigate small-group skills-training programs in a mixed-ethnicity community sample and use a RCT to exclude confounding (e.g., allocation and selection bias) as much as possible. We will use a multi-informant (i.e., students, parents and trainers) and multi-method (i.e., questionnaires and physiological measurements) approach. Furthermore, we will use a sufficiently large sample size to examine potential moderators of the effectiveness of the interventions (i.e., student, parent and program characteristics).
There are several challenges in this study, such as the recruitment of an ethnically diverse and representative sample of students and the prevention of dropout of students and parents. First, the self-selection of students to enroll in a skills-training program may constitute a challenge, as students may feel unable to make the correct decision or think they have problems in one domain, while these are caused by another underlying problem. We try to diminish this by involving parents and teachers to help students make the right choice. On the other hand, the self-selection of students may also be advantageous, because students are likely to be more motivated which may contribute to a more beneficial program outcome . Second, it may be difficult to recruit sufficient numbers of mixed-ethnicity students and parents and maintain their involvement over the course of this study. Recruitment may be difficult because of language barriers and unfamiliarity and mistrust with research. It is therefore of great importance that the schools are actively involved in giving information to students and parents, because they rely and trust on their school. Since participation of students requires active informed consent from their parents, it is of utmost importance to reach all parents and provide them with clear information about study participation. In our efforts to reach students and their parents, we will use multiple sources (i.e., researchers and teachers or other school contacts) and multiple methods (i.e., information during classes, written information and phone calls) to provide them with information and to emphasize the relevance of the study. In order to make giving permission as easy as possible, participants can also give digital consent. Moreover, the questionnaires for students, parents and trainers are brief, clear and can be completed online, to avoid lost or non-returned questionnaires. These precautions are expected to increase the likelihood of participation and retention. In addition, with these precautions we aim to reduce the risk that socially disadvantaged groups, that are difficult to recruit and retain in health research , are underrepresented in our sample.
Third, a challenge in this study is to implement the skills-training programs at schools because of logistics (i.e., different locations and scheduling) and the participation and involvement of multiple parties (i.e., researchers, schools, students, parents, youth care organizations that provide the skills-training programs, and municipalities involved). With regard to the latter, multiple schools and multiple organizations offering skills-training programs are involved in this study. To strengthen the collaboration between the schools and the youth care organizations, we will organize regular meetings with all parties involved and aim to match the schools with suitable youth care organizations. With regard to logistics, it is very important that enrollment in the skills-training programs and arrangements in terms of schedules and locations will be timely communicated with the schools and youth care organizations. The researchers will try to overcome potential implementation issues by being present at the schools at least once a week, by giving weekly updates to the youth care organizations, and by being available for consultation at any time during the referral process and skills-training programs.
Fourth, the participating schools do not belong to the same municipality geographically, which complicates the financial conditions for the skills-training programs. To overcome financial issues, we will talk to different municipalities to obtain funding for the skills-training programs. Finally, it is important to be aware that in the Netherlands, school and youth care systems are completely separate, both organizationally as well as financially, which makes cooperation complicated. We try to overcome this challenge by giving regular updates and organize meetings with all parties involved, to promote a fruitful collaboration.
Overall, in order to overcome the practical challenges in this study, we aim to be as flexible as possible (e.g., being available for consultation at all times) towards the different parties involved. Of foremost importance is investing time in clear and timely communication with all parties in order to collaborate as effectively as possible. We try to involve the parties as much as possible by organizing regular meetings and actively involving them in the decisions in the research. Furthermore, we try to work according to a standardized research protocol as much as possible, to avoid miscommunications and ensure a positive and productive collaboration.
The current protocol describes a study that will investigate the effectiveness of school-based skills-training programs targeting skills to deal with performance anxiety or social skills promoting mental health in adolescents. It is of crucial importance that the school environment can provide students with interventions to help them cope with stress-inducing factors and to prevent the development of mental health problems, school dropout and dysfunction later in life.
Analysis of Covariance.
Child and Adolescent Perfectionism Scale
Cognitive Emotion Regulation Questionnaire
Chronic Stress Questionnaire for Children and Adolescents
Multidimensional Inventory of Perfectionism in Sports
Performance Failure Appraisal Inventory
Revised Child Anxiety and Depression Scale
Randomized Controlled Trial
Rosenberg’s Self Esteem Scale
Response to Intervention
Scale for Interpersonal Behavior for Adolescents
Social Support Inventory-Interactions
Test Anxiety Inventory
Therapy Alliance Scale for Children
Visual Analogue Mood Scale
Dutch Abbreviated Scale of Parenting Behavior
Youth Outcome Questionnaire
Christie D, Viner R. ABC of adolescence adolescent development. BMJ. 2005;330:301–4.
Andersen SL, Teicher MH. Stress, sensitive periods and maturational events in adolescent depression. Trends Neurosci. 2008;31:183–91.
Fuhrmann D, Knoll LJ, Blakemore SJ. Adolescence as a sensitive period of brain development. Trends Cogn Sci. 2015;19:558–66.
Romeo RD. The teenage brain: the stress response and the adolescent brain. Curr Dir Psychol Sci. 2013;22:140–5.
de Bruin EI, Sieh DS, Zijlstra BJH, Meijer AM. Chronic childhood stress: psychometric properties of the chronic stress questionnaire for children and adolescents (CSQ-CA) in three independent samples. Child Indic Res. 2018;11:1389–406.
Jayanthi P, Thirunavukarasu M, Rajkumar R. Academic stress and depression among adolescents: a cross-sectional study. Indian Pediatr. 2015;52:217–9.
Snyder HR, Young JF, Hankin BL. Chronic stress exposure and generation are related to the P-factor and externalizing specific psychopathology in youth. J Clin Child Adolesc Psychol. 2017;00:1–10.
Walburg V. Burnout among high school students: a literature review. Child Youth Serv Rev. 2014;42:28–33.
Burger K, Samuel R. The role of perceived stress and self-efficacy in Young People’s life satisfaction: a longitudinal study. J Youth Adolesc. 2017;46:78–90.
Chappel AM, Suldo SM, Ogg JA. Associations between adolescents’ family stressors and life satisfaction. J Child Fam Stud. 2014;23:76–84.
Kiang L, Buchanan CM. Daily stress and emotional well-being among Asian American adolescents: same-day, lagged, and chronic associations. Dev Psychol. 2014;50:611–21.
Lupien SJ, McEwen BS, Gunnar MR, Heim C. Effects of stress throughout the lifespan on the brain, behaviour and cognition. Nat Rev Neurosci. 2009;10:434–45.
Arsenio WF, Loria S. Coping with negative emotions: connections with adolescents’ academic performance and stress. J Genet Psychol. 2014;175:76–90.
Kaplan DS, Liu RX, Kaplan HB. School related stress in early adolescence and academic performance three years later: the conditional influence of self expectations. Soc Psychol Educ. 2005;8:3–17.
Liu Y, Lu Z. The Chinese high school student’s stress in the school and academic achievement. Educ Psychol. 2011;31:27–35.
Dupéré V, Leventhal T, Dion E, Crosnoe R, Archambault I, Janosz M. Stressors and turning points in high school and dropout. Rev Educ Res. 2015;85:591–629.
Suldo S, Thalji A, Ferron J. Longitudinal academic outcomes predicted by early adolescents’ subjective well-being, psychopathology, and mental health status yielded from a dual factor model. J Posit Psychol. 2011;6:17–30.
Lazarus RS. Psychological stress and the coping process; 1966.
Nicolai KA, Laney T, Mezulis AH. Different stressors, different strategies, different outcomes: how domain-specific stress responses differentially predict depressive symptoms among adolescents. J Youth Adolesc. 2013;42:1183–93.
de Anda D, Baroni S, Boskin L, Buchwald L, Morgan J, Ow J, Gold JS, Weiss R. Stress, stressors and coping among high school students. Child Youth Serv Rev. 2000;22:441–63.
Regehr C, Glancy D, Pitts A. Interventions to reduce stress in university students: a review and meta-analysis. J Affect Disord. 2013;148:1–11.
Saunders T, Driskell JE, Johnston JH, Salas E. The effect of stress inoculation training on anxiety and performance. J Occup Health Psychol. 1996;1:170–86.
Juth V, Dickerson S. Social stress. In: Gellman MD, Turner JR, editors. Encycl. Behav. Med. New York, NY: Springer New York; 2013. p. 1842–3.
Ditzen B, Heinrichs M. Psychobiology of social support: the social dimension of stress buffering. Restor Neurol Neurosci. 2014;32:149–62.
Lazarus RS, Folkman S. Coping and adaptation; 1984.
Compas BE, Jaser SS, Dunbar JP, Watson KH, Bettis AH, Gruhn MA, Williams EK. Coping and emotion regulation from childhood to early adulthood: points of convergence and divergence. Aust J Psychol. 2014;66:71–81.
Yahav R, Cohen M. Evaluation of a cognitive-behavioral intervention for adolescents. Int J Stress Manag. 2008;15:173–88.
Weems CF, Scott BG, Graham RA, et al. Fitting anxious emotion-focused intervention into the ecology of schools: results from a test anxiety program evaluation. Prev Sci. 2014;16:200–10.
Yeo LS, Goh VG, Liem GAD. School-based intervention for test anxiety. Child Youth Care Forum. 2016;45:1–17.
Putwain DW, Pescod M. Is reducing uncertain control the key to successful test anxiety intervention for secondary school students? Findings from a randomized control trial. Sch Psychol Q. 2018;33:283–92.
Bradley RT, McCraty R, Atkinson M, Tomasino D, Daugherty A, Arguelles L. Emotion self-regulation, psychophysiological coherence, and test anxiety: results from an experiment using electrophysiological measures. Appl Psychophysiol Biofeedback. 2010;35:261–83.
Durlak JA, Weissberg RP, Dymnicki AB, Taylor RD, Schellinger KB. The impact of enhancing students’ social and emotional learning: a meta-analysis of school-based universal interventions. Child Dev. 2011;82:405–32.
Schonert-Reichl KA, Lawlor MS. The effects of a mindfulness-based education program on pre- and early adolescents’ well-being and social and emotional competence. Mindfulness (N Y). 2010;1:137–51.
Durlak JA, Weissberg RP, Pachan M. A meta-analysis of after-school programs that seek to promote personal and social skills in children and adolescents. Am J Community Psychol. 2010;45:294–309.
Harrell AW, Mercer SH, DeRosier ME. Improving the social-behavioral adjustment of adolescents: the effectiveness of a social skills group intervention. J Child Fam Stud. 2009;18:378–87.
Offord DR. Selection of levels of prevention. Addict Behav. 2000;25:833–42.
Bonevski B, Randell M, Paul C, Chapman K, Twyman L, Bryant J, Brozek I, Hughes C. Reaching the hard-to-reach: a systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC Med Res Methodol. 2014;14:42.
Sen CP, Saucier DA, Hafner E. Meta-analysis of the relationships between social support and well-being in children and adolescents. J Soc Clin Psychol. 2010;29:624–45.
Rothon C, Head J, Klineberg E, Stansfeld S. Can social support protect bullied adolescents from adverse outcomes? A prospective study on the effects of bullying on the educational achievement and mental health of adolescents at secondary schools in East London. J Adolesc. 2011;34:579–88.
Beckner V, Howard I, Vella L, Mohr DC. Telephone-administered psychotherapy for depression in MS patients: moderating role of social support. J Behav Med. 2010;33:47–59.
Thrasher S, Power M, Morant N, Marks I, Dalgleish T. Social support moderates outcome in a randomized controlled trial of exposure therapy and (or) cognitive restructuring for chronic posttraumatic stress disorder. Can J Psychiatr. 2010;55:187–90.
Fox JK, Halpern LF, Ryan JL, Lowe KA. Stressful life events and the tripartite model: relations to anxiety and depression in adolescent females. J Adolesc. 2010;33:43–54.
Meng XH, Tao FB, Wan YH, Hu Y, Wang RX. Coping as a mechanism linking stressful life events and mental health problems in adolescents. Biomed Environ Sci. 2011;24:649–55.
Low NCP, Dugas E, O’Loughlin E, Rodriguez D, Contreras G, Chaiton M, O’Loughlin J. Common stressful life events and difficulties are associated with mental health symptoms and substance use in young adolescents. BMC Psychiatry. 2012;12:116.
Nanni V, Uher R, Danese A. Childhood maltreatment predicts unfavorable course of illness and treatment outcome in depression: a meta-analysis. Am J Psychiatry. 2012;169:141–51.
Henderson CE, Dakof GA, Greenbaum PE, Liddle HA. Effectiveness of multidimensional family therapy with higher severity substance-abusing adolescents: report from two randomized controlled trials. J Consult Clin Psychol. 2010;78:885–97.
Menting ATA, Orobio de Castro B, Matthys W. Effectiveness of the incredible years parent training to modify disruptive and prosocial child behavior: a meta-analytic review. Clin Psychol Rev. 2013;33:901–13.
Flett GL, Hewitt PL, Besser A, Su C, Vaillancourt T, Boucher D, Munro Y, Davidson LA, Gale O. The child–adolescent perfectionism scale: development, psychometric properties, and associations with stress, distress, and psychiatric Symptos. J Psychoeduc Assess. 2016;34:634–52.
Deb S, Strodl E, Sun J. Academic stress, parental pressure, anxiety and mental health among Indian high school students. Int J Psychol Behav Sci. 2015;5:26–34.
O’Connor RC, Rasmussen S, Hawton K. Predicting depression, anxiety and self-harm in adolescents: the role of perfectionism and acute life stress. Behav Res Ther. 2010;48:52–9.
Limburg K, Watson HJ, Hagger MS, Egan SJ. The relationship between perfectionism and psychopathology: a meta-analysis. J Clin Psychol. 2017;73:1301–26.
Quach AS, Epstein NB, Riley PJ, Falconier MK, Fang X. Effects of parental warmth and academic pressure on anxiety and depression symptoms in Chinese adolescents. J Child Fam Stud. 2013;24:106–16.
Zuroff DC, Blatt SJ, Sotsky SM, Krupnick JL, Martin DJ, Sanislow CA, Simmens S. Relation of therapeutic alliance and perfectionism to outcome in brief outpatient treatment of depression. J Consult Clin Psychol. 2000;68:114–24.
Nobel R, Manassis K, Wilansky-Traynor P. The role of perfectionism in relation to an intervention to reduce anxious and depressive symptoms in children. J Ration Emotive Cogn Behav Ther. 2012;30:77–90.
Mitchell JH, Newall C, Broeren S, Hudson JL. The role of perfectionism in cognitive behaviour therapy outcomes for clinically anxious children. Behav Res Ther. 2013;51:547–54.
Festen H, Hartman CA, Hogendoorn S, de Haan E, Prins PJM, Reichart CG, Moorlag H, Nauta MH. Temperament and parenting predicting anxiety change in cognitive behavioral therapy: the role of mothers, fathers, and children. J Anxiety Disord. 2013;27:289–97.
Taylor ZE, Conger RD, Robins RW, Widaman KF. Parenting practices and perceived social support: longitudinal relations with the social competence of Mexican-origin children. J Lat Psychol. 2016;3:193–208.
Pinquart M. Do the parent-child relationship and parenting behaviors differ between families with a child with and without chronic illness? A meta-analysis. J Pediatr Psychol. 2013;38:708–21.
Liber JM, McLeod BD, Van Widenfelt BM, Goedhart AW, van der Leeden AJM, Utens EMWJ, Treffers PDA. Examining the relation between the therapeutic alliance, treatment adherence, and outcome of cognitive behavioral therapy for children with anxiety disorders. Behav Ther. 2010;41:172–86.
Anderson REE, Spence SH, Donovan CL, March S, Prosser S, Kenardy J. Working alliance in online cognitive behavior therapy for anxiety disorders in youth: comparison with clinic delivery and its role in predicting outcome. J Med Internet Res. 2012;14(3):e88. https://doi.org/10.2196/jmir.1848.
Perepletchikova F, Treat TA, Kazdin AE. Treatment integrity in psychotherapy research: analysis of the studies and examination of the associated factors. J Consult Clin Psychol. 2007;75:829–41.
Asscher JJ, Dijkstra S, Stams GJJM, Deković M, Creemers HE. Family group conferencing in youth care: characteristics of the decision making model, implementation and effectiveness of the family group (FG) plans. BMC Public Health. 2014;14:154.
Faul F, Erdfelder E, Lang A-G, Buchner A. G*Power 3:a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Psychon Soc Inc. 2007;39:175–91.
Kearney CA, Graczyk P. A response to intervention model to promote school attendance and decrease school absenteeism. Child Youth Care Forum. 2014;43:1–25.
Garnefski N, Kraaij V. Cognitive emotion regulation questionnaire - development of a short 18-item version (CERQ-short). Pers Individ Dif. 2006;41:1045–53.
Garnefski N, Kraaij V, Spinhoven P. Negative life events, cognitive emotion regulation and depression. Pers Individ Dif. 2001;30:1311–27.
Arrindel WA, de Groot PM, Walburg JA. Handleiding Schaal voor Interpersoonlijk Gedrag. SIG, deel 1. Lisse: Swets & Zeitlinger; 1984.
Bijstra JO, Oostra L. Sociale vaardigheid van adolescenten in kaart gebracht. Kind en Adolesc. 2000;21:166–71.
Conroy DE, Willow JP, Metzler JN. Multidimensional fear of failure measurement: the performance failure appraisal inventory. J Appl Sport Psychol. 2002;14:76–90.
Conroy DE, Coatsworth JD. The effects of coach training on fear of failure in youth swimmers: a latent growth curve analysis from a randomized, controlled trial. J Appl Dev Psychol. 2004;25:193–214.
Van der Ploeg HM. The validation of the Dutch form of test anxiety inventory. Adv Test Anxiety Res Lisse, Netherlands. 1983;2:191–202.
Spielberger CD. Test anxiety inventory: sampler set: manual, test scoring: “test attitude inventory”: preliminary professional manual. Mind Gard; 1980.
Spielberger CD, Gonzalez HP, Taylor CJ, Algaze B, Anton WD. Examination stress and test anxiety. Stress Anxiety. 1978;5:167–91.
Chorpita BF, Yim L, Moffitt C, Umemoto LA, Francis SE. Assessment of symptoms of DSM-IV anxiety and depression in children: a revised child anxiety and depression scale. Behav Res Ther. 2000;38:835–55.
Ferdinand RF, Lang NDJV, Ormel J, Verhulst FC. No distinctions between different types of anxiety symptoms in pre-adolescents from the general population. J Anxiety Disord. 2006;20:207–21.
Shimmer3 GSR+. Shimmer Res. Ltd, Dublin Irel.
Piferi RL, Kline KA, Younger J, Lawler KA. An alternative approach for achieving cardiovascular baseline: viewing an aquatic video. Int J Psychophysiol. 2000;37:207–17.
Stern RA, Arruda JE, Hooper CR, Wolfner GD, Morey CE. Visual analogue mood scales to measure internal mood state in neurologically impaired patients: description and initial validity evidence. Aphasiology. 1997;11:59–71.
Dunn TW, Burlingame GM, Walbridge M, Smith J, Crum MJ. Outcome assessment for children and adolescents: psychometric validation of the youth outcome questionnaire 30.1 (Y-OQ®-30.1). Clin Psychol Psychother. 2005;12:388–401.
WHO. Wellbeing measures in primary health care/the Depcare project. Copenhagen: WHO Reg. Off. Eur; 1998.
De Wit M, Pouwer F, Gemke RJBJ, Delemarre-Van De Waal HA, Snoek FJ. Validation of the WHO-5 well-being index in adolescents with type 1 diabetes. Diabetes Care. 2007;30:2003–6.
Allgaier AK, Pietsch K, Frühe B, Prast E, Sigl-Glöckner J, Schulte-Körne G. Depression in pediatric care: is the WHO-five well-being index a valid screening instrument for children and adolescents? Gen Hosp Psychiatry. 2012;34:234–41.
Rosenberg M. Society and the adolescent self-image. Princeton: Princet Univ Press; 1965.
Franck E, De Raedt R, Barbez C, Rosseel Y. Psychometric properties of the Dutch Rosenberg self-esteem scale. Psychol Belg. 2008;48:25–35.
Van Leeuwen K, Vermulst A, Kroes G, De Meyer G, Veerman J (2011) De verkorte schaal voor ouderlijk gedrag. VSOG voor ouders van jeugdigen van 4 t/m 18 jaar. Nijmegen, Netherlands Kathol. Univ. Leuven/Praktikon bv.
van Eijk LM, Kempen GIJM, van Sonderen FLP. A short scale for measuring social support in the elderly: the SSL12-I. Tijdschr Gerontol Geriatr. 1994;25:192–6.
Kempen GIJM, van Eijk LM. The psychometric properties of the SSL12-I, a short scale for measuring social support in the elderly. Soc Indic Res. 1995;35:303–12.
O’Connor RC, Dixon D, Rasmussen S. The structure and temporal stability of the child and adolescent perfectionism scale. Psychol Assess. 2009;21:437–43.
Wills TA, Cleary S, Filer M, Shinar O, Mariani J, Spera K. Temperament related to early-onset substance use: test of a developmental model. Prev Sci. 2001;2:145–63.
Wills TA, McNamara G, Vaccaro D, Hirky AE. Escalated substance use: a longitudinal grouping analysis from early to middle adolescence. J Abnorm Psychol. 1996;105:166–80.
Stoeber J, Otto K, Stoll O. Multidimensional Inventory of Perfectionism in Sport (MIPS): English version: Sch. Psychol. Univ. Kent; 2006. https://kar.kent.ac.uk/41560/
Stoeber J, Rambow A. Perfectionism in adolescent school students: relations with motivation, achievement, and well-being. Pers Individ Dif. 2007;42:1379–89
O’rourke DJ, Smith RE, Smoll FL, Cumming SP. Trait anxiety in young athletes as a function of parental pressure and motivational climate: is parental pressure always harmful? J Appl Sport Psychol. 2011;23:398–412.
Shirk SR, Saiz CC. Clinical, empirical, and developmental perspectives on the therapeutic relationship in child psychotherapy. Dev Psychopathol. 1992;4:713–28.
DeVet KA, Kim YJ, Charlot-Swilley D, Ireys HT. The therapeutic relationship in child therapy: perspectives of children and mothers. J Clin Child Adolesc Psychol. 2003;32:277–83.
Ormhaug SM, Shirk SR, Wentzel-Larsen T. Therapist and client perspectives on the alliance in the treatment of traumatized adolescents. Eur J Psychotraumatol. 2015;6:27705.
Sin NL, Lyubomirsky S. Enhancing well-being and alleviating depressive symptoms with positive psychology interventions: a practice-friendly meta-analysis. J Clin Psychol. 2009;65:467–87.
We would like to thank the collaborating schools and youth care organizations for their corporation and practical support with this study.
This study is financially supported by a grant from the Netherlands Organisation of Scientific Research (NWO), grant number 400.17.601, work package 3. The funding body had no role in the design of the study, data collection, analysis and interpretation of the data, nor in writing this manuscript or the decision to publish the manuscript.
Availability of data and materials
The data obtained in the current study will be available from the corresponding author on reasonable request after publication of the results on the main research questions.
Ethics approval and consent to participate
The independent Ethical Committee Psychology of Leiden University approved the design of this study (number CEP18–1105/419). Students and parents gave written active informed consent for participation of the student. Additionally, parents and trainers gave written active informed consent for their own participation.
Consent for publication
The authors declare that there are no competing interests with regards to the study.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
van Loon, A.W.G., Creemers, H.E., Vogelaar, S. et al. The effectiveness of school-based skills-training programs promoting mental health in adolescents: a study protocol for a randomized controlled study. BMC Public Health 19, 712 (2019). https://doi.org/10.1186/s12889-019-6999-3
- Randomized controlled trial
- School-based skills-training programs
- Social skills
- Performance anxiety
- Mental health