- Study protocol
- Open Access
Comparative effectiveness of guided internet-based stress management training versus established in-person group training in employees – study protocol for a pragmatic, randomized, non-inferiority trial
BMC Public Health volume 21, Article number: 2177 (2021)
Occupational stress is a major public health challenge that requires a variety of evidence-based preventative approaches to increase their reach within the working population. Behavioral stress management interventions are considered an established approach for occupational stress prevention. Both in-person group-based stress management training (gSMT) and individual Internet-based training (iSMT) have been shown to be effective at reducing stress in employees. However, there remains a lack of evidence on the comparative efficacy of the newer digital format compared to well-established, in-person, group-based training. This study aims (1) to directly compare an evidence-based iSMT with an established gSMT on stress in employees, (2) to analyze the two conditions from a cost perspective, and (3) to explore moderators of the comparative efficacy.
In a randomized, controlled, non-inferiority trial employees from the general working population will be allocated to iSMT or gSMT. The primary outcome will be perceived stress, assessed using the Perceived Stress Scale, three months after randomization. The non-inferiority margin for the primary outcome measure will be set at 2 points (Cohen’s d = 0.29). This trial will also compare the two interventions from a health economics perspective, and conduct explorative analyses to identify potential effect moderators.
To reach a larger proportion of the working population, well-established gSMT should be complemented with interventions that fit today’s society’s increasingly digital lifestyle. The current trial will provide evidence supporting the responsible implementation of Internet-based stress management training if the digital format proves to at least be non-inferior to established group-based training. Additional explorative moderator analyses may guide future practices to aid in matching select programs with select users.
German Register of Clinical Studies (DRKS): DRKS00024892, date of registration: 2021-04-09.
Protocol version: 02, 16-10-2021.
There is meta-analytic evidence documenting an association between work-related stress and an increased risk of developing severe disorders like depression , musculoskeletal conditions  and coronary artery disease [3,4,5]. Work-related stress also accounts for considerable societal costs, the majority due to productivity losses [6, 7].
In recent decades, numerous interventions have been developed and evaluated to protect workers from stress and its adverse health effects [8, 9]. Among the various measures developed to reduce stress, strong evidence exists suggesting beneficial effects from stress management training (SMT) [10,11,12,13]. Typically, SMT is offered as group stress management training (gSMT), mostly led by external trainers [10, 11]. At larger companies, specialized professional units for health and safety usually offer such mental health programs to the employees as in-house training programs. This requires organizational and financial resources, as well as the motivation of enough employees to participate in group training . However, in the United States, for example, about half of all employees currently work at a small or medium-sized company , and similar figures exist for the European Union . Furthermore, employers may not always be aware of the business benefits of employee health interventions . Additionally, some practical issues specific to small businesses may hamper the provision of gSMT. Examples of this are the non-availability of service providers in rural areas  and inadequate numbers of employees to justify in-house group training at small businesses. One alternative is joint group training involving employees from multiple companies. Potential disadvantages of this are that such training might require extra travel-time to reach the venue and a loss in autonomy due to externally-determined time schedules.
To overcome some of the barriers of in-person group stress management training, Internet-based interventions have been advocated as an alternative . They fit with an increasingly digital lifestyle and employees can use Internet-based stress management training (iSMT) at any time and place, at their own pace, and without the time and costs required for travel. Moreover, anonymous participation is possible, if participants prefer to not disclose struggling with stress to their colleagues and/or employer. Studies have shown that iSMT can be effective in different employee groups with, on average, moderate-to-large effects, in terms of reducing stress [19, 20]. In addition, meta-analyses have revealed small to moderate effects on the symptoms of depression and anxiety [20, 21], large effects on insomnia  and even effects on work productivity . ISMT appears to work, either with or without additional human support (i.e., guided by written feedback on training assignments), but guided interventions generate larger mean effects than unguided programs . Moreover, initial studies have shown that iSMT can be cost-effective for both employers  and society , relative to non-active control conditions.
However, the majority of previously published studies compared interventions to non-active control conditions . These results are helpful for deciding whether iSMT is effective in principle. For policy and decision makers, however, it is critical to have robust evidence indicating whether iSMT is as effective at reducing stress as existing group and in-person SMT programs. Currently, there is only indirect evidence that iSMT might be an adequate alternative to established in-person SMT. For example, the effect size of in-person SMT programs on stress was Cohen’s d = .73 in a meta-analysis published by Richardson and Rothstein (2008). Meanwhile, in a recent meta-analysis on Internet-based interventions for employees, Hedges’ effect size g was .76 for guided interventions . This said, there was significant heterogeneity in the two meta-analyses. To this end, we consider it particularly important to compare the effectiveness of these two different types of intervention directly in the same study.
Few randomized controlled trials have compared the effects of stress management training in offered in different forms (i.e., gSMT versus Internet- or computer-based SMT) [25, 26]. Findings to date have revealed no significant difference between these interventions, with regard to stress reduction [25, 26]. Although these studies suggest that employees benefit equally from iSMT and gSMT, such a conclusion must be considered premature. Statements about equivalence or non-inferiority require adequate sample calculations and a pre-defined margin of equivalence or non-inferiority . Given non-inferiority regarding mental health benefits, it is also important to allocate limited financial resources as efficiently as possible. Therefore, trial-based economic evaluations are recommended, since comparing the costs and consequences of gSMT and iSMT can provide valuable information for policymakers .
In the recent years, different iSMT programs, like “GET.ON Stress”, have been implemented in routine care. This program has already been evaluated in a series of randomized controlled trials in the general working population with elevated stress, demonstrating significant effects, in terms of reducing stress and depressive symptoms, relative to be being on a waiting list [29,30,31]. Moreover, economic evaluation of the intervention indicates that it is likely to be of good value for the money in occupational healthcare . In this trial, the iSMT “GET.ON Stress” will be compared to the established in-person gSMT intervention — “Gelassen und sicher im Stress “[Calm and safe under stress] [32, 33] — which may be the most often used stress management training program in German-speaking countries and has been demonstrated in a randomized controlled trial to be effective at reducing stress, as well [33, 34].
As for any complex intervention, beyond comparative efficacy and cost-effectiveness, it is important to obtain insights into moderators of the SMT under study . While assessing the non-inferiority of iSMT, in terms of reducing stress, it would also be valuable to explore if there are indicators that identify who benefits most from which SMT format. Research on moderators of either format, however, is scarce. For group stress management training, different intervention- and person-related moderators of efficacy have already been studied (e.g., intervention length, an intervention’s number of components, industry sector)  without generating clear conclusions for implementation in occupational settings. On average, iSMT yields greater effects when complemented with guidance through personal contact, contrary to pure self-help interventions [19, 20] and when it takes no more than nine weeks to finish the intervention . Identifying moderators for each of these formats would provide novel information for intervention developers and decision makers, both in public and occupational health, especially if non-inferiority is established. For example, one study evaluating how well different health service formats were accepted by respondents of the general population revealed that people found face-to-face counselling and therapies more useful than Internet-based counselling, and that participants reported greater intentions to use face-to-face services to address emotional problems . Similarly, respondents to another survey preferred traditional face-to-face stress management over Internet-based stress management . To our knowledge, no study to date has empirically investigated the characteristics of the different training formats as potential moderators of these interventions’ efficacy.
In keeping with the above issues, the primary objective of this pragmatic trial is (1) to test the hypothesis that the iSMT intervention “GET.ON Stress”  is non-inferior to the gSMT intervention “Gelassen und sicher im Stress – calm and safe under stress” [32, 33] in employees, in terms of reducing perceived stress. Further aims are (2) to compare the two conditions from a cost perspective, and (3) to explore moderators of this comparative efficacy.
The declaration of Helsinki for Ethical Principles for Medical Research Involving Human states that “The benefits, risks, burdens and effectiveness of a new intervention must be tested against those of the best proven intervention(s)” . Accordingly, we will test the non-inferiority and cost-effectiveness of the relatively new iSMT intervention “GET.ON Stress”  against the best proven and probably most adopted intervention for stress management in Germany speaking countries, that is the gSMT intervention “Gelassen und sicher im Stress – calm and safe under stress” [32, 33]. In a randomized controlled trial, one group of participants will receive access to iSMT, whereas another group will participate in gSMT.”
Outcome assessments will take place at baseline (T1), and both three months (T3) and six months (T4) after allocation to the study conditions. To mimic real-life occupational practice, participants in iSMT will be granted access to the intervention immediately after randomization, whereas participants in gSMT will have to wait until a sufficiently large number of participants has been recruited and randomized into this condition to fill group sessions. For this reason, participants in each condition will also need to attend in a short extra assessment (T2) right before their first training session.
The primary outcome will be the level of perceived stress at T3. There will be no restrictions to the use of other care as usual (CAU). This study protocol describes the design of the pragmatic trial based on Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guidelines . The study also will be conducted in accordance with the Consolidated Standards of Reporting Non-inferiority Trials (CONSORT) .
The target group will consist of workers from small- and medium-sized enterprises. For pragmatic reasons, the study will take place in the Rhine-Ruhr metropolitan region in Germany, which has a population of approximately 10 million people. Participants must a) be employed, b) have access to the Internet to complete online training sessions weekly, c) be able to visit gSMT sessions in their local area, d) give their informed consent to participate, and e) complete the baseline assessment (T1). Referring to national guidelines for prevention, we keep the exclusion criteria at a minimum. Applicants a) who are participating in another stress management training at the time of study registration will be excluded. To ensure participants’ safety, those b) with an elevated risk of suicide, indicated as a score > 1 on item 9 of the Beck Depression Inventory , will be also excluded and receive an email containing information about how to obtain adequate help and listing the telephone numbers of relevant services.
Recruitment and procedures
Recruitment will occur in the winter through summer of 2022, employing two major strategies. To attract interested organizations, we will use different media channels to motivate owners of small- and medium-sized companies (staff headcount < 250) in the target region to offer the intervention to their workforce. To attract individuals, health insurance companies will advertise the study via announcements in print membership magazines and media channels (e.g., Facebook, Twitter). Registration into the study will take place at an individual level, so employees need not self-disclose to their employer that they want to participate. An open-access website will provide information on study conditions. Applicants will sign up by providing an email address and name or pseudonym on the website. Applicants who fulfill all inclusion criteria assessed by an online screening questionnaire, return their informed consent, and complete the baseline assessment (T1) will receive a phone call by a research assistant to inform them about further requirements for inclusion (e.g., time scheduling for each of the interventions) and to answer the applicants’ questions. Thereafter, each applicant will be randomly allocated to one of the study conditions (Fig. 1), with randomization stratified by whether or not employees work in companies concurrently conducting organizational mental health programs. Allocation concealment will be ensured by an independent researcher not otherwise involved in this study, who will randomize study participants with a separate randomization sequence for each strata (concurrent organizational health program = yes/no). We will use a restricted randomization procedure using varying block sizes of 4 to 6 generated by the software RandList (randomization.eu). This researcher will send randomization results to the study administration responsible for subject allocation. For practical reasons, blinding of participants will not be possible.
After they are informed about the outcome of randomization, participants offered iSMT will receive immediate access to the program. Participants allocated to gSMT will receive information about the times and locations of group sessions; they then will need to select in which workshop they want to participate. Just before their first training session, all participants will need to complete a short pre-intervention assessment (T2). Participants will receive up to three automated e-mail reminders if they did not attend the assessments. All assessment and Internet-based training data will be collected using a secure assessment and training system (AES, 256-bit encrypted). Both the web-based assessment system and training platform are physically located on servers belonging to Leuphana University of Lueneburg. The principal investigator DL and the first author LB will have full access to the final dataset. Two researchers will assure data integrity, independently.
Internet-based stress management training
The iSMT program “GET.ON Stress” [29, 41] was designed to enhance two strategies of stress coping: problem solving  and emotion regulation  (Table 1). The intervention consists of seven modules that participants should work on following a weekly schedule. Each module consists of general information; interactive exercises; prototype training participants – so called personas – who represent different stressed employee groups; quizzes; audio and video files; and downloadable work sheets. In addition, at the end of sessions 2 to 6, users can choose to attain extra information and perform short exercises about the following common stress-related topics: time management, rumination and worrying, psychological detachment from work, sleep hygiene, sleeping habit rhythm and regularity, nutrition and exercise, organization of breaks during work, and social support [29, 41]. For this trial, we adapted GET.ON Stress to employees working at small- to medium-size enterprises, the adaptation pertaining to the personas within the online program who guide participants through the program. Personas are a well-established element of user-centered design in software engineering  that has been also used to tailor Internet-based interventions to specific target groups [45, 46]. Within GET.ON Stress, the personas fulfill several functions, following the Efficiency Model of Support . The integration of personas aims to increase user engagement in the iSMT program, providing knowledge about how to complete the exercises within the program and helping users to transfer what they learn from the exercises into their daily lives; for instance, by giving examples of how employees working under similar circumstances apply a given problem-solving strategy in their daily life. In addition to program content, participants will receive written feedback from an e-coach on their exercises after each of training module, the feedback provided in accordance with the training manual. E-coaches will be psychotherapists or master’s degree-level psychologists. Based upon our experiences from previous studies, we anticipate that the e-coaches will spend roughly 30 min per feedback. To improve participants’ adherence with the intervention, the e-coaches will send reminders to participants any time they fail to complete a training module within seven days. All communication between the participant and the e-coach will take place in a secured, web-based, open-source platform, located at the Leuphana University of Lueneburg.
In-person group stress management training
The gSMT program “Gelassen und sicher im Stress “[Calm and safe under stress] [32, 33] was one of the first stress management training programs developed in Germany to prevent mental disorders and somatic disease. It is based on cognitive-behavioral techniques and conducted in groups. Since its development, it has become very influential in health promotion practice and is considered standard stress-management training in Germany. The training program consists of four major modules, incorporating different strategies for coping with stress: progressive muscle relaxation; problem-solving techniques; cognitive restructuring of dysfunctional attitudes; and enhancement of pleasant activities. To mirror everyday practices using “Calm and safe under stress”, all the while optimizing the intervention’s effects, present-trial participants will take part in three training sessions, each lasting roughly three hours over seven weeks (Table 2), with a maximum of 15 participants in each group. The group sessions will take place in seminar rooms in the target region, rented by the research team. A stress management trainer will lead every training group. In each training session, the trainer will provide an introduction describing the forthcoming exercises. Within the exercises, participants will provide examples of their daily work life. Participants then will complete the exercises within the whole group, in pairs, or by themselves. All trainers will be either psychologists or psychotherapists who have participated in a five-day train-the-trainer program, held by the developer of the ‘Calm and safe under stress’ training concept. A point-by-point description of both interventions, based on the template for intervention description and replication (TIDieR) , can be viewed in the Supplementary Information.
The primary outcome will be the level of self-rated stress (Table 3), measured using the Perceived Stress Scale (PSS-10) , a validated measure of stress that has been used extensively during evaluations of both in-person and Internet-based stress-management interventions. The PSS-10 assesses the extent to which participants experience their lives as stressful (e.g., as overstraining, unmanageable, and/or unforeseeable over the past month). It consists of ten items, each having the following 5-point Likert scale response options of 0 = never; 1 = almost never; 2 = sometimes; 3 = fairly often; and 4 = very often (summation score range 0–40). In this trial, respondents will answer items referring only to the past week. The German version of the scale has been validated in the general population . An online version of the German scale has also been validated in samples of German employees, exhibiting good reliability with McDonald’s omega (ω) = .89 and good measurement invariance .
To assess negative mental health consequences of stress, reflected as depressive symptoms, we will use the 15-item version of the German adaptation of the Center for Epidemiological Studies Depression Scale (CES-D) [52, 53]. In addition, we will use the German version of the Brief Resilience Scale (BRS) [54, 55] to assess changes in participants’ ability to recover from stress. To assess work-related stress we will use the relatively new Perceived Occupational Stress scale (POS) , translated to German.
We will use different measures for the training evaluation. To assess user satisfaction with the intervention received, we will use the Client Satisfaction Questionnaire (CSQ) [57, 58] and the adaptation CSQ-I  to measure satisfaction with Internet-based interventions. Adherence to the interventions will be measured using log data from iSMT participants and self-rated questions for all participants (e.g., “How often have you tried to apply tips and techniques you have learned in your daily life?”). Within gSMT, trainers will be instructed to record participants’ attendance at every training session. We also will assess the extent to which trainers adhere to the training manual (e.g., “How well did you manage to communicate the intended training content?”).
To collect health service utilization and productivity losses, we will use the Treatment Inventory of Costs in Psychiatric Patients (TIC-P) , adapted to the German context. This questionnaire includes different single questions to estimate the impact of presenteeism on productivity losses. To estimate productivity loss due to impaired performance directly, we adapted an item extracted from the Health and Labor Questionnaire (HLQ) : “If you now have to catch up on work that you have not done in the last 4 weeks due to your health problems, how many hours would you have to work?”. For cost-utility analyses, we will calculate quality-adjusted life years (QUALYs), based on the Assessment of Quality of Life (AQoL-6D) .
We will explore potential moderating variables of iSMT and gSMT that might affect the comparative efficacy of the two interventions: participant age, gender, technology readiness, training experience (online vs. in-person group, yes/no), training expectations, and preferences for Internet-based or group training. To assess technology readiness, we will use the Technology Readiness Index 2.0 (TRI) . We will use the Patient Questionnaire on Therapy Expectation and Evaluation (PATHEV) , adapted to stress-management training, to assess outcome expectancies and suitability, with regard to the training received. To explore the potential impact of workplace conditions on stress, we will use the German Version of the Effort-Reward-Imbalance Questionnaire – Short Form , which has reliably shown associations between work-related stress and both physical  and mental health  impairment. In addition, we will use the German version of the Job Crafting Scale, by Tims and Bakker [67, 68], which measures self-initiated changes that employees make in their own job demands and job resources to optimize their personal goals. We will use the three items with the highest item-scale-correlation for each subdomain , resulting in twelve questions for the whole scale. In addition, we will use self-developed questions to explore preferences for either of the training formats under study (e.g., for iSMT: “It was important, to me, that I was able to do the training at my own pace”; for gSMT: “It was important, to me, that I was in contact with other participants in one place”).
We used a multi-method approach to calculate the sample size required for this trial . Sample size was calculated using a non-inferiority criterion, based on previous meta-analytic evidence for the reference group (gSMT). In a systematic meta-analysis, Richardson & Rothstein identified an effect of d = 0.73 on stress reduction for occupational stress management interventions, relative to being on a waiting list . This effect would correspond to 4.6 points on the PSS, assuming a standard deviation of 6.3 points, derived from normative sample data from German employees . In addition, we took anchor-based clinical judgement of a meaningful difference in primary outcome measure scores into account. We conducted two focus groups with clinical (n = 4) and occupational health experts (n = 4). All the focus group members agreed that a reduction of at least 3 points on the PSS would display a meaningful difference for people with elevated levels of stress at baseline (PSS score ≥ 22). Finally, we set the non-inferiority margin ∆ at 2 points on the PSS at T3. Based on the predefined margin, we need 314 subjects to reject the null hypothesis of inferiority of the iSMT with 80% statistical power, assuming deterioration of not more than 2 points relative to gSMT, with two-sided α = .05. Based upon previous studies on the Internet-based intervention [29, 30], we expect that approximately 30% of participants will drop out of the study during the intervention phase (between T1 and T3). Accordingly, we aim to recruit 448 participants.
Primary outcome analysis
Referring to the CONSORT guidelines for non-inferiority trials , our analysis of non-inferiority will be limited to participants who adhere to the study protocol (per-protocol analysis). That means we will only include participants who have been randomly allocated, have complete data, and attended at least five out of seven iSMT or gSMT sessions. To declare non-inferiority, the upper bound of the 95% confidence interval (CI) for the difference in mean scores on the PSS between iSMT and gSMT must be below the margin ∆. For non-inferiority, we also will conduct a separate analysis based on an intention-to-treat (ITT) sample, including all randomly-allocated subjects, to test for iSMT superiority with a one-sided alpha error set at α = .05. This procedure has been found appropriate for non-inferiority evaluations [27, 70]. For analysis of the primary outcome, we will use linear models adjusted for the baseline outcome value and stratification variable. The primary outcome will be perceived stress at T3.
Secondary outcome analyses
Because empirical evidence on the comparative efficacy and utility of both intervention formats under investigation is scarce, we will conduct a variety of explorative analyses on secondary outcomes. We will analyze improvements in perceived stress at an individual level in both study groups, by examining the number of participants who exhibit “reliable improvement”, employing the reliable change index (RCI) proposed by Jacobson and Truax in 1991. Participants will be defined as reliably improved if their PSS-10-score declines, from T1 to T3, with a reliable change index greater than 1.96, which equals 5.8 points on the PSS-10, based on an SDpost = 6.3  and Mc Donald’s ω = .89 . Response rates will be analyzed using the Pearson χ2 statistic.
For all other continuous outcomes (Table 3), we will conduct separate linear model analyses examining for any superiority of iSMT over gSMT. Group differences in newly developed Items (e.g., user preferences towards using Internet-based vs. in-person training) will be analyzed using non-parametric statistics. In addition, for all outcomes and assessment points, Cohen’s d will be calculated to quantify the size of interventional effects by subtracting the average score of the Internet-based condition from the average score of the group condition and dividing the result by the pooled standard deviations at the corresponding assessment point.
We will perform economic evaluations from different perspectives. From an employer’s perspective, we will conduct cost-benefit analysis (CBA), considering opportunity costs due to the time required for employees to partake of the intervention and changes in productivity loss due to reduced absenteeism and presenteeism. Since health insurance companies in several countries are legally obligated to fund preventative actions, we will conduct a separate CBA from an insurer’s perspective, including intervention costs, reflected as the common market prices for the interventions. In addition, we will conduct cost-effectiveness analysis (CEA). For the CEA, we will calculate intervention costs and costs due to productivity loss and effects of the interventions, in terms of the number of reliable improvements in perceived stress at T4. The costs and effects of both interventions will then be combined in an incremental cost-effectiveness ratio . A non-parametric boot-strapping method with 95% confidence intervals will be used to account for the uncertainty of differences between the two intervention groups. Finally, we will determine the probability that the intervention is cost effective for different values of willingness-to-pay per effect unit.
Furthermore, we will explore different moderators that might affect the comparative efficacy of iSMT vs. gSMT: patient age, gender, training experiences, technology readiness, training expectations, preferences towards in-person group- or Internet-based training, working conditions in terms of an effort-reward-imbalance, and job crafting. To explore whether any of these characteristics moderate either intervention’s effect, we will conduct separate analyses. For each potential moderator, we will add to the statistical model the potential moderator as the main effect, as well as the interaction effect between the moderator and study condition on the primary outcome.
The currently planned study aims to examine for non-inferiority of a guided, Internet-based stress management training (iSMT) program, relative to an established, in-person group-based, stress management training (gSMT) program, in terms of their effectiveness reducing stress in employees. To the best of our knowledge, this is the first non-inferiority trial comparing the relatively new iSMT format versus an established SMT format. Findings from the present trial will broaden evidence on SMTs in several ways.
First, there is broad evidence supporting the efficacy of both traditional occupational stress management (i.e., in-person gSMT) [9, 10, 13] and iSMT [19,20,21,22] in employees. However, data comparing the efficacy of this digital training format against an established format of stress management are limited. Previous studies performing direct comparisons [25, 26] were either not powered to demonstrate iSMT non-inferiority or equivalence, relative to gSMT, or identified no improvement with iSMT . Likewise, in the field of psychotherapy, meta-analyses directly comparing these two formats revealed comparable effects on psychiatric symptomology with Internet-based and established face-to-face psychotherapy [71, 72]. Although some authors are optimistic about the comparable effect sizes of Internet interventions and face-to-face interventions, such results should be interpreted with caution. Hardly any prior study was designed to directly investigate non-inferiority or equivalence employing state-of-the-art methodology . Most importantly, it seems unclear if the face-to-face comparator represents the gold-standard intervention recommended by respective national guidelines. For example, in a German-speaking sample , eight sessions of group therapy for depression served as a comparator. However, in Germany, outpatient therapy for depression is usually individual therapy, with a minimum of 25 sessions for CBT. Therefore, previous studies may be biased, by design, in favor of digital interventions, leading to false conclusions and offering people less-effective interventions. To overcome those shortcomings, in the present trial we will use an evidence-based and widely-applied group stress management intervention in Germany as a gold-standard comparator.
Second, the trial will conduct economic analyses comparing two stress-management formats. This is of importance for employers and policy-makers seeking to adapt strategies for occupational healthcare. Especially at small- to medium-sized companies, resources for occupational health are often limited; consequently, it is important to allocate financial resources as efficiently as possible . To the best of our knowledge, this is the first trial to include an economic comparison between an Internet-based stress-management intervention and a group intervention.
Third, although non-inferiority is assumed, certain subgroups might benefit more from one stress management training format versus the other. For gSMT  and for iSMT [19, 20], different intervention- and person-related moderators of the intervention have already been studied. In the present trial, we will explore a variety of moderators that may guide future practices and research on selective indications for either iSMT or gSMT. The identification of format-specific moderators (e.g., preferences towards one format over another) could help employers and health service providers to offer suitable interventions for specific target groups. Likewise, person-related moderators, like technology readiness, may provide important information about the basic requirements needed before successful implementation of new technology.
In the case of non-inferiority, Internet-based stress management training is complementary to established, in-person group training in occupational settings. Furthermore, if participants in iSMT experience additional benefits from this format, relative to gSMT (e.g., in terms of cost-effectiveness, availability, flexibility of use in terms of time and/or place, other aspects of convenience), iSMT would be a promising solution, especially for those companies with few resources for occupational health promotion.
Despite the potential contributions of the present study, several limitations must be considered. First, participants allocated to the Internet-based condition will be offered immediate access to the online intervention, whereas participants assigned to the in-person intervention will have to wait some time before their first workshop session. Although we will make efforts to hold the workshops promptly and to schedule these training workshops as soon as possible, we expect that people in this intervention group will start considerably later than those assigned to the Internet-based treatment group. This said, this difference between these two interventions reflects the real-life characteristics of the two training formats and contribute to the study’s external validity. Second, group allocation cannot consider participant preferences, which limits conclusions regarding the implementation and actual reach of the two formats in the general employee population [74, 75]. Due to the randomized trial design, it is difficult to compare the attractiveness of the two interventions, in terms of uptake rates. Consequently, whether the Internet-based intervention shows non-inferiority or even superiority over group-based training, it remains unclear which training format would reach more employed individuals and make a greater contribution to stress reduction in the population. In psychotherapy, surveys suggest that people prefer traditional over digital care , but little is known about employees’ preferences regarding SMT. To address these issues, we will measure participants’ preferences towards the two intervention formats. Insights from this trial will provide a basis for allocating resources to iSMT or gSMT and facilitate future studies offering participants the opportunity to choose their preferred format. Third, digitalization of the intervention format (Internet-based vs. in-person) may be the most obvious distinctive feature of the two interventions in the present study. However, the interventions differ in several other characteristics, like the social context (individual-based vs. group-based training), major coping strategies used (problem solving plus emotional regulation techniques vs. problem solving plus cognitive restructuring), and mode of communication (asynchronous vs. synchronous). Therefore, findings from this trial cannot be attributed solely to one interventional feature. Nonetheless, the two interventions have in common that they have already been evaluated in randomized controlled trials, are offered widely in routine practice nationally, are considered gold-standard programs for either iSMT or gSMT, and offer real-life scenarios for the delivery of Internet-based and traditional healthcare programs for stress prevention . Finally, participants will experience stress elicited by a great variety of work-related stressors. The Effort-Reward-Imbalance Questionnaire as a short measure for work-related stressors is unlikely to provide a multidimensional picture of the workplace situation according to, for example, the Health and Safety Executive Management Standards . This would require more detailed and substantially longer assessments. However, identifying the sources of stress in detail is a complex task and require elaborated epidemiological research that is hardly feasible to conduct alongside an intervention study.
Stress reduction is a major challenge that requires a variety of evidence-based approaches to reach a greater proportion of the population. Therefore, standard formats of stress-management training should be complemented with interventions that fit society’s increasingly digital lifestyle. The current study will provide evidence about the non-inferiority, cost-effectiveness, and potential effect moderators of guided Internet-based stress management compared to in-person group-based stress management. Its findings might be an important future resource for occupational health and policy decision makers.
Availability of data and materials
Internet-based stress management training
Intention to treat
Group-based stress management training
Madsen IEH, Nyberg ST, Magnusson Hanson LL, Ferrie JE, Ahola K, Alfredsson L, et al. Job strain as a risk factor for clinical depression: systematic review and meta-analysis with additional individual participant data. Psychol Med. 2017;47:1342–56.
Hauke A, Flintrop J, Brun E, Rugulies R. The impact of work-related psychosocial stressors on the onset of musculoskeletal disorders in specific body regions: a review and meta-analysis of 54 longitudinal studies. Work Stress. 2011;25:243–56.
Sara JD, Prasad M, Eleid MF, Zhang M, Widmer RJ, Lerman A. Association between work-related stress and coronary heart disease: a review of prospective studies through the job strain, Effort-Reward Balance, and Organizational Justice Models. J Am Heart Assoc. 2018;7:e008073.
Dragano N, Siegrist J, Nyberg ST, Lunau T, Fransson EI, Alfredsson L, et al. Effort-reward imbalance at work and incident coronary heart disease: a multicohort study of 90,164 individuals. Epidemiology. 2017;28:619–26.
Kivimäki M, Nyberg ST, Batty GD, Fransson EI, Heikkilä K, Alfredsson L, et al. Job strain as a risk factor for coronary heart disease: a collaborative meta-analysis of individual participant data. Lancet. 2012;380:1491–7.
Hassard J, Teoh KRH, Visockaite G, Dewe P, Cox T. The cost of work-related stress to society: a systematic review. J Occup Health Psychol. 2018;23:1–17.
Cocker F, Martin A, Scott J, Venn A, Sanderson K. Psychological distress, related work attendance, and productivity loss in small-to-medium Enterprise owner/managers. Int J Environ Res Public Health. 2013;10:5062–82.
Joyce S, Modini M, Christensen H, Mykletun A, Bryant R, Mitchell PB, et al. Workplace interventions for common mental disorders: a systematic meta-review. Psychol Med. 2016;46:683–97.
Bhui KS, Dinos S, Stansfeld SA, White PD. A synthesis of the evidence for managing stress at work: a review of the reviews reporting on anxiety, depression, and absenteeism. J Environ Public Health. 2012;2012:1–21.
Richardson KM, Rothstein HR. Effects of occupational stress management intervention programs: a meta-analysis. J Occup Health Psychol. 2008;13:69–93.
Bartlett L, Martin A, Neil AL, Memish K, Otahal P, Kilpatrick M, et al. A systematic review and meta-analysis of workplace mindfulness training randomized controlled trials. J Occup Health Psychol. 2019;24:108–26.
Tan L, Wang M-J, Modini M, Joyce S, Mykletun A, Christensen H, et al. Preventing the development of depression at work: a systematic review and meta-analysis of universal interventions in the workplace. BMC Med. 2014;12:74.
Ruotsalainen JH, Verbeek JH, Mariné A, Serra C. Preventing occupational stress in healthcare workers. Cochrane Database Syst Rev. 2015;2015:CD002892.
Harris JR, Hannon PA, Beresford SAA, Linnan LA, McLellan DL. Health promotion in smaller workplaces in the United States. Annu Rev Public Health. 2014;35:327–42.
United States Census Bureau. 2017 SUSB Annual Data Tables by Establishment Industry. https://www.census.gov/data/tables/2017/econ/susb/2017-susb-annual.html. Accessed 23 Nov 2021.
Eurostat. Small and medium-sized enterprises: an overview. https://ec.europa.eu/eurostat/de/web/products-eurostat-news/-/EDN-20191125-1. Accessed 23 Nov 2021.
European Agency for Safety and Health at work (EU-OSHA). Second European Survey of Enterprises on New and Emerging Risks (ESENER-2). https://osha.europa.eu/sites/default/files/esener-ii-summary-en.PDF. Accessed 23 Nov 2021.
Lehr D, Geraedts A, Asplund RP, Khadjesari Z, Heber E, de Bloom J, et al. Healthy at work. In: Wiencke M, Cacace M, Fischer S, editors. Heal work – Interdiscip Perspect. Cham: Springer International Publishing; 2016. p. 257–81.
Heber E, Ebert DD, Lehr D, Cuijpers P, Berking M, Nobis S, et al. The benefit of web- and computer-based interventions for stress: a systematic review and Meta-analysis. J Med Internet Res. 2017;19:e32.
Phillips EA, Gordeev VS, Schreyögg J. Effectiveness of occupational e-mental health interventions: a systematic review and meta-analysis of randomized controlled trials. Scand J Work Environ Health. 2019;45:560–76.
Stratton E, Lampit A, Choi I, Calvo RA, Harvey SB, Glozier N. Effectiveness of eHealth interventions for reducing mental health conditions in employees: A systematic review and meta-analysis. PLoS One. 2017;12:e0189904.
Carolan S, Harris PR, Cavanagh K. Improving employee well-being and effectiveness: systematic review and Meta-analysis of web-based psychological interventions delivered in the workplace. J Med Internet Res. 2017;19:e271.
Ebert DD, Kählke F, Buntrock C, Berking M, Smit F, Heber E, et al. A health economic outcome evaluation of an internet-based mobile-supported stress management intervention for employees. Scand J Work Environ Health. 2018;44:171–82.
Kählke F, Buntrock C, Smit F, Berking M, Lehr D, Heber E, et al. Economic evaluation of an internet-based stress management intervention alongside a randomized controlled trial. JMIR Ment Heal. 2019;6:e10866.
Wolever RQ, Bobinet KJ, McCabe K, Mackenzie ER, Fekete E, Kusnick CA, et al. Effective and viable mind-body stress reduction in the workplace: a randomized controlled trial. J Occup Health Psychol. 2012;17:246–58.
Eisen KP, Allen GJ, Bollash M, Pescatello LS. Stress management in the workplace: a comparison of a computer-based and an in-person stress-management intervention. Comput Human Behav. 2008;24:486–96.
Piaggio G, Elbourne DR, Pocock SJ, Evans SJW, Altman DG. Reporting of noninferiority and equivalence randomized trials. JAMA. 2012;308:2594.
van Dongen JM, van Wier MF, Tompa E, Bongers PM, van der Beek AJ, van Tulder MW, et al. Trial-based economic evaluations in occupational health. J Occup Environ Med. 2014;56:563–72.
Heber E, Lehr D, Ebert DD, Berking M, Riper H. Web-based and Mobile stress management intervention for employees: a randomized controlled trial. J Med Internet Res. 2016;18:e21.
Ebert DD, Lehr D, Heber E, Riper H, Cuijpers P, Berking M. Internet- and mobile-based stress management for employees with adherence-focused guidance: efficacy and mechanism of change. Scand J Work Environ Health. 2016;42:382–94.
Ebert DD, Heber E, Berking M, Riper H, Cuijpers P, Funk B, et al. Self-guided internet-based and mobile-based stress management for employees: results of a randomised controlled trial. Occup Environ Med. 2016;73:315–23.
Kaluza G. Stressbewältigung - Trainingsmanual zur psychologischen Gesundheitsförderung. Berlin, Heidelberg: Springer Berlin Heidelberg; 2015.
Kaluza G. Effekte eines kognitiv-behavioralen Streßbewältigungstrainings auf Belastungen, Bewältigung und (Wohl-)Befinden - eine randomisierte, kontrollierte, prospektive Interventionsstudie in der primären Prävention. Z Klin Psychol. 1998;27:234–43.
Kaluza G. Sind die Effekte eines primärpräventiven Streßbewältigungstrainings von Dauer? Eine randomisierte, kontrollierte follow-up-Studie [are the effects of a stress management training in primary prevention long-lasting? - a randomized, controlled follow-up study]. Eur J Heal Psychol. 1999;7:88–95.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Int J Nurs Stud. 2013;50:587–92.
Apolinário-Hagen J, Fritsche L, Bierhals C, Salewski C. Improving attitudes toward e-mental health services in the general population via psychoeducational information material: a randomized controlled trial. Internet Interv. 2018;12:141–9.
Apolinário-Hagen J, Hennemann S, Kück C, Wodner A, Geibel D, Riebschläger M, et al. Exploring user-related drivers of the early acceptance of certified digital stress prevention programs in Germany. Heal Serv Insights. 2020;13:117863292091106.
World Medical Association. World medical association declaration of HelsinkiEthical principles for medical research involving human subjects. JAMA. 2013;310:2191.
Chan A-W, Tetzlaff JM, Gotzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346:e7586.
Beck AT, Steer A, Brown GK. BDI-II: Beck depression inventory manual. 2nd ed. San Antonio: Psychological Corporation; 1996.
Heber E, Ebert DD, Lehr D, Nobis S, Berking M, Riper H. Efficacy and cost-effectiveness of a web-based and mobile stress-management intervention for employees: design of a randomized controlled trial. BMC Public Health. 2013;13:655.
D’Zurilla TJ, Nezu AM. Problem-solving therapies. In: Dobson KS, editor. Handb Cogn Ther. 2nd ed. New York: Guilford; 2001. p. 211–45.
Berking M, Whitley B. Affect regulation training: a practitioners’ manual. New York: Springer; 2014.
Cooper A, Reimann R, Cronin D. About face 3. The Essentials of Interaction Design. Indianapolis: Wiley Publishing, Inc; 2007.
Holden RJ, Kulanthaivel A, Purkayastha S, Goggins KM, Kripalani S. Know thy eHealth user: development of biopsychosocial personas from a study of older adults with heart failure. Int J Med Inform. 2017;108:158–67.
Serio CD, Hessing J, Reed B, Hess C, Reis J. The effect of online chronic disease personas on activation: within-subjects and between-groups analyses. JMIR Res Protoc. 2015;4:e20.
Schueller SM, Tomasino KN, Mohr DC. Integrating human support into behavioral intervention technologies: the efficiency model of support. Clin Psychol Sci Pract. 2017;24:27–45.
Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687.
Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983;24:385–96.
Klein EM, Brähler E, Dreier M, Reinecke L, Müller KW, Schmutzer G, et al. The German version of the perceived stress scale – psychometric characteristics in a representative German community sample. BMC Psychiatry. 2016;16:159.
Reis D, Lehr D, Heber E, Ebert DD. The German version of the perceived stress scale (PSS-10): evaluation of dimensionality, validity, and measurement invariance with exploratory and confirmatory Bifactor modeling. Assessment: SAGE Publications Inc; 2017.
Radloff LS. The CES-D Scale. Appl Psychol Meas. 1977;1:385–401.
Hautzinger M, Bailer M, Hofmeister D, Keller F. Allgemeine Depressionsskala (ADS) [Center for Epidemiological Studies Depression Scale (CES-D; Radloff, L.S., 1977) - German adaptation.]. Psychiatr Prax. 2012;39:302–4.
Smith BW, Dalen J, Wiggins K, Tooley E, Christopher P, Bernard J. The brief resilience scale: assessing the ability to bounce back. Int J Behav Med. 2008;15:194–200.
Chmitorz A, Wenzel M, Stieglitz R-D, Kunzler A, Bagusat C, Helmreich I, et al. Population-based validation of a German version of the brief resilience scale. Tran US, editor. PLoS One. 2018;13:e0192761.
Marcatto F, Di Blas L, Luis O, Festa S, Ferrante D. The perceived occupational stress scale: a brief tool for measuring workers’ perceptions of stress at work. Eur. J Psychol Assess. 2021. https://doi.org/10.1027/1015-5759/a000677.
Schmidt J, Lamprecht F, Wittmann WW. Zufriedenheit mit der stationären Versorgung. Entwicklung eines Fragebogens und erste Validitätsuntersuchungen [Satisfaction with inpatient management. Development of a questionnaire and initial validity studies]. Psychother Psychosom Med Psychol. 1989;39:248–55.
Attkisson CC, Zwick R. The client satisfaction questionnaire - psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plann. 1982;5:233–7.
Boß L, Lehr D, Reis D, Vis C, Riper H, Berking M, et al. Reliability and validity of assessing user satisfaction with web-based health interventions. J Med Internet Res. 2016;18:e234.
Bouwmans C, De Jong K, Timman R, Zijlstra-Vlasveld M, Van der Feltz-Cornelis C, Tan Swan S, et al. Feasibility, reliability and validity of a questionnaire on healthcare consumption and productivity loss in patients with a psychiatric disorder (TiC-P). BMC Health Serv Res. 2013;13:217.
Van Roijen L, Essink-bot M-L, Koopmanschap MA, Bonsel G, Rutten FFH. Labor and health status in economic evaluation of health care: the health and labor questionnaire. Int J Technol Assess Health Care. 1996;12:405–15.
Richardson JR, Peacock SJ, Hawthorne G, Iezzi A, Elsworth G, Day NA. Construction of the descriptive system for the assessment of quality of life AQoL-6D utility instrument. Health Qual Life Outcomes. 2012;10:38.
Parasuraman A, Colby CL. An updated and streamlined technology readiness index. J Serv Res. 2015;18:59–74.
Schulte D. Messung der Therapieerwartung und Therapieevaluation von Patienten (PATHEV) [measurement of patient’s therapy expectancy and therapy evaluation]. Z Klin Psychol Psychother. 2005;34:176–87.
Siegrist J, Wege N, Pühlhofer F, Wahrendorf M. A short generic measure of work stress in the era of globalization: effort-reward imbalance. Int Arch Occup Environ Health. 2009;82:1005–13.
Rugulies R, Aust B, Madsen IE. Effort-reward imbalance at work and risk of depressive disorders. A systematic review and meta-analysis of prospective cohort studies. Scand J Work Environ Health. 2017;43:294–306.
Tims M, Bakker AB, Derks D. Development and validation of the job crafting scale. J Vocat Behav. 2012;80:173–86.
Lichtenthaler PW, Fischbach A. The conceptualization and measurement of job crafting. Zeitschrift für Arbeits- und Organ A&O. 2016;60:173–86.
Cook JA, Julious SA, Sones W, Hampson LV, Hewitt C, Berlin JA, et al. DELTA2 guidance on choosing the target difference and undertaking and reporting the sample size calculation for a randomised controlled trial. Trials. 2018;19:606.
Committee for Proprietary Medicinal Products (CPMP). Points to consider on switching between superiority and non-inferiority. Br J Clin Pharmacol. 2001;52:223–8.
Cuijpers P, Donker T, van Straten A, Li J, Andersson G. Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med. 2010;40:1943–57.
Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn. Behav Ther. 2018;47:1–18.
Wagner B, Horn AB, Maercker A. Internet-based versus face-to-face cognitive-behavioral intervention for depression: a randomized controlled non-inferiority trial. J Affect Disord. 2014;152–154:113–21.
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Admin Pol Ment Health. 2011;38:65–76.
Glasgow RE. eHealth evaluation and dissemination research. Am J Prev Med. 2007;32:S119–26.
Apolinário-Hagen J, Harrer M, Kählke F, Fritsche L, Salewski C, Ebert DD. Public attitudes toward guided internet-based therapies: web-based survey study. JMIR Ment Heal. 2018;5:e10735.
Lehr D, Geraedts A, Persson Asplund R, Khadjesari Z, Heber E, de Bloom J, et al. Occupational e-mental health: current approaches and promising perspectives for promoting mental health in workers. Heal Work. Cham: Springer International Publishing; 2016. p. 257–81.
Cousins R, MacKay CJ, Clarke SD, Kelly C, Kelly PJ, McCaig RH. ‘Management standards’ work-related stress in the UK: practical development. Work Stress. 2004;18:113–36.
Open Access funding enabled and organized by Projekt DEAL. This study is funded by the German Federal Ministry of Education and Research (BMBF) within the Framework Concept „Future of work “(fund number 02L16D020 to 02L16D023) and managed by the Project Management Agency Forschungszentrum Karlsruhe, Production and Manufacturing Technologies Division (PTKA). The funder is not involved in the design of the study, collection, analysis, interpretation of data, or in writing the manuscript.
Ethics approval and consent to participate
The University of Lueneburg (Germany) ethics committee have already approved subject recruitment and all other procedures involved in the proposed study prior to any subject recruitment (reference: EB-Antrag_Boß_2019-08_PragmatiKK). All participants need to give a written informed consent to participate in the study.
Consent for publication
DE, EH & DL are shareholders of GET.ON Institut für Online Gesundheitstrainings GmbH, which offers Internet-based interventions similar to the iSMT under investigation.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Boß, L., Angerer, P., Dragano, N. et al. Comparative effectiveness of guided internet-based stress management training versus established in-person group training in employees – study protocol for a pragmatic, randomized, non-inferiority trial. BMC Public Health 21, 2177 (2021). https://doi.org/10.1186/s12889-021-12229-y
- Web-based intervention
- Stress management
- Occupational health
- E-mental health
- Randomized controlled trial
- Non-inferiority trial