Efficacy and cost-effectiveness of a web-based and mobile stress-management intervention for employees: design of a randomized controlled trial
BMC Public Health volume 13, Article number: 655 (2013)
Work-related stress is associated with a variety of mental and emotional problems and can lead to substantial economic costs due to lost productivity, absenteeism or the inability to work. There is a considerable amount of evidence on the effectiveness of traditional face-to-face stress-management interventions for employees; however, they are often costly, time-consuming, and characterized by a high access threshold. Web-based interventions may overcome some of these problems yet the evidence in this field is scarce. This paper describes the protocol for a study that will examine the efficacy and cost-effectiveness of a web-based guided stress-management training which is based on problem solving and emotion regulation and aimed at reducing stress in adult employees.
The study will target stressed employees aged 18 and older. A randomized controlled trial (RCT) design will be applied. Based on a power calculation of d=.35 (1-β of 80%, α = .05), 264 participants will be recruited and randomly assigned to either the intervention group or a six-month waitlist control group. Inclusion criteria include an elevated stress level (Cohen’s Perceived Stress Scale-10 ≥ 22) and current employment. Exclusion criteria include risk of suicide or previously diagnosed psychosis or dissociative symptoms. The primary outcome will be perceived stress, and secondary outcomes include depression and anxiety. Data will be collected at baseline and seven weeks and six months after randomization. An extended follow up at 12 months is planned for the intervention group. Moreover, a cost-effectiveness analysis will be conducted from a societal perspective and will include both direct and indirect health care costs. Data will be analyzed on an intention-to-treat basis and per protocol.
The substantial negative consequences of work-related stress emphasize the necessity for effective stress-management trainings. If the proposed internet intervention proves to be (cost-) effective, a preventative, economical stress-management tool will be conceivable. The strengths and limitations of the present study are discussed.
German Register of Clinical Studies (DRKS): DRKS00004749
Stress at work is known to almost every employee. Shorter periods of work-related stress elicit acute stress reactions but are not hazardous to health and can even enhance the workers’ ability to meet challenges at work. However, if stress passes into a chronic state of tension, serious negative health consequences can result [1–3].
The prevalence rate of stress in employees varies considerably depending on several factors, such as the definition of stress, the measurement that is used or the occupation of the workers. A European survey of 21,703 workers showed that 28% of workers experience work-related stress, and the percentages range from 17% in elementary occupations to 40% in professionals .
A meta-analysis found that job strain and effort-reward imbalances produce consistent increases in the risks for common mental disorders . Work stress may precipitate diagnosable depression and anxiety in workers who were previously healthy . Furthermore, sleep disturbances are a common negative consequence of stress . Moreover, work-related stress can have adverse effects on a societal level due to sickness leave  and work disability . According to the American Institute of Stress, the cost estimates of job stress amount to “over $300 billion annually due to increased absenteeism, employee turnover, diminished productivity, medical, legal, and insurance expenses, and workers' compensation payments.” 
A variety of effective traditional occupational stress-management trainings are available [10, 11]. An extensive meta-analysis  showed that the effectiveness of these trainings varies depending on the type of intervention and outcome measure applied. The efficacy of these interventions ranges from d=.11 (the effect of relaxation on depression) to d=.70 (the effect of cognitive behavioral therapy on anxiety) and the overall effect size across all studies is d=.34. However, occupational stress-management interventions are often taught to only a small group of people who are guided by an instructor and these interventions take place at a specific location .
Compared to face-to-face interventions, web-based guided self-help interventions offer the advantages of being independent of time, location and group. Furthermore, web-based guided self-help interventions have shown moderate to large effect sizes in numerous studies of, for example, depression and anxiety . A number of randomized controlled trials on web-based stress-management interventions have been performed, including studies on general populations of people with stress-related complaints [13–15] or work-related stress , college students , women , families [19, 20], and individuals with burnout symptoms . Thereby, the effect sizes for studies that use stress as an outcome measure vary from non-significant  to moderate (d = .60) . However, only few studies have investigated the potential of web-based stress-management interventions in employees [22–27]. These studies of employees apply a wide range of outcome measures, including stress, depression, anxiety, and physiological measurements. With regard to the respective outcome measure, the results again vary between non-significant for stress [23, 26] to moderate for depression .
To date, the cost-effectiveness of worksite mental health interventions has been studied only in a limited number of trials . A recent systematic review  found that these interventions might be cost-effective; however, more high-quality economic evaluations are needed to draw firm conclusions. Data on cost-effectiveness has not been presented in any of the afore-mentioned studies on web-based stress-management for employees. For web-based interventions, studies currently investigate the cost-effectiveness of mental health interventions for workers, including studies of a workers’ health surveillance mental module  and a guided self-help course for workers with depressive symptoms . However, to the best of our knowledge, there are no studies with full economic evaluations of the cost-effectiveness of web-based stress-management training for employees.
The theoretical basis of web-based interventions for stress is diverse and ranges from cognitive behavioral methods [16, 22, 24, 27], mindfulness [15, 31], problem solving , social cognitive theory [23, 32] to health behavior change theory . Surprisingly however, there are currently no web-based interventions with content based on specific stress models, such as the job-demand control model , the effort-reward imbalance model  or more generic models of stress, such as the transactional model of stress . Lazarus’ transactional model of stress  can be applied to working contexts and other life areas. This model identifies two strategies of coping with stressors: problem- and emotion-oriented coping. Some web-based stress-management interventions have already included problem solving as part of their intervention content [14, 24, 32, 36] or as stand-alone interventions . However, web-based stress-management interventions employing a combination of problem- and emotion-oriented coping methods currently do not exist.
At work, people often must not only cope with difficult situations and solve problems but also face challenging emotions that arise from these situations. Deficits in emotion regulation skills are an important factor in the development and maintenance of a broad range of mental disorders, such as depression [37–39] and anxiety . As deficits in the ability to adaptively cope with difficult emotions are related to various mental health problems , the usage of emotion regulation techniques in a low-threshold web-based stress-management training to reduce symptoms of distress appears promising.
The available web-based stress-management interventions cover a wide range of primary treatment components, including several cognitive behavioral methods [16, 21, 22, 24]. In addition, some interventions focus not only on stress but also on health behavior problems such as alcohol abuse [22, 23] and unhealthy diets . A meta-analysis  found that occupational stress-management interventions that focus on fewer treatment components are more effective than interventions that include several components. The authors of this meta-analysis found combined effect sizes of d=.64 and d=.61 for the effectiveness of interventions with one or two treatment components, respectively; and treatments with four or more components yielded a smaller effect size of d=.27 . Thus, it appears promising to opt for fewer intervention components in stress-management interventions.
Aims of the study and hypotheses
In this study, we will investigate the efficacy and cost-effectiveness of a newly developed web-based stress-management training for stressed employees that is based on problem solving and emotion regulation. As a theoretical basis we have followed the transactional stress model of Richard Lazarus . We hypothesize that participants in the intervention group will show a greater reduction in the primary outcome perceived stress from pre-test to post-test compared to a waitlist control group, and that this effect will be maintained over six months. We will conduct cost-effectiveness and cost-utility analyses from a societal perspective and hypothesize that the intervention will be cost-effective. Furthermore, explorative moderator analyses will be conducted.
A randomized controlled trial with two conditions will be conducted. The intervention group will receive the web-based stress-management training “GET.ON Stress” and the control group will obtain access to this intervention after 6 months (see Figure 1). There will be no restriction with regard to the use of medication or other treatment as usual (TAU) (e.g., psychotherapy) in either of the groups. To control for potential confounding effects, TAU will be monitored.
Inclusion and exclusion criteria
The target group will consist of stressed employees from the general working population aged 18 years or older. Inclusion criteria will include current employment and a distinct level of perceived stress as indicated by scores of 22 and above on the Perceived Stress Scale (PSS-10). This cut-off was chosen to select participants with a marked level of subjective stress as identified by one standard deviation (SD = 6.2) above the mean (PSS-10 = 15.3) in a large working population . Applicants with an elevated risk of suicide (Beck Suicide Item > 1) and applicants that self-report having been diagnosed with psychosis or dissociative symptoms in the past will be excluded. For applicants that are at risk for suicide we follow a standardized procedure. These applicants will be advised to seek help via their general practitioner, the local psychiatric emergency room or an official emergency number, and the relevant information and telephone numbers will be provided by email.
Participants will be recruited from the general working population via mass media (e.g., newspaper articles, television) between March and October 2013. Moreover, a major health insurance company (BARMER GEK) will support the recruitment process within their occupational health management program. On an open access website (http://www.geton-training.de) the interested people will be able to sign up to participate with an email-address. Applicants will receive further informational material concerning the study conditions and the training and will be asked to provide an email-address and a first and last name (which can be pseudonyms if desired) to participate. Afterwards, a link to the screening questionnaire will be sent. Applicants who are eligible for participation will be required to provide their informed consent and fill out the online baseline questionnaire. Subsequently, these participants will be randomly allocated to either the intervention or the control group. Participants will be informed of the outcome of the randomization and participants in the intervention group will receive immediate access to the GET.ON Stress training. Assessments will be scheduled at seven weeks and six months after randomization for both groups. However, the participants in the control group will receive the login data required to complete the training six months later than the intervention group. An extended follow-up for the intervention group is scheduled 12 months post-randomization.
The randomization will be carried out by an independent researcher using a web-based randomization program (randomisation.eu) that will be set to allocate participants (N=264) into each group at a ratio of 1:1.
We aim to include 264 participants to produce a statistically relevant effect size of d = .35, a power (1-β) of 80% and an alpha of .05 (two-tailed test) for intention-to-treat analysis using PASS 12. A comprehensive meta-analysis on traditional face-to-face interventions for work-related stress yielded an overall combined effect size of d=.34, whereby the results for web-based stress-management interventions are mixed, ranging from non-significant  to moderate effect sizes . Therefore, we aim for an effect size of d=.35.
The web-based GET.ON Stress intervention is based on two main components: problem solving and emotion regulation. The intervention consists of seven sessions composed of modules for psycho-education (session 1), problem solving (sessions 2–3), emotion regulation (sessions 4–6) and plan for the future (session 7) (see Table 1). Each session can be completed in approximately 45–60 minutes and consists of general information, examples related to work, interactive exercises, quizzes, audio and video files, and downloadable work sheets and mp3 files. We will recommend to log in once or twice a week and to fill out a short daily stress diary. Depending on the module, further homework will be assigned. Using responsive web-design, participants can follow the program on the internet, a tablet or mobile phone. An integrated read-aloud function allows participants to follow the lessons in an audio-narrated way.
Psycho-education (session 1)
In the first session, the participants will be provided with psycho-educational information about stress based on Lazarus’ transactional model of stress , that includes emotion-focused and problem-focused coping strategies. Following a video introduction to the basic information about stress and coping methods, an interactive quiz will be presented to equip participants with general knowledge of the most appropriate coping strategies for common problem situations. The participants will identify their personal stressors and define their goals and motivation during the training. Furthermore, participants will be asked to choose one positive activity each day.
Problem solving (session 2–3)
In sessions 2 and 3, the participants will work on their problem-solving skills. This module is based on problem solving therapy . The participants will learn a systematic six-step problem-solving method that can be applied to their individual problems. This method has already been adopted in other web-based studies [13, 44]. Typical scenarios involving work-related stress will be presented. The participants will fill out their own six-step procedures and use their personal solution in the time between the sessions. In session 3, the participants will have the opportunity to either work on the same problem as in the previous session or choose a new problem.
Emotion regulation (session 4–6)
In sessions 4 to 6, the participants will work on the emotion regulation module. The emotion regulation techniques are based on the Affect Regulation Training (ART) [37, 45] and include muscle and breathing relaxation, acceptance and tolerance of emotions and effective self-support. ART has been shown to be effective as additional treatment component in cognitive-behavioral treatment for depression  and also as stand-alone intervention in at risk-populations (i.e., police officers) . In each session, one strategy will be taught, and the participants will practice that strategy with a 15-minute audio file according to the instruction of an expert. The techniques will be introduced using examples of typical emotional reactions related to work contexts. Video files explaining the psychological background will be displayed. The participants will be asked to listen to the appropriate 15-minute audio file on a daily basis and will be provided with downloadable mp3 files.
Plan for the future (session 7)
In the last session, the participants will be asked to reassess their goals for the training. Furthermore, they will identify their personal warning signs for stress. Additionally, they will be asked to strengthen something important in their lives and write a letter to themselves about how they imagine their life will be after four weeks of applying the stress-management methods they have been trained in.
Optional booster session
Four weeks after completing the training, the participants will be given the option of completing a booster session and evaluating their training progress. They will be given the opportunities to evaluate the letter they wrote to themselves in the last training session, reassess their goals and make plans to continue to include positive activities in their daily lives.
At the end of sessions 2 to 6, information and exercise sections about common stress-related topics will be provided to enhance the recognition value of the stress-management training. These sections are optional and will not be part of the main intervention. These sections will cover the following topics: time management, rumination and worrying, psychological detachment from work, sleep hygiene, rhythm and regularity of sleeping habits, nutrition and exercise, organization of breaks during work, and social support.
Text message coach
If desired, participants will receive automatic motivational text messages and small exercises on their mobile phones. These messages will support the participant in transferring the exercises of the training into their daily lives (e.g., short relaxation exercises: “Relax your muscles in your hands and arms for 3 seconds now. Follow your breathing and each time you breathe out, relax a little more“). The participants will have the opportunity to choose between “light coach” (one text message every other day) and “intensive coach” (2–3 text messages every day) options.
Within 48 hours, the participants will receive personalized written feedback on the exercises they have completed in each session from an eCoach. The eCoaches are psychologists and trained master’s-level psychology students and will follow guidelines about the feedback process that are defined according to the standardized manual for the intervention. The eCoaches will send reminders in cases in which the participants do not complete one session within 7 days. The training itself, and the communication between the participant and the eCoach will take place in a secured web-based platform (AES 256-bit encryption). The participants will have access to the platform based on their email-addresses and self-designated passwords.
Primary and secondary outcomes
Primary outcome will be perceived stress. In secondary analyses, we will explore the effects of depression, anxiety, emotional exhaustion, emotion regulation, attitudes toward seeking professional psychological help, work engagement, psychological detachment, insomnia severity, worrying, and absenteeism/presenteeism. Economic analyses will be conducted assessing cost-effectiveness and cost-utility from a societal perspective including the costs of all types of health services and the costs that stem from productivity losses.
The German version of the ten-item-Perceived Stress Scale (PSS-10) [47, 48] will be used as a primary outcome measure. The PSS is “the most widely used instrument for measuring perceived stress”  and assesses the degree to which people perceive their lives as stressful, particularly regarding how “unpredictable, uncontrollable and overloading respondents find their lives” . Cronbach’s alphas range for this scale from .78 to .91 . The scale is based on Lazarus’ transactional model of stress and, therefore, fits well with the theoretical basis of the intervention. Participants in this study will be asked to answer questions relating to the past week as opposed to the past month to avoid confounding with the training period. Similar procedures have been adopted in previous studies .
Depressive symptoms will be measured with the German version of the Center for Epidemiological Studies’ Depression Scale (CES-D) [50, 51]. This frequently used self-report instrument consists of 20 items that are answered on a four-point Likert scale referring to the previous week. Total scores range from 0 to 60. The internal consistency of this measure has been found to be α = .89 .
Anxiety symptoms will be measured with the German version of the anxiety subscale of the Hospital Anxiety and Depression Scales (HADS) [52, 53]. This subscale contains seven items related to the previous week. Each item is scored from 0 to 3; thus, the scores range from 0 to 21. The HADS has been shown to have a high internal consistency of α = .80 .
The German version of the Maslach Burnout Inventory (MBI-GS-D) [54, 55] will be used to measure emotional exhaustion, the basic stress dimension of burnout. This commonly used self-report instrument consists of five items and uses a six-point Likert-type scale anchored by 1 = “never” and 6 = “very often”. The internal consistency of this subscale was α = .85 in a German sample .
To assess emotion regulation skills we will use the German Emotion Regulation Skills Questionnaire (ERSQ-27 and ERSQ-ES) [57, 58]. The subscales of acceptance, emotional self-support and comprehension will be included to assess the usage of general emotion regulation skills during the previous week and measured on a five-point Likert-type scale (ranging from “not at all” to “almost always”). Each scale is comprised of three items. The overall Cronbach’s α of the ERSQ-27 was .90 in a community-based sample, and for the subscales of acceptance, emotional self-support and comprehension .68, .72 and .73, respectively . Furthermore, the subscale “general distress” from the emotion-specific version of the ERSQ (ERSQ-ES)  will be used to assess the regulation of stress. Therefore, the general distress subscale containing 12 items will be applied to assess constructive coping with stress and tension within the last week on a five-point Likert-type scale (ranging from “not at all” to “always”). This subscale has a high internal consistency of α = .85 .
Attitudes toward seeking professional psychological help
We will measure the influence of attitudes toward mental health care service utilization with the ten-item Attitudes Toward Seeking Professional Psychological Help Scale – Short Version (ATSPPHS-SF) . The items of this scale are answered on a four-point Likert scale and result in total scores of 0–30; higher scores indicate more positive attitudes. In a normative sample, this scale showed an adequate internal consistency (α = .84).
The Utrecht Work Engagement Scale (UWES)  assesses work engagement defined as a “positive, fulfilling, work-related state of mind that is characterized by vigor, dedication, and absorption” . This scale has nine items, and the internal consistency of the total score is α = .91.
Psychological detachment from work will be measured with a subscale of the Recovery Experience Questionnaire (REQ) . This subscale contains four items that are rated on a five-point Likert scale. The internal consistency of this subcscale is α = .85 .
Insomnia severity will be measured with the Insomnia Severity Index (ISI) [63, 64]. This seven-item scale measures the nature, severity and impact of insomnia and is rated on a five-point Likert scale. The total score indicates overall insomnia severity and ranges from 0 to 28. This scale has been validated as a web-based measure . Internal consistency has been found to be Chronbach’s α =.90 and .91 .
We will use the ultra-brief three-item version of the Penn State Worry Questionnaire (PSWQ) [67, 68] to assess worrying. Each item is answered on a seven-point scale. We have adapted the questionnaire to encompass the previous week based on the three items of the Penn State Worry Questionnaire-Past Week (PSWQ-PW) [69, 70]. Total scores range from 0-18. The psychometric properties of the English ultra-brief version are similar to the standard English 16-item version (Cronbach’s α = .85 compared to α = .91 for the standard version)  and the German version of the PSWQ-PW.
Quality of life
We will use the Short Form 12 (SF-12)  and the EuroQol (EQ-5D)  to assess quality of life. The SF-12  covers eight health domains (physical functioning, role limitations, pain, general health perception, vitality, mental health, emotional role and social functioning) and allows for the calculation of two sum scores for physical and mental health. We will also use the EQ-5D which is a widely applied, valid and reliable measurement of quality of life and consists of five items related to mobility, self-care, common activities, pain/discomfort and anxiety/depression. Furthermore, this measurement contains a visual analogue scale concerning health state.
The economic evaluation will be conducted from a societal perspective; thus, we will include direct medical (e.g., medicine), direct non-medical (e.g., parking) and indirect costs (e.g., productivity loss)  over the previous three months. We have adapted the Trimbos and Institute of Medical Technology Assessment Cost Questionnaire for Psychiatry (TiC-P)  for application to the German health care system.
Other measurements include demographic variables (e.g., age, gender, occupation etc.), the Effort Reward Imbalance Questionnaire – Short Form (ERI-SF) , the hope of improvement subscale of the German Patient Questionnaire on Therapy Expectation and Evaluation (PATHEV)  (adapted to the online training context), the German version of the Client Satisfactory Questionnaire (CSQ-8) [77, 78] (adapted to the online training context), and a German questionnaire on the negative effects of psychotherapy (Ladwig, I., Rief, W., & Nestoriuc, Y.: Hat Psychotherapie auch Nebenwirkungen? Entwicklung des Inventars zur Erfassung Negativer Effekte von Psychotherapie (INEP) [Does psychotherapy have side effects? Development of an Inventory of Negative Effects of Psychotherapy (INEP)], submitted); adapted to the online training context). Approximately 30 minutes will be required to complete all questionnaires. For an overview of all outcome measures, see Table 2.
Data will be analyzed on an intention-to-treat basis. Additionally, completers-only and per-protocol analyses will be carried out. Missing data will be dealt with following the recommendations of Little and Rubin  and Schafer . We will use repeated measurements analysis of variance to examine differences in the primary and secondary outcome measures between the two groups, and we will compute standardized effect sizes (Cohen’s d). The number needed to treat (NNT) and the clinical significance in terms of a reliable change and recovery rates will be investigated using the method of Jacobson and Truax .
Moderator analyses will be conducted including potential moderators as interaction with treatment condition as independent variable in the main effect analyses. Potential moderators to be examined include severity of baseline stress symptoms, socio-demographic variables (e.g. age, gender, education etc.), sleep disturbances, emotion regulation skills, worrying and psychological detachment.
We will perform an economic evaluation from a societal perspective that includes all relevant costs and outcomes. A cost-effectiveness analysis and a cost-utility analysis will be conducted. For cost-utility analyses, quality-adjusted life years (QUALYs) will be calculated. A non-parametric boot-strapping method with 95% confidence intervals in percentiles will be used to assess differences between the intervention and control group. We will compare the intervention and control groups in terms of incremental costs and incremental effects. Towards this end, we will calculate the incremental cost-effectiveness ratio (ICER). We will use bootstrapping (5,000 times) to test the robustness of the ICER and to quantify the uncertainty in the ratios. The results will be shown in a cost-effectiveness plane and in a cost-effective acceptability curve. Additionally, the robustness of the base-case findings will be tested with a multi-way sensitivity analysis.
In this study, we will examine the efficacy and cost-effectiveness of a web-based stress-management training in highly stressed employees. We expect that the stress levels of participants in the intervention group will be significantly lower after the training and after six months compared to these measures from the control group. Furthermore, we expect this intervention to be cost-effective.
The intervention content of the stress training was developed with a clear theoretical background based on Lazarus’ transactional model of stress . A recent study of a web-based intervention aimed at reducing stress provides psycho-educational information about problem- and emotion-focused coping in one out of 13 training sessions . However, there are currently no web-based stress-management interventions that have used the combination of problem- and emotion-focused coping based on the definitions of Lazarus  as the main theoretical basis. In the research on stress, problem-oriented coping methods such as the problem-solving training have always been strongly emphasized. In addition, many interventions include cognitive restructuring as a treatment component. With this focus, emotions are only addressed indirectly as a sub-area of cognitions. Recently, deficits in emotion regulation skills as a treatment target have gained attention in the field of mental health [37, 41]. Emotion regulation has been shown to be relevant in a broad range of mental disorders  including depression [37–39] and anxiety . Only now, the targeting of emotions as an autonomous treatment component finds its way into stress-management training formats. A study in hypertensive employees found that a stress-management intervention consisting of positive emotion refocusing and emotional restructuring techniques can improve stress, depression, emotional health and systolic blood pressure . Furthermore, an intervention designed to increase emotional competence has been found to result in decreased perceived stress and lower cortisol secretion in adulthood . However, to the best of our knowledge, there is no web-based stress-management training available that has included emotion regulation as main treatment component. Thus, the web-based training proposed herein includes emotion regulation as an innovative and promising component in addition to the traditional, established component of problem solving.
The inclusion of an economic evaluation constitutes a strong advantage of this study. There are a limited number of economic evaluations of worksite mental health interventions available . The afore-mentioned RCTs on web-based interventions for managing stress [13–16, 18, 21–24] have not included full economic evaluations.
A number of studies of web-based stress-management interventions have been undertaken; however, little is known about moderators of treatment outcome. It is essential to know for which people with which personal characteristics, which training is most effective under what circumstances in order to optimize training circumstances for each individual based on his or her personal characteristics. Therefore, we include a variety of measurements that are assumed to be related to stress, such as emotion regulation skills [86, 87], sleep disturbances , psychological detachment or worrying [62, 88, 89].
A major strength of the current study is the robust randomized controlled trial design, which is the gold standard for clinical trials. Furthermore, this is the first study in Germany to examine the efficacy and cost-effectiveness of a web-based stress-management intervention. We also include a large number of participants in this study to ensure sufficient power.
Limitations and future directions
One limitation of this study is that no objective measurement of stress (e.g., cortisol levels) will be included. Due to feasibility limitations, only self-report measurements will be examined. Although self-reports always carry the risk of introducing subjective biases, it has been suggested that replacing self-reports with stress-related physiological measurements is not promising . However, the completion of self-reports can be complemented with other measurements to provide more reliable data .
Another limitation of the proposed study is the potential self-selection bias. Participants will be eligible to apply for the intervention if they experience work-related stress. One must assume that the individuals who participate in this study possess certain characteristics. Therefore, the results of the current trial will only be applicable to persons who select themselves for this stress-management training.
For future studies it is worthwhile to investigate how much support is needed within the training to achieve the optimal trade-off between treatment outcome and economical costs, i.e., to maximize treatment outcome and simultaneously maintain the lowest level of support possible. A meta-analysis showed that interventions without an eCoach can be effective; however, the effect sizes without an eCoach are much smaller than those of interventions with an eCoach . The current training includes a personal feedback from an eCoach after each of the seven sessions. It would be interesting to examine the size of the treatment effects that result from the application of other, less intensive and more economical support concepts, such as feedback on demand.
In the current trial, we will examine the efficacy of a guided self-help training in comparison to a waitlist control group. This design is therefore not suited to draw any conclusions regarding the effectiveness of the training compared to a face-to-face training, as has been examined elsewhere . In a systematic review and meta-analysis, guided self-help programs for depression and anxiety have been found to have effects similar to those of face-to-face therapies ; thus, an investigation of the efficacy of a face-to-face version of the training compared to the web-based training would be worthwhile.
This study aims to evaluate the efficacy and cost-effectiveness of a newly developed stress-management training for employees. If successful, this training could be made available to a large number of employees because of the low threshold accessibility and potentially low costs.
This study has been approved by the ethics committee of the Philipps University Marburg (registration number AZ-2012-43K).
Randomized controlled trial
Perceived stress scale
Deutsches Register für Klinische Studien
Treatment as usual
Affect regulation training
Center for Epidemiological Studies Depression Scale
Hospital Anxiety and Depression Scales
Maslach Burnout Inventory
Emotion regulation skills questionnaire
Emotion regulation skills questionnaire - emotion specific version
Attitudes toward seeking professional psychological help scale – short version
Utrecht work engagement scale
Recovery experience questionnaire
Insomnia severity index
Penn state worry questionnaire-past week
Quality-adjusted life years
Short form 12
Trimbos and institute of medical technology assessment cost questionnaire for psychiatry
Effort reward imbalance questionnaire – short form
Patient questionnaire on therapy expectation and evaluation
Client satisfactory questionnaire
the Consolidated standards of reporting trials
Number needed to treat
Incremental cost-effectiveness ratio
Backé E, Seidler A, Latza U, Rossnagel K, Schumann B: The role of psychosocial stress at work for the development of cardiovascular diseases: a systematic review. Int Arch Occup Environ Health. 2012, 85: 67-79. 10.1007/s00420-011-0643-6.
Kivimäki M, Virtanen M, Elovainio M, Kouvonen A, Väänänen A, Vahtera J: Work stress in the etiology of coronary heart disease–a meta-analysis. Scand J Work Environ Health. 2006, 32: 431-442. 10.5271/sjweh.1049.
Stansfeld S, Candy B: Psychosocial work environment and mental health–a meta-analytic review. Scand J Work Environ Health. 2006, 32: 443-462. 10.5271/sjweh.1050.
Paoli P, Merllié D: Third European survey on working conditions 2000. 2001, Luxembourg: Office for Official Publications of the European Communities
Melchior M, Caspi A, Milne BJ, Danese A, Poulton R, Moffitt TE: Work stress precipitates depression and anxiety in young, working women and men. PsycholMed. 2007, 37: 1119-1129.
Akerstedt T: Psychosocial stress and impaired sleep. Scand J Work Environ Health. 2006, 32: 493-501. 10.5271/sjweh.1054.
Moreau M: Occupational stress and incidence of sick leave in the Belgian workforce: the Belstress study. J Epidemiol Community Health. 2004, 58: 507-516. 10.1136/jech.2003.007518.
Ahola K, Toppinen-Tanner S, Huuhtanen P, Koskinen A, Väänänen A: Occupational burnout and chronic work disability: An eight-year cohort study on pensioning among Finnish forest industry workers. J Affect Disord. 2009, 115: 150-159. 10.1016/j.jad.2008.09.021.
Rosch PJ: The quandary of job stress compensation. Health and Stress. 2001, 3: 1-4.
van der Klink JJL, Blonk RWB, Schene AH, van Dijk FJH: The benefits of interventions for work-related stress. Am J Public Health. 2001, 91: 270-276.
Richardson KM, Rothstein HR: Effects of occupational stress management intervention programs: A meta-analysis. J Occup Health Psychol. 2008, 13: 69-93.
Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V: Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: A meta-analysis. Psychol Med. 2007, 37: 319-10.1017/S0033291706008944.
van Straten A, Cuijpers P, Smits N: Effectiveness of a web-based self-help intervention for symptoms of depression, anxiety, and stress: Randomized controlled trial. J Med Internet Res. 2008, 10: e7-10.2196/jmir.954.
Zetterqvist K, Maanmies J, Ström L, Andersson G: Randomized controlled trial of internet-based stress management. Cognit Behav Ther. 2003, 32: 151-160. 10.1080/16506070302316.
Drozd F, Raeder S, Kraft P, Bjørkli CA: Multilevel growth curve analyses of treatment effects of a web-based intervention for stress reduction: Randomized controlled trial. J Med Internet Res. 2013, 15: e84-10.2196/jmir.2570.
Ruwaard J, Lange A, Bouwman M, Broeksteeg J, Schrieken B: E‒Mailed standardized cognitive behavioural treatment of work‒related stress: A randomized controlled trial. Cognit Behav Ther. 2007, 36: 179-192. 10.1080/16506070701381863.
Chiauzzi E, Brevard J, Thurn C, Decembrele S, Lord S: MyStudentBody–Stress: An online stress management intervention for college students. J Health Commun. 2008, 13: 555-572. 10.1080/10810730802281668.
Wiegand B, Luedtke K, Friscia D, Nair M, Aleles M, McCloskey R: Efficacy of a comprehensive program for reducing stress in women: A prospective, randomized trial. Curr Med Res Opin. 2010, 26: 991-1002. 10.1185/03007991003688193.
Wade SL, Carey J, Wolfe CR: An online family intervention to reduce parental distress following pediatric brain injury. J Consult Clin Psychol. 2006, 74: 445-454.
Hänggi Y: Kann web-basierte Stressprävention erfolgreich sein? [Can web-based parental training on coping with familiy stress be successful?]. Z Klin Psychol Psychother. 2006, 35: 169-177. 10.1026/1616-3422.214.171.124.
Lange A, van de Ven J, Schrieken B, Smit M: 'Interapy' Burn-out: Prävention und Behandlung von Burn-out über das Internet [‘Interapy’ burnout: prevention and therapy of burnout via the internet]. Verhaltenstherapie. 2004, 14: 190-199. 10.1159/000080915.
Billings DW, Cook RF, Hendrickson A, Dove DC: A web-based approach to managing stress and mood disorders in the workforce. J Occup Environ Med. 2008, 50: 960-968. 10.1097/JOM.0b013e31816c435b.
Cook RF, Billings DW, Hersch RK, Back AS, Hendrickson A: A field test of a web-based workplace health promotion program to improve dietary practices, reduce stress, and increase physical activity: Randomized controlled trial. J Med Internet Res. 2007, 9: e17-10.2196/jmir.9.2.e17.
Grime PR: Computerized cognitive behavioural therapy at work: A randomized controlled trial in employees with recent stress-related absenteeism. Occup Med. 2004, 54: 353-359. 10.1093/occmed/kqh077.
Hasson D, Anderberg U, Theorell T, Arnetz BB: Psychophysiological effects of a web-based stress management system: A prospective, randomized controlled intervention study of IT and media workers. BMC Public Health. 2005, 5: 78-10.1186/1471-2458-5-78.
Abbott J, Klein B, Hamilton C, Rosenthal AJ: The impact of online resilience training for sales managers on wellbeing and performance. E-JAP. 2009, 5: 89-95.
Eisen KP, Allen GJ, Bollash M, Pescatello LS: Stress management in the workplace: A comparison of a computer-based and an in-person stress-management intervention. Comput Hum Behav. 2008, 24: 486-496. 10.1016/j.chb.2007.02.003.
Hamberg-van Reenen HH, Proper KI, van den Berg M: Worksite mental health interventions: a systematic review of economic evaluations. Occup Environ Med. 2012, 69: 837-845. 10.1136/oemed-2012-100668.
Gärtner FR, Ketelaar SM, Smeets O, Bolier L, Fischer E, van Dijk FJH, Nieuwenhuijsen K, Sluiter JK: The Mental Vitality @ Work study: Design of a randomized controlled trial on the effect of a workers' health surveillance mental module for nurses and allied health professionals. BMC Public Health. 2011, 11: 290-10.1186/1471-2458-11-290.
Geraedts AS, Kleiboer AM, Wiezer NM, van Mechelen W, Cuijpers P: Web-based guided self-help for employees with depressive symptoms (Happy@Work): Design of a randomized controlled trial. BMC Psychiatry. 2013, 13: 61-10.1186/1471-244X-13-61.
Krusche A, Cyhlarova E, King S, Williams JMG: Mindfulness online: A preliminary evaluation of the feasibility of a web-based mindfulness course and the impact on stress. BMJ Open. 2012, 2: e000803-10.1136/bmjopen-2011-000803.
Shimazu A, Kawakami N, Irimajiri H, Sakamoto M, Amano S: Effects of web-based psychoeducation on self-efficacy, problem solving behavior, stress responses and job satisfaction among workers: A controlled clinical trial. J Occup Health Psychol. 2005, 47: 405-413. 10.1539/joh.47.405.
Karasek RA: Job demands, job decision latitude, and mental strain: Implications for job redesign. Admin Sci Quart. 1979, 24: 285-308. 10.2307/2392498.
Siegrist J: Reciprocity in basic social exchange and health: Can we reconcile person-based with population-based psychosomatic research?. J Psychosom Res. 1998, 45: 99-105.
Lazarus RS, Folkman S: Stress, appraisal, and coping. 1984, New York: Springer
van Vliet H, Andrews G: Internet-based course for the management of stress for junior high schools. Aust NZ J Psychiatry. 2009, 43: 305-309. 10.1080/00048670902721145.
Berking M, Ebert D, Cuijpers P, Hofmann SG: Emotion regulation skills training enhances the efficacy of inpatient cognitive behavioral therapy for major depressive disorder: A randomized controlled trial. Psychother Psychosom. 2013, 82: 234-245. 10.1159/000348448.
Ehring T, Tuschen-Caffier B, Schnülle J, Fischer S, Gross JJ: Emotion regulation and vulnerability to depression: Spontaneous versus instructed use of emotion suppression and reappraisal. Emotion. 2010, 10: 563-572.
Liverant GI, Brown TA, Barlow DH, Roemer L: Emotion regulation in unipolar depression: The effects of acceptance and suppression of subjective emotional experience on the intensity and duration of sadness and negative affect. Behav Res Ther. 2008, 46: 1201-1209. 10.1016/j.brat.2008.08.001.
Cisler JM, Olatunji BO: Emotion regulation and anxiety disorders. Curr Psychiatry Rep. 2012, 14: 182-187. 10.1007/s11920-012-0262-2.
Berking M, Wupperman P: Emotion regulation and mental health: recent findings, current challenges, and future directions. Curr Opin Psychiatry. 2012, 25: 128-134. 10.1097/YCO.0b013e3283503669.
Lesage F, Berjot S, Deschamps F: Psychometric properties of the French versions of the Perceived Stress Scale. Int J Occup Med Environ Health. 2012, 25: 178-184. 10.2478/S13382-012-0024-8.
D'Zurilla T, Nezu A: Problem-Solving Therapy. In Handbook of cognitive-behavioral therapies. 3rd edition. Edited by Dobson KS. 2010, New York: Guilford Press
Donker T, van Straten A, Riper H, Marks I, Andersson G, Cuijpers P: The design of a randomized controlled trial. Trials. 2009, 10: 59-
Berking M: Training emotionaler Kompetenzen. 2010, Berlin, Heidelberg: Springer-Verlag Berlin Heidelberg, 2
Berking M, Meier C, Wupperman P: Enhancing emotion-regulation skills in police officers: Results of a pilot controlled study. Behav Ther. 2010, 41: 329-339. 10.1016/j.beth.2009.08.001.
Cohen S, Kamarck T, Mermelstein R: A global measure of perceived stress. J Health Soc Behav. 1983, 24: 385-396. 10.2307/2136404.
Michalsen A, Jeitler M, Brunnhuber S, Lüdtke R, Büssing A, Musial F, Dobos G, Kessler C: Iyengar Yoga for distressed women: A 3-armed randomized controlled trial. Evid Based Complement Alternat Med. 2012, 2012: 1-9.
Cohen S, Janicki-Deverts D: Who's stressed? Distributions of psychological stress in the United States in probability samples from 1983, 2006, and 2009. J Appl Soc Psychol. 2012, 42: 1320-1334. 10.1111/j.1559-1816.2012.00900.x.
Radloff LS: The CES-D Scale: A self-report depression scale for research in the general population. Appl Psych Meas. 1977, 1: 385-401. 10.1177/014662167700100306.
Hautzinger M, Bailer M: (Eds): Allgemeine Depressions Skala. 1993, Manual. Göttingen: Beltz Test GmbH,
Zigmond AS, Snaith RP: The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983, 67: 361-370. 10.1111/j.1600-0447.1983.tb09716.x.
Hermann-Lingen C, Buss U, Snaith R: Hospital Anxiety and Depression Scale - Deutsche Version (HADS-D). 2011, Bern: Hans Huber, 3
Büssing A, Glaser J: Managerial stress und burnout. A collaborative international study (CISMS). Die deutsche Untersuchung (Bericht Nr. 44). 1998, München: Technische Universität, Lehrstuhl für Psychologie
Schaufeli W, Leiter M, Maslach C, Jackson SE: Maslach Burnout Inventory - General Survey (MBI-GS). In Maslach Burnout Inventory Manual. Edited by Maslach C, Jackson SE, Leiter MP. 1996, Palo Alto, CA: Consulting Psychologists Press
Neubach B, Schmidt K: Gütekriterien einer deutschen Fassung des Maslach Burnout Inventory (MBI-D) - Eine Replikationsstudie bei Altenpflegekräften [Psychometric properties and validity of a German version of the Maslach Burnout Inventory (MBI—D): A replication study among professionals caring for the elderly]. Z Arb Organ. 2000, 44: 140-156.
Berking M, Znoj H: Entwicklung und Validierung eines Fragebogens zur standardisierten Selbsteinschätzung emotionaler Kompetenzen (SEK-27) [Development and validation of a self-report instrument for the assessment of emotion-specific regulation skills (ERSQ-27)]. Z Psychiatr Psychol Psychother. 2008, 56: 141-153.
Ebert D, Christ O, Berking M: Entwicklung und Validierung eines Fragebogens zur emotionsspezifischen Selbsteinschätzung emotionaler Kompetenzen (SEK-ES) [Development and validation of a self-report instrument for the assessment of emotion-specific regulation skills (ERSQ-ES)]. Diagnostica. 2013, 59: 17-32. 10.1026/0012-1924/a000079.
Fischer EH, Farina A: Attitudes toward seeking professional psychological help: a shortened form and considerations for research. J Coll Student Dev. 1995, 36: 368-373.
Schaufeli W: Bakker A (Eds): UWES - Utrecht Work Engagement Scale: Test Manual. 2003, Utrecht, The Netherlands: Department of Psychology, Utrecht University,
Schaufeli W, Salanova M, González-Romá V, Bakker AB: The measurement of engagement and burnout: A two sample confirmatory factor analytic approach. J Happiness Stud. 2002, 3: 71-92. 10.1023/A:1015630930326.
Sonnentag S, Fritz C: The recovery experience questionnaire: Development and validation of a measure assessing recuperation and unwinding at work. J Occup Health Psychol. 2007, 12: 204-221.
Morin CM: Insomnia: Psychological assessment and management. 1993, New York: Guilford Press
Bastien CH, Vallières A, Morin CM: Validation of the Insomnia Severity Index as an outcome measure for insomnia research. Sleep Med. 2001, 2: 297-307. 10.1016/S1389-9457(00)00065-4.
Thorndike FP, Ritterband LM, Saylor DK, Magee JC, Gonder-Frederick LA, Morin CM: Validation of the insomnia severity index as a web-based measure. Behav Sleep Med. 2011, 9: 216-223. 10.1080/15402002.2011.606766.
Morin CM, Belleville G, Bélanger L, Ivers H: The Insomnia Severity Index: psychometric indicators to detect insomnia cases and evaluate treatment response. Sleep. 2011, 34: 601-608.
Meyer T, Miller M, Metzger R, Borkovec TD: Development and validation of the Penn State Worry Questionnaire. Behav Res Ther. 1990, 28: 487-495. 10.1016/0005-7967(90)90135-6.
Berle D, Starcevic V, Moses K, Hannan A, Milicevic D, Sammut P: Preliminary validation of an ultra-brief version of the Penn State Worry Questionnaire. Clin Psychol Psychother. 2011, 18: 339-346. 10.1002/cpp.724.
Stöber J, Bittencourt J: Weekly assessment of worry: An adaptation of the Penn State Worry Questionnaire for monitoring changes during treatment. Behav Res Ther. 1998, 36: 645-656. 10.1016/S0005-7967(98)00031-X.
Stöber J: PSWQ-PW: Penn State Worry Questionnaire-Past Week. Diagnostische Verfahren in der Psychotherapie [Assessment procedures in psychotherapy. Edited by: Brähler E, Schumacher J, Strauß B. 2002, Göttingen, Germany: Hogrefe, 291-294.
Ware J, Kosinski M, Keller SD: A 12-Item Short-Form Health Survey: Construction of scales and preliminary tests of reliability and validity. Med Care. 1996, 34: 220-233. 10.1097/00005650-199603000-00003.
Group EQ: EuroQol - a new facility for the measurement of health-related quality of life. The EuroQol Group. Health Policy. 1990, 16: 199-208.
Krauth C: Methoden der Kostenbestimmung in der gesundheitsökonomischen Evaluation [Cost estimation methods in health economic evaluation]. Gesundheitsökonomie & Qualitätsmanagement. 2010, 15: 251-259.
Hakkaart-van Roijen L: Manual Trimbos/iMTA Questionnaire for costs associated with psychiatric illness (in Dutch). 2002, Rotterdam: Institute for Medical Technology Assessment
Siegrist J, Wege N, Pühlhofer F, Wahrendorf M: A short generic measure of work stress in the era of globalization: effort–reward imbalance. Int Arch Occup Environ Health. 2009, 82: 1005-1013. 10.1007/s00420-008-0384-3.
Schulte D: Messung der Therapieerwartung und Therapieevaluation von Patienten (PATHEV) [Measurement of patient's therapy expectancy and therapy evaluation]. Z Klin Psychol Psychother. 2005, 34: 176-187. 10.1026/1616-34126.96.36.199.
Schmidt J, Lamprecht F, Wittmann WW: Zufriedenheit mit der stationären Versorgung. Entwicklung eines Fragebogens und erste Validitätsuntersuchungen [Satisfaction with inpatient management. Development of a questionnaire and initial validity studies]. Psychother Psychosom Med Psychol. 1989, 39: 248-255.
Attkisson CC, Zwick R: The client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plann. 1982, 5: 233-237. 10.1016/0149-7189(82)90074-X.
Altman DG: Better reporting of randomised controlled trials: the CONSORT statement. BMJ. 1996, 313: 570-571. 10.1136/bmj.313.7057.570.
Eysenbach G: CONSORT-EHEALTH: Improving and standardizing evaluation reports of web-based and mobile health interventions. J Med Internet Res. 2011, 13: e126-10.2196/jmir.1923.
Little RJA, Rubin DB: Statistical analysis with missing data. Wiley series in probability and statistics. Edited by: Hoboken NJ. 2002, : Wiley, 2
Schafer JL: Analysis of incomplete multivariate data. 2000, Boca Raton: Chapman & Hall/CRC, [Monographs on statistics and applied probability, vol. 72], 1
Jacobson NS, Truax P: Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol. 1991, 59: 12-19.
McCraty R, Atkinson M, Tomasino D: Impact of a workplace stress reduction program on blood pressure and emotional health in hypertensive employees. J Altern Complement Med. 2003, 9: 355-369. 10.1089/107555303765551589.
Kotsou I, Nelis D, Grégoire J, Mikolajczak M: Emotional plasticity: Conditions and effects of improving emotional competence in adulthood. J Appl Psychol. 2011, 96: 827-839.
Ciarrochi J, Scott G: The link between emotional competence and well-being: a longitudinal study. Brit J Guid Couns. 2006, 34: 231-243. 10.1080/03069880600583287.
Ciarrochi J, Deane FP, Anderson S: Emotional intelligence moderates the relationship between stress and mental health. Pers Indiv Differ. 2002, 32: 197-209. 10.1016/S0191-8869(01)00012-5.
Sonnentag S, Binnewies C, Mojza EJ: Staying well and engaged when demands are high: The role of psychological detachment. J Appl Psychol. 2010, 95: 965-976.
Berset M, Elfering A, Lüthy S, Lüthi S, Semmer NK: Work stressors and impaired sleep: rumination as a mediator. Stress and Health. 2011, 27: e71-e82. 10.1002/smi.1337.
Semmer NK, Grebner S, Elfering A: Beyond self-report: Using observational, physiological, and situation-based measures in research on occupational stress. Research in Occupational Stress and Well-being. 2003, Bingley: Emerald (MCB UP), 205-263.
Cuijpers P, Donker T, van Straten A, Li J, Andersson G: Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med. 2010, 40: 1943-1957. 10.1017/S0033291710000772.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/13/655/prepub
Recruitment began on March 1, 2013 and will continue through October 2013. Trial findings at post-test are likely to be available in November 2013.
We would like to acknowledge the European Union (EFRE) for funding this project within the Lueneburg Innovation Incubator, TM 1.1 (project number: CCI 2007DE161PR001). Additionally, this study is funded by the health insurance company “Barmer GEK”, which provides financial support for the payments of the eCoaches. Furthermore, we would like to acknowledge our colleagues Hanne Thiart and Torsten Tarnowski for contributing to the development of the intervention.
Professor Berking is minority shareholder of Minddistrict GmbH which will provide the online training platform.
MB obtained funding for this study. All authors contributed to the design of the study. EH, DL, DE and SN developed the intervention content. EH wrote the draft of the manuscript. HR supervised the writing process. All authors contributed to the further writing of the manuscript and approved the final version of the manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
About this article
Cite this article
Heber, E., Ebert, D.D., Lehr, D. et al. Efficacy and cost-effectiveness of a web-based and mobile stress-management intervention for employees: design of a randomized controlled trial. BMC Public Health 13, 655 (2013). https://doi.org/10.1186/1471-2458-13-655
- Internet intervention
- Work-related stress
- Stress management
- Occupational health