Skip to main content
  • Research article
  • Open access
  • Published:

Moving to business – changes in physical activity and sedentary behavior after multilevel intervention in small and medium-size workplaces

Abstract

Background

Regular physical activity (PA) promotes and excessive sedentary behavior (SB) deteriorates health. Yet the Finnish working-aged population spends most of the day sitting. A 1-year Moving To Business (MTB) -intervention supported small and medium-size workplaces to combat sedentariness. This paper reports the changes in employees’ PA and SB from before MTB (baseline) to 1 year after baseline (follow-up).

Methods

Twelve workplaces with a total of 396 employees participated. Each workplace nominated a team to promote PA and reduce SB at organizational, working unit and employee level. The teams were mentored regionally through meetings, workshop and tools. Changes in PA and SB were assessed with a questionnaire and an accelerometer. Wald Confidence Interval (Cl) for a difference of proportions with matched pairs was used in the questionnaire data (%-points with 95% CI) and linear mixed model in the accelerometer data (minutes and % of wear-time with 95% CI).

Results

The mean age of the respondents to the questionnaire (N = 296; 75%) was 42.6 (SD 10.9), 64% were women, 95% had some education after high school, 74% worked in the day shift, 71% did sedentary work and 51% were overweight. The mean number of actions implemented in the workplaces was 6.8 and the multilevel approach was fully applied in 6 workplaces. Based on the questionnaire the time spent in SB decreased from baseline to follow-up 16% (95% CI −29 to −3) in total and 22% (−41 to −3) at work. The accelerometer showed daily increases of 33.7 min (15.3 to 52.1) and 6.8% (3.1 to 10.4) in total PA, 30.9 min (15.3 to 46.5) and 6.1% (2.9 to 9.2) in light PA and 673 (209 to 1139) more steps at work. Daily SB at work decreased 44.9 min (−68.0 to −21.8) and 7.6% (−11.9 to −3.2). Daily leisure PA declined 11.0 min (−24.9 to 2.9) and 3.2% (−6.2 to −0.2). Number of levels or actions had no effect on changes.

Conclusions

Employees’ PA increased and SB reduced at work during the intervention. At the same time leisure PA decreased slightly. Workplaces can achieve meaningful changes in employees’ PA and SB if assisted systematically. Controlled studies are needed to confirm the present findings.

Trial registration

NCT01999205, registration date 11/01/2013.

Peer Review reports

Background

To preserve cardio-vascular health, every working-aged adult should be engaged in moderate-intensity physical activity (PA) for at least 150 min a week [1]. However, only 25 to 30% of the Finnish working-aged population meets this endurance part of the PA recommendation [2].

Being less active than recommended (i.e. being inactive) increases the risk for several chronic diseases such as type 2 diabetes, coronary heart disease, stroke, and colon cancer [3]. Recently it has been discovered that beyond inactivity, excessive prolonged sitting or more broadly sedentary behavior (SB) defined as “any waking behavior characterized by an energy expenditure ≤1.5 METs while in a sitting or reclining posture“[4] deteriorates health irrespective of meeting the PA recommendation [58]. Yet the Finnish working-aged population is sedentary almost two thirds of their day [2], which is the situation also in many other developed countries [9, 10].

Workplaces have been identified as an important forum for health promotion because they offer a setting for reaching large groups of working-aged population and enable utilization of existing social networks [11]. However, the impacts of workplace health promotion on, for example, employees’ weight, dietary habits, work absences and productivity have generally been small, short-lived or inconclusive [12], which applies also to PA promotion interventions [1315]. The findings on reducing SB seem more promising and consistent especially when it comes to the use of sit-stand desks and active workstations [16, 17]. Results from interventions using larger diversity of strategies to reduce SB are not quite as optimistic [1820].

To promote worksite health more productively, documents outlining best practices [21, 22], scientific reviews [23], expert opinions [2426] and individual studies [27] all advocate the use of socio-ecological models [28, 29]. These models suggest that people’s health behavior is influenced not only by their intrapersonal characteristics but also by a wide variety of factors in their social, environmental and policy contexts. Thus, to achieve change in health behavior, actions should be targeted simultaneously at multiple levels. A multilevel approach is also thought to benefit in reaching such groups of people, who may remain underrepresented in single-level interventions [29]. For instance, in worksite PA promotion, people may be reached more effectively through modifying their physical environment at work to favor physical activity rather than providing them with personal incentive for leisure exercise.

The use of a multilevel approach seems particularly justified to promote PA and reduce SB since both behaviors are strongly connected with specific contexts such as schools, workplaces and homes [30]. Moreover, several previous studies indicate that the determinants of PA [3135] and SB [3639] are multilevel in nature. Also, studies examining the effectiveness of interventions on PA promotion [30, 40, 41] and SB reduction [19, 42, 43] speak for the use of multilevel approach.

However, so far the application of multilevel approach to health promotion and PA interventions has been limited [23, 4446]. It seems that the empirical evidence on the effectiveness of multilevel interventions in relation to single-level interventions is yet unclear [44], which may be one reason for the limited use. As to the worksite setting in particular, only one review [47] has compared the effectiveness of single and multilevel strategies in changing employees’ PA behavior. The review showed most promise for interventions, which combined both environmental, policy and individual-level strategies.

The aim of this 1-year Moving To Business (MTB) intervention was to 1) support small and medium-size workplaces to plan and implement a multilevel intervention to increase PA and reduce SB among employees and to 2) evaluate the implementation and outcomes of the intervention. This paper reports the planning and implementation of the intervention as well the changes in PA and SB from baseline to 1-year follow-up.

Methods

MTB was designed and evaluated by two research organizations, The UKK Institute for Health Promotion Research and the Finnish Institute of Occupational Health, and was implemented by the Finnish Sports Confederation in collaboration with three regional sports federations. The study plan was approved by the Ethics Committee of the Tampere Region, under the auspices of University of Tampere, Human Sciences (http://www.uta.fi/english/research/ethics/review/committee.html, running number 17/2013).

Workplaces and employees

Regional contact persons from the collaborating regional sports federations recruited three to five small or medium-size workplaces to MTB. According to Finnish criteria enterprises with fewer than 50 employees are defined small and those with 50–249 employees medium-size [48]. During the recruitment process the contact persons kept record of the workplaces approached, recruitment methods used and the reasons given by the workplaces for not participating in MTB.

The workplaces accepting the invitation paid a participation fee of 2000 or 3000 Euros depending on their size. The fee engaged the workplaces in MTB and covered some of the implementation costs. For the additional commitment the workplaces signed a written contract with the Finnish Sports Confederation. The workplaces were also urged to contact their occupational health care provider to facilitate their participation in MTB. However, the workplaces were not obliged to do so because especially in small workplaces it may have required expansion of their existing occupational health care contracts leading to extra costs.

Intervention

The MTB intervention comprised starting, active and closing phases. For the execution of the intervention each workplace nominated an internal MTB team involving members from the management, human resources, working staff and desirably also from occupational health care. With the support of regional contact persons the MTB teams specified the goals for increasing PA and reducing SB among employees and planned and implemented actions at organization, working unit and individual employee level to reach the goals.

At the starting phase (January 2014), a 4-hour opening meeting was arranged for the MTB teams by their regional contact person to discuss the needs and current practices for promoting PA and reducing SB at each particular workplace (Table 1). Results from the baseline measurements (November 2013) were presented to help the teams in goal setting and action planning. Also, the teams were provided with a planning sheet, which they used for scheduling the multilevel actions.

Table 1 Pre-scheduled actions and measurements by month during 2013–15 in the Moving To Business (MTB) –intervention

At the active phase (February – October 2014) the regional contact persons provided the MTB teams with support and tools to implement the actions: a 4-hour training on occupational health care’s role in workplace health promotion, two meetings for the MTB teams in the same area to facilitate their networking and one workplace-specific meeting to preview the multilevel action plans. In addition, the MTB teams had an opportunity to get extra support from the regional contacts upon request (face-to-face and telephone consultation, group exercise services, help and material for organizing campaigns etc.). Moreover, the employees of the participating workplaces were offered a one to two-hour workshop on reducing SB and a possibility to use free-of-charge an internet-based platform (https://www.heiaheia.com) to monitor their PA and to share the information with their workmates and friends. Two separate Heiaheia-campaigns were also provided for the MTB teams to be delivered in the workplaces (“Active commuting to work” and “Active working day”).

At the closing phase (March 2015), a 4-hour closing-up meeting was arranged for the MTB teams located in the same regional area to share general and workplace-specific feedback on implementation and results.

The regional contact persons reported monthly on the progress in their workplaces with a structured form to the MTB coordinators (AR, ML) at the Finnish Sports Confederation. The coordinators, in turn, updated the developments to the MTB core group, which had also representatives from the research organizations (MA, SL).

Measures

MTB targeted at all the employees in the participating workplaces. However, completing the baseline and follow-up measures was voluntary for ethical reasons. The follow-up measurements were repeated 1 year after the baseline the same time of the year (November) for seasonal comparability.

Questionnaire

At baseline the employees completed a self-administered questionnaire on their demographics, work, work ability, work engagement and recovery, PA, SB (sitting), perceived health, smoking and sleep. The questions on the quantity of PA and SB were the same as used in Finnish population surveys [49] (Additional file 1). The SB questions in the surveys are identical to the Workforce Sitting Questionnaire, which has been found valid for assessing sitting time at work as well as total sitting time during working and non-working day [50]. Compliance with the questionnaire was facilitated in the cover letter by emphasizing that the evaluation focused on the multilevel intervention, not on individual employees.

Accelerometer

PA and SB (sitting + reclining posture) were objectively assessed with a hip-worn accelerometer (Hookie AM13, Traxmeet Ltd., Espoo, Finland, http://company.traxmeet.com), which employed a triaxial sensor component (ADXL345; Analog Devices, Norwood MA). This device has been found as valid as the most commonly used accelerometer (Actigraph GTX3, Actigraph LLC, Pensacola FL, USA, http://actigraphcorp.com) in assessing adults’ PA and SB [51]. The employees were provided with written instructions to wear the accelerometer on their right hip during the waking hours for seven consecutive days and to remove the device only for sauna, shower and water activities.

To facilitate the accelerometer use, the employees were offered a graphical feedback about their PA and SB after the baseline and follow-up measurements. The MTB teams were also provided with graphics about PA and SB in the entire workplace. The baseline graphics were also utilized for goal setting and action planning.

Diary

A diary was used to extract specific data on working days and hours from the accelerometer data accumulated. On each accelerometer day the employees were asked to enter the following information to the day-specific row: date, working day/non-working day, time leaving home and arriving at work (hours: minutes), transportation modes used (1 = walking, 2 = cycling, 3 = other active mode, 4 = car, 5 = bus, 6 = train, 7 = moped or motorcycle, 8 = other motorized mode), time starting and ending the work (hours: minutes), time leaving work and arriving at home (hours: minutes), and transportation modes used.

Data analysis

The acceleration data were collected in raw mode, which presents the acceleration data in actual G-force units (milligravity, mg). The accelerometer collected the data with a 100 Hz sampling frequency and a ± 16,000 mg dynamic range. After the measurement period the stored data were transferred to a hard disk and analysed in 6-s epochs.

Earlier studies indicate that 4 days with at least 10 h from each day is the minimal amount of data needed for reliable estimates of adults’ weekly PA with hip-worn monitors [52, 53]. Accordingly data from at least three working days with the minimum of 10 h wear-time was considered sufficient to describe employees’ PA and SB during regular working days. This amount of data covered 60% of the working days during a regular 5-day working week. In addition, 4 hours of wear-time was required during the working hours equaling 50% of the normal 8-h working day. Non-wear time was defined as no movement detected in any epoch for at least 30 min.

Accelerometer-specific cut-points, which have been previously determined from adults’ raw acceleration data by using mean amplitude deviation (MAD), were used to classify the intensity of employees’ intensity-specific PA (light, moderate, vigorous) and to separate SB (sitting + reclined posture) from PA [54]. Transition from sitting to standing posture was calculated on the basis of the number of SB periods ending with standing up. Standing was detected if the MAD value was greater than 50 mg for the preceding or same epoch when the measured posture changed to standing. [Vähä-Ypyä H, Husu P, Suni J, Vasankari T: Reliable recognition of lying, sitting and standing with a waist waist-worn accelerometer, submitted].

Descriptive information is presented as means and standard deviations (SD). Changes in the questionnaire-based PA and SB (sitting) variables from baseline to 1-year follow-up are reported in percentage points (%-points) and their 95% confidence intervals (95% CI) calculated with Wald Confidence Interval for a difference of proportions with matched pairs. The analysis included employees, who had questionnaire-based PA and SB data from both time points.

The accelerometer data at baseline and follow-up as well as the changes from baseline to follow-up are reported in mean minutes (or steps) and percentages of wear-time (% wear-time). Linear mixed model was used in analyzing the changes in employees, who had data from both time points. The outcome was the difference between the follow-up and baseline value of each specific variable. The baseline value was used as a confounding factor and the workplace served as a random effect. Sex, age (continuous), perceived health (poor or fairly poor; average; good or fairly good), physical exertion of the work (sedentary work; work with light or moderate mobility; heavy or extremely heavy physical work), working time (regular day shift; shift work without nightshifts; other) and education (basic education, high school or vocational school; university of applied sciences; university) were added as potential confounding factors in the model but removed one by one if they did not improve the model’s Bayesian Information Criterion [55] and were not statistically significant (p < 0.05). In addition, the change in wear-time was taken into account when analyzing the changes in mean minutes.

Linear Mixed Models with workplace as random effect were used to test whether the changes in PA and SB were different between the workplaces implementing more actions and fewer actions than on average and between the workplaces implementing actions at all three levels and at one or two levels.

Changes in PA and SB at individual workplaces were not analyzed because the number of employees having both baseline and follow-up accelerometer data was too small in the majority of workplaces for statistical or pragmatic conclusions (less than 10 employees in 5 workplaces, less than 20 in 5 workplaces, 38 in one workplace and 42 in one workplace).

Results

Participants

The three regional contact persons approached altogether 18 workplaces. Of them, 12 (67%) agreed to participate in the study (Table 2). One regional contact person succeeded in enrolling 3, one 4 and one 5 workplaces. The most common reasons for refusing participation were lack of time and resources as well as too many concurrent projects. The total number of employees in the participating workplaces was 396 ranging from 13 to 107 by individual workplace.

Table 2 Participating workplaces, their field of activity, number of employees, respondents to the questionnaire, employees with valid accelerometer data (Acc)a and employees with valid accelerometer data and diary entries (Acc + D)b at baseline and follow-up

The baseline questionnaire was responded by 295 (74%) employees and 266 (67%) had valid accelerometer and diary information. At 1-year follow-up all the workplaces enrolled were still involved but the number of employees in the workplaces had reduced by 37 (9% out of 396) leaving 359 persons to the follow-up sample. Of them, 201 (56%) responded to the follow-up questionnaire and 175 (49%) had relevant accelerometer and diary data.

The mean age of the respondents to the baseline questionnaire was 42.6 (SD 10.9), 64% were women, 95% had some education after high school, 74% worked in the day shift, 71% did sedentary work and 51% were overweight. More specific information on the baseline characteristics is given in Table 3.

Table 3 Baseline characteristics of the respondents to the baseline questionnaire (n = 296, 75%)

Both baseline and 1-year follow-up data were obtained with the questionnaire from 186 employees (47% from the original sample of 396 employees) and with the accelerometer and diary from 147 (37%) employees. The employees completing both the baseline and follow-up questionnaire were similar to those responding only to the baseline questionnaire in terms of sex, age, health, physical exertion of the work, working time, education, PA and SB (Additional file 2). Similarly, there was no statistical difference for sex, age, health, physical exertion of the work, working time, education, PA, SB between the employees, who completed the accelerometer and diary measurements only at baseline compared to those with data from both time points (Additional file 2).

Implementation

All workplaces nominated the MTB teams, made actions plans for promoting PA and reducing SB and participated in the follow-up measurements. The size of the MTB teams ranged from 2 to 8 members. All 12 MTB teams included at least one representative from HR, working staff and occupational safety. Nine teams included also at least one member from the management, and 4 teams from the occupational health care.

Nearly all (11/12) workplaces aimed primarily at reducing SB. Sit-stand workstations, exercise equipment for collective use and opportunity to experience different modes of instructed exercise were among the most common actions implemented by the MTB teams (Table 4). The total number of various actions during MTB was 39: 16 (41%) focused on organization, 15 (38%) on working unit and 8 (21%) on individual employee. The mean number of actions implemented in the workplaces was 6.8 ranging from 2 to 11. Multilevel implementation at all three levels was applied in six workplaces (1, 4, 5, 7, 11, 12), while five workplaces (2, 3, 6, 9, 10) implemented actions at two levels and one workplace (8) at one level.

Table 4 Actionsa implemented at the three levels (organization, working unit, employee) in the participating workplaces for increasing physical activity and reducing sedentary behavior

PA and SB

Questionnaire

Table 5 presents the questionnaire-based findings on employees’ weekly PA and SB (sitting) at baseline, 1-year follow-up and changes from baseline to 1-year follow-up. At baseline the employees reported an average of 240 min of weekly leisure PA. The mean total duration of daily SB was 509 min, which was mostly (305 min) accumulated at work. At follow-up, the daily self-reported SB at work reduced from baseline to 275 min denoting a 22% (95% CI −41% to −3%) decrease. At the same time the proportion of active commuters decreased by 6% (95% CI −11% to −2%). No significant changes were observed in the weekly leisure PA or in daily breaks in sitting at work. Neither were there any changes in the amount of SB during transportation or leisure (TV, video, computer, other).

Table 5 Questionnaire-based employees’ physical activity (PA), sitting and daily breaks from sitting at baseline and at 1-year follow-up

Accelerometer

Table 6 presents the accelerometer-based findings on daily PA and SB (sitting + reclined posture) at baseline, 1-year follow-up and changes from baseline to 1-year follow-up.

Table 6 Accelerometer-based employees’ physical activity (PA) and sitting during a working day (working time + leisure time) at baseline and 1-year follow-up in mean minutes (\( \overset{-}{x} \)) and their standard deviations (SD) and change from baseline to 1-year follow-upa in mean minutes (\( \overset{-}{x} \))b and percentages of wear-time (% wear-time) and their 95% confidence intervals (95% CI)

At baseline the employees were physically active at work on average 111 min a day, took 3802 steps, spent 299 min in SB and had 24 breaks from SB. During leisure, they had on average 127 min daily PA with 4952 steps and they spent 245 min in SB. At follow-up, the employees’ daily PA at work increased on average 34 min (95%CI 15.3 to 52.1) and 6.8% (95%CI 3.1 to 10.4) of wear-time compared with baseline. The employees also increased their daily steps by 673 (95%CI 209 to 1139) from baseline. In addition, their daily light-intensity PA increased 31 min (95%CI 15.3 to 46.5) and 6.1% (95%CI 2.9 to 9.2) of wear-time. Moreover, daily SB at work decreased 45 min (95% CI −68.0 to −21.8) and 7.6% (95%CI −11.9 to −3.2) of wear-time. However, at the same time, the total minutes of daily leisure PA decreased 11 min (95%CI −24.9 to 2.9) and 3.2% (95%CI −6.2 to −0.2) of wear-time.

Changes in PA and SB in relation to implementation

No significant differences were observed in changes in PA or SB between the workplaces implementing more or fewer actions than on average (6.8) or between the workplaces implementing actions at all three levels and just one or two levels (Additional file 3).

Discussion

The purpose of the MTB intervention was to support small and medium-size workplaces to plan and implement multilevel intervention to increase PA and reduce SB among employees and to evaluate the implementation and outcomes of the intervention.

Summary of key findings

During MTB, the employees’ self-reported SB (sitting) at work and in total decreased. Also objectively measured SB at work decreased and the minutes spent in total and light-intensity PA at work increased, which was also seen as the higher number of steps taken at work. If sustained, the magnitudes of these changes may be relevant for health although light-intensity PA contributed most of the increase in PA at work and was presumably also the primary replacement for SB. Nevertheless, recent studies indicate that light activity is better than SB for health [56].

Employees seemed to compensate part of their increased PA at work with a decrease in leisure PA, since the objectively measured total minutes of leisure PA were somewhat lower at follow-up compared with baseline. However, the mean decrease in leisure PA (−11 min) did not reach statistical significance and was clearly less than the concurrent increase in PA at work (+33.7 min). Yet this finding supports previous studies indicating a similar compensatory effect [57] and may pertain to the risk compensation theory, which suggests that a positive change achieved at one level (at work) can be replaced with opposite behaviour at another level (leisure) if the change is not facilitated simultaneously at both levels [44]. This may also explain the decrease in the self-reported proportion of active commuters at least to some extent, since only one workplace (8) with 20 employees implemented actions to promote active commuting to work apart from the HeaiHeia –campaign arranged in all the workplaces.

Implementing actions at all three levels or only at one or two levels seemed not to have effect on objectively measured PA or SB changes. This finding is contradictory to the views favoring multilevel approaches in encouraging healthy behaviors in worksite settings [47]. However, to our knowledge, no intervention studies have yet been published, which have compared multilevel and single-level approaches in PA promotion or SB reduction at worksite setting. Thus, more research is needed in this area.

Strengths

The primary strength of the study was that it was conducted in a real-world setting. This may allow more direct translation of the results to practice and policy [23]. Also, the actions were mostly planned and implemented by the workplaces themselves, which made the intervention highly participative. In a study by Witt et al. [58] involving small and midsize enterprises the employee buy-in was seen the most prominent barrier to health promotion. The companies thus emphasized the role of building employee-ownership to health promotion actions by involving workers in the planning process or surveying their preferences before planning and implementing the health promotion actions. Participatory actions may also enhance the sustainability of behavior changes beyond the intervention period [59].

Another strength pertains to the fact that the implementation of the present intervention followed a theoretical background, particularly multilevel principles of the socio-ecological models, but within the terms of each individual workplace. The novelty and value of the MTB intervention is thus that the selection of the multilevel actions was based on the needs and resources of individual workplaces. However, it is acknowledged that this inevitably led to a wide variety in actions and their contents and delivery, which made it impractical to evaluate which actions or combinations of actions were the ones behind the changes. Nevertheless, according to previous studies [60] it seems unlikely that one specific action or combination of actions would be equally feasible and effective in all workplaces where it is being implemented. It has been suggested that multiple factors [61], for instance organizational climate [62], affect employees’ participation in workplace health promotion and subsequently the feasibility and outcomes of the health promotion activities [63]. In this study the actions were planned and implemented by the MTB teams, who had the best internal knowledge about what was likely to work in their workplaces. This way we expected to maximize employees’ participation in the measurements and also their exposure to the actions.

The present study included also the evaluation of implementation covering both the organization of the MTB teams and the actions carried out in the workplaces. The importance of evaluating the implementation of workplace interventions has been highlighted to understand the intervention effects [64, 65] and to better translate research to practice [66, 67].

One asset of the present study was also the diversity of the size and field of activity of the workplaces involved. This supports the feasibility of similar interventions in various types of workplaces. On the other hand, the diversity prevents one from analyzing changes and making conclusions about the generalizability of the results in certain types of workplaces since the number of similar workplaces, e.g. those with manual workers only, was low.

Finally, the strength of the study was the use of objective measures along with self-reports in assessing PA and SB. Although accelerometers have become common in real-world interventions, their use is still subject to various issues in wearing compliance and data processing issues [53]. In this study, the compliance of the employees was facilitated by providing them with personal feedback from the accelerometer data. The data processing issues were managed by using raw acceleration data and a novel classification method, which is able to distinguish intensity-specific PA and to separate SB and reclining posture from standing. However, as reclined posture could not be reliably separated from sitting, it was combined with sitting. This is in line with the current definition of SB [4].

Limitations

The study also had some limitations, which should be recognized when interpreting the results. Most notably, the study did not include a control group. Thus, it cannot be exclusively concluded that the positive changes in PA and SB from baseline to follow-up were due to the intervention. It may have been, for example, that the general awareness among the working-aged population about the health hazards of SB increased during the intervention and decreased SB in the workplaces regardless of the intervention. To see the actual effects of the intervention on PA and SB, the study needs to be repeated in a condition, where some workplaces are randomly assigned to the similar intervention as conducted in this study and other workplaces are randomly exposed to measurements only. Unfortunately the financial and human resources allocated to the present study did not allow a controlled design.

Attention should also be paid to the fact that the workplaces participating in the study were interested in developing their practices related to PA promotion and therefore represented a convenience sample of workplaces with high receptivity. The same applies to the employees, who participated in the intervention by responding to the questionnaire and completing the accelerometer and diary measurements. This kind of selectivity happens to some extent in most studies conducted in worksite setting [68] and may not be totally removed because the workplaces and employees cannot be forced to participate due to ethical reasons. In this study the completion rates of the self-reported and objective baseline measurements (74% and 67%, respectively) were higher than the mean adherence in worksite health promotion programs [68]. Furthermore, no difference was detected in the background characteristics, PA and SB between the employees who participated only in the baseline measurements and in those who participated in both measurements. It seems, therefore, that despite the possible selection bias caused by the voluntariness in the beginning of the study, no such selection happened within the participating employees during the course of the intervention, which would have weakened the representativeness of the sample and furthermore the interpretation of the results in this particular sample.

Comparison with earlier studies

We are not aware of any study, which has applied socio-ecological models in exactly similar way to promote PA and reduce SB in a workplace setting. The randomized controlled 12-week Move to Improve –intervention by Dishman et al. [69] resembles the present intervention. It aimed to increase moderate and vigorous-intensity PA in 16 worksites with 1442 employees by applying goal setting at organizational, team and employee level. The outcomes in PA (SB was not evaluated) were assessed with the short form of International Physical Activity Questionnaire (IPAQ) and a pedometer. As a result, walking and the levels of moderate and vigorous-intensity PA increased in comparison with the control group. Also the proportion of employees meeting the PA recommendations increased. The findings indicate the effectiveness of multilevel approach also in a controlled condition. However, due to different contents of the intervention the results are not comparable with the present study.

Another resemplance to the present MTB-intervention is a cluster randomized Stand Up Victoria trial conducted in the 14 sites of one large government organization in Australia [70]. It targeted at reducing employees’ (n = 231) sitting and applied multilevel approach including organizational (management consultation, workshop for the representatives, brainstorming session for the participants), environmental (sit-stand workstation + instructions on ergonomics) and individual (health coaching and supportive telephone calls) strategies. Assessment of sitting was based on 7-day accelerometer-use at baseline and at three and 12 months. Large differences in changes were discovered between the intervention and control sites at both follow-ups in daily workplace sitting (−99 and −45 min) and standing (+95 and +43 min) as well as in daily overall sitting (−78 and −36 min) and standing (+76 and +41 min) [71]. Although the changes in sitting were larger and the procedure and elements of the Stand Up Victoria -intervention were different from MTB, it supports the present findings. Encouraging results have also been obtained from smaller multilevel trials aiming to reduce sitting (e.g. [72]).

In summary, MTB and similar interventions with comparison groups seem to support multilevel approaches to promote PA and reduce SB at worksite setting. However, more research is still needed on the effectiveness of specific actions or combinations of actions implemented at different levels.

Conclusions

During the MTB intervention, employees’ PA increased and SB reduced at work. Concurrently, employees’ leisure PA slightly decreased but clearly less than the PA at work increased. Number of actions or levels seemed not to have effect on results. Apparently, workplaces can achieve meaningful changes in employees’ PA and SB if assisted systematically. Controlled studies with similar participatory multilevel approach are needed to confirm the present findings.

Abbreviations

MAD:

Mean amplitude deviation

MTB:

Moving to Business -intervention

PA:

Physical activity

SB:

Sedentary behavior

References

  1. Physical Activity Guidelines Advisory Committee. Physical Activity guidelines advisory committee report, 2008. Washington DC: U.S. Department of Health and Human Services; 2008. https://health.gov/paguidelines/Report/pdf/CommitteeReport.pdf

    Google Scholar 

  2. Husu P, Suni J, Vähä-Ypyä H, Sievänen H, Tokola K, Valkeinen H, Mäki-Opas T, Vasankari T. Objectively measured sedentary behavior and physical activity in a sample of Finnish adults: A cross-sectional study. BMC Public Health 2016; doi:10.1186/s12889-016-3591-y.

  3. Lim SS, Vos T, Flaxman AD et al. A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 2012; doi:10.1016/S0140-6736(12)61766-8.

  4. Sedentary Behaviour Research Network. Letter to the Editor: Standardized use of the terms “Sedentary” and “sedentary behaviours.” Appl Physiol Nutr Metab 2012; doi:10.1139/H2012-024.

  5. Biswas A, Oh PI, Faulkner GE, Bajaj RR, Silver MA, Mitchell MS, Alter DA. Sedentary time and its association with risk for disease incidence, mortality, and hospitalization in adults: a systematic review and meta-analysis. Annals Intern Med 2015; doi:10.7326/M14-1651.

  6. Gibbs BB, Hergenroeder AL, Katzmarzyk PT, Lee IM, Jakicic JM. Definition, measurement, and health risks associated with sedentary behavior. Med Sci Sports Exerc 2015; doi:10.1249/MSS.0000000000000517.

  7. Lee I-M, Shiroma EJ, Lobelo F et al. for the Lancet Physical Activity Series Working Group. Effect of physical inactivity on major non-communicable diseases worldwide: an analysis of burden of disease and life expectancy. Lancet 2012; doi:10.1016/S0140-6736(12)61031-9.

  8. Matthews CE, George SM, Moore SC, Bowles HR, Blair A, Park Y, Troiano RP, Hollenbeck A, Schatzkin A. Amount of time spent in sedentary behaviors and cause-specific mortality in US adults. Am J Clin Nutr 2012; doi:10.3945/ajcn.111.019620.

  9. Milton K, Gale J, Stamatakis E, Bauman A. Trends in prolonged sitting time among European adults: 27 country analysis. Prev Med 2015; doi:10.1016/j.ypmed.2015.04.016.

  10. Loyen A, van der Ploeg HP, Bauman A, Brug J, Lakerveld J. European sitting championship: prevalence and correlates of self-reported sitting time in the 28 European Union member states. PLoS ONE 2016; doi:10.1371/journal.pone.0149320.

  11. Dishman RK, Oldenburg B, O'Neal H, Shephard RJ. Worksite physical activity interventions. Am J Prev Med. 1998;15:344–61.

    Article  CAS  PubMed  Google Scholar 

  12. Rongen A, Robroek SJW, van Lenthe FJ, Burdorf A. Workplace health promotion. A meta-analysis of effectiveness. Am J Prev Med 2013; doi:10.1016/j.amepre.2012.12.007.

  13. Conn VS, Hafdahl AR, Cooper PS, Brown LM, Lusk SL. Meta-analysis of workplace physical activity interventions. Am J Prev Med 2009; doi:10.1016/j.amepre.2009.06.008.

  14. Hutchinson AD, Wilson C. Improving nutrition and physical activity in the workplace: a meta-analysis of intervention studies. Health Prom Int 2011; doi:10.1093/heapro/dar035.

  15. Van Dongen JM, Proper KI, van Wier MF, van der Beek AJ, Bongers PM, van Mechelen W, van Tulder MW. A systematic review of the cost-effectiveness of worksite physical activity and/or nutrition programs. Scand J Work Environ Health 2012; doi: 10.5271/sjweh.3275.

  16. Commissaris DACM, Huysmans MA, Mathiassen SE, Srinivasan D, Koppes LLJ, Hendriksen IJM. Interventions to reduce sedentary behavior and increase physical activity during productive work: a systematic review. Scand J Work Environ Health 2016; doi:10.5271/sjweh.3544.

  17. Neuhaus M, Eakin EG, Straker L, Owen N, Dunstan DW, Reid N, Healy GN. Reducing occupational sedentary time: a systematic review and meta-analysis of evidence on activity-permissive workstations. Obes Rev 2014; doi:10.1111/obr.12201.

  18. Chau JY, der Ploeg HP, van Uffelen JG, Wong J, Riphagen I, Healy GN, Gilson ND, Dunstan DW, Bauman AE, Owen N, Brown WJ. Are workplace interventions to reduce sitting effective? A systematic review. Prev Med 2010; doi:10.1016/j.ypmed.2010.08.012.

  19. Gardner B, Smith L, Lorencatto F, Hamer M, Biddle SJ. How to reduce sitting time? A review of behaviour change strategies used in sedentary behaviour reduction interventions among adults. Health Psychol Rev 2015; doi:10.1080/17437199.2015.

  20. Shrestha N, Kukkonen-Harjula KT, Verbeek JH, Ijaz S, Hermans V, Bhaumik S. Workplace interventions for reducing sitting at work. Cochrane Database of Systematic Reviews 2016, Issue 3. Art. No.: CD010912. doi:10.1002/14651858.CD010912.pub3.

  21. Carnethon M, Whitsel LP, Franklin BA, Kris-Etherton P, Milani R, Pratt CA, Wagner GR. Worksite wellness programs for cardiovascular disease prevention: a policy statement from the American Heart Association. Circulation 2009; doi:10.1161/CIRCULATIONAHA.109.192653.

  22. WHO. Healthy workplaces: a model for action for employers, workers, policymakers and practitioners. Geneva: WHO; 2010. http://www.who.int/occupational_health/publications/healthy_workplaces_model.pdf. Accessed 10 Sep 2015

    Google Scholar 

  23. Pronk N. Physical activity promotion in business and industry. Evidence, context, and recommendations for a national plan. J Phys Act Health 2009; 6 Suppl 2:220–235.

  24. Glasgow R, Vogt T, Boles S. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89:1322–7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Green LW, Kreuter MW. Health promotion planning: an educational and ecological approach. New York: McGraw-Hill; 1999.

    Google Scholar 

  26. Sallis JF, Owen N, Fisher EB. Ecological models of health behavior. In: Glanz K, Rimer BK, Viswanath K, editors. Health behavior and health education: theory, research, and practice. San Francisco: Jossey-Bass; 2008. p. 465–86.

    Google Scholar 

  27. Healy GN, Eakin EG, LaMontagne AD, Owen N, Winkler EAH, Wiesner G, Gunning L, Neuhaus M, Lawler S, Fjeldsoe BS, Dunstan DW. Reducing sitting time in office workers: Short-term efficacy of a multicomponent intervention. Prev Med 2013; doi:10.1016/j.ypmed.2013.04.004.

  28. McLeroy KR, Bibeau D, Steckler A, Glanz K. An ecological perspective on health promotion programs. Health Educ Behav. 1988;15:351–77.

    Article  CAS  Google Scholar 

  29. Stokols D, Allen J, Bellingham RL. The social ecology of health promotion: implications for research and practice. Am J Health Promot. 1996;10:247–51.

    Article  CAS  PubMed  Google Scholar 

  30. Sallis JF, Bauman A, Pratt M. Environmental and policy interventions to promote physical activity. Am J Prev Med. 1998;15:379–97.

    Article  CAS  PubMed  Google Scholar 

  31. Ball K, Timperio A, Salmon J, Giles-Corti B, Roberts R, Crawford D. Personal, social and environmental determinants of educational inequalities in walking: a multilevel study. J Epidemiol Commun Health. 2007;61:108–14.

    Article  Google Scholar 

  32. Bauman AE, Reis RS, Sallis JF, Loos RJ, Martin BW for the Lancet Physical Activity Series Working Group. Correlates of physical activity: why are some people physically active and others not? Lancet 2012; doi:10.1016/S0140-6736(12)60735-1.

  33. Cerin E, Leslie E, Sugiyama T, Owen N. Perceived barriers to leisure-time physical activity in adults: an ecological perspective. J Phys Act Health. 2010;7:451–9.

    Article  PubMed  Google Scholar 

  34. Rhodes RE, Pfaeffli LA. Mediators of physical activity behavior change among adult non-clinical populations: a review update. Int J Behav Nutr Phys Act 2010; doi:10.1186/1479-5868-7-37.

  35. Saelens BE, Sallis JF, Frank LD, Cain KL, Conway TL, Chapman JE, Slymen DJ, Kerr J. Neighborhood environment and psychosocial correlates of adults’ physical activity. Med Sci Sports Exerc. 2012; doi:10.1249/MSS.0b013e318237fe18.

  36. Bennie JA, Timperio AF, Crawford DA, Dunstan DW, Salmon JL. Associations between social ecological factors and self-reported short physical activity breaks during work hours among desk-based employees. Prev Med 2011; doi:10.1016/j.ypmed.2011.05.015.

  37. O’Donoghue G, Perchoux C, Mensah K, Lakerveld J, van der Ploeg H, Bernaards C, Chastin SFM, Simon C, O’Gorman D, Nazare J-A on behalf of the DEDIPAC consortium. A systematic review of correlates of sedentary behaviour in adults aged 18–65 years: a socio-ecological approach. BMC Public Health 2016; doi:10.1186/s12889-016-2841-3.

  38. Owen N, Sugiyama T, Eakin EE, Gardiner PA, Tremblay MS, Sallis JF. Adults' sedentary behavior determinants and interventions. Am J Prev Med 2011; doi:10.1016/j.amepre.2011.05.013.

  39. Owen N, Salmon J, Koohsari MJ, Turrell G, Giles-Corti B. Sedentary behavior and health: mapping environmental and social contexts to underpin chronic disease prevention. Br J Sports Med 2014; doi:10.1136/bjsports-2013-093107.

  40. Cochrane T, Davey RC. Increasing uptake of physical activity: a social ecological approach. J R Soc Promot Heal. 2008;128:31–40.

    Article  Google Scholar 

  41. To QG, Chen TTL, Magnussen CG, To KG. Workplace physical activity interventions: A systematic review. Am J Health Promot 2013; doi:10.4278/ajhp.120425-LIT-222.

  42. Manini TM, Carr LJ, King AC, Marshall S, Robinson TN, Rejeski WJ. Interventions to reduce sedentary behavior. Med Sci Sports Exerc 2015; doi:10.1249/MSS.0000000000000519.

  43. Martin A, Fitzsimons C, Jepson R, Saunders DH, van der Ploeg HP, Teixeira PJ, Gray CM, Mutrie N; Euro FIT consortium. Interventions with potential to reduce sedentary time in adults: systematic review and meta-analysis. Br J Sports Med 2015; doi:10.1136/bjsports-2014-094524.

  44. Schölmerich VLN, Kawachi I. Translating the socio-ecological perspective into multilevel interventions: Gaps between theory and practice. Health Educ Behav 2016; doi:10.1177/1090198115605309.

  45. Richard L, Gauvin L, Raine K. Ecological models revisited: their uses and evolution in health promotion over two decades. Annu Rev Public Health 2011; doi:10.1146/annurev-publhealth-031210-101141.

  46. Golden SD, Earp JAL. Social ecological approaches to individuals and their contexts: Twenty years of Health Education & Behavior health promotion interventions. Health Educ Behav 2012; doi:10.1177/1090198111418634.

  47. Kahn-Marshall JL, Gallant MP. Making healthy behaviors the easy choice for employees: A review of the literature on environmental and policy changes in worksite health promotion. Health Educ Behav 2012; doi:10.1177/1090198111434153.

  48. Official Statistics of Finland (OSF): Finnish enterprises [e-publication]. Helsinki: Statistics Finland [accessed: 15 Sept 2015]. Access method: http://www.stat.fi/til/syr/kas_en.html

  49. Helldan A, Helakorpi S, Virtanen S, Uutela A. Health behaviour and health among the Finnish adult population, spring 2013. National Institute for Health and Welfare, Report 21/2013. https://www.julkari.fi/bitstream/handle/10024/110841/URN_ISBN_978-952-302-051-1.pdf?sequence=1

  50. Chau JY, van der Ploeg HP, Dunn S, Kurko J, Bauman AE. A tool for measuring workers’ sitting time by domain: the Workforce Sitting Questionnaire. Br J Sports Med 2011; doi:10.1136/bjsports-2011-090214.

  51. Vähä-Ypyä H, Vasankari T, Husu P, Suni J, Sievänen H. A universal, accurate intensity-based classification of different physical activities using raw data of accelerometer. Clin Physiol Funct Imaging 2015; doi:10.1111/cpf.12127.

  52. Trost SG, McIver KL, Pate RR. Conducting accelerometer-based activity assessments in field-based research. Med Sci Sport sci Exerc 2005; 37(11 Suppl):S531-S543.

  53. Matthews CE, Hagströmer M, Pober DM, Bowles HR. Best practices for using physical activity monitors in population-based research. Med Sci Sports Exerc 2012; doi:10.1249/MSS.0b013e3182399e5b.

  54. Vähä-Ypyä H, Vasankari T, Husu P, Mänttäri A, Vuorimaa T, Suni J, Sievänen H. Validation of cut-points for evaluating the intensity of physical activity with accelerometry-based mean amplitude deviation (MAD). PLoS One 2015; doi:10.1371/journal.pone.0134813.

  55. Schwarz GE. “Estimating the dimension of a model”, Annals of Statistics 1978; doi:10.1214/aos/1176344136.

  56. Chastin SF, Egerton T, Leask C, Stamatakis E. Meta-analysis of the relationship between breaks in sedentary behavior and cardiometabolic health. Obesity (Silver Spring) 2015; doi:10.1002/oby.21180.

  57. Mansoubi M, Pearson N, Biddle SJ, Clemes SA. Using sit-to-stand workstations in offices: is there a compensation effect? Med Sci Sports Exerc 2016; doi:10.1249/MSS.0000000000000802.

  58. Witt LB, Olsen D, Ablah E. Motivating factors for small and midsized businesses to implement worksite health promotion. Health Promot Pract 2013; doi:10.1177/1524839912472504.

  59. Leykum LK, Pugh JA, Lanham HJ, Harmon J, McDaniel RR. Implementation research design: integrating participatory action research into randomized controlled trials. Implement Sci 2009; doi:10.1186/1748-5908-4-69.

  60. Parry S, Straker L, Gilson ND, Smith AJ. Participatory workplace interventions can reduce sedentary time for office workers – a randomized controlled trial. PLoS One 2013; doi:10.1371/journal.pone.0078957.

  61. Linnan LA, Sorensen G, Colditz G, Klar DN, Emmons KM. Using theory to understand the multiple determinants of low participation in worksite health promotion programs. Health Educ Behav. 2001;28:591–607.

    Article  CAS  PubMed  Google Scholar 

  62. Hall ME, Bergman RJ, Nivens S. Worksite health promotion program participation: A study to examine the determinants of participation. Health Promot Pract 2014; doi:10.1177/1524839913510721.

  63. Glasgow RE, McCaul KD, Fisher KJ. Participation in worksite health promotion: a critique of the literature and recommendations for future practice. Health Educ Q. 1993;20:391–408.

    Article  CAS  PubMed  Google Scholar 

  64. Wilson MG, Basta TB, Bynum BH, DeJoy DM, Vandenberg RJ, Dishman RK. Do intervention fidelity and dose influence outcomes? Results from the Move to Improve worksite physical activity program. Health Educ Res 2010; doi:10.1093/her/cyn065.

  65. Dishman RK, Vandenberg RJ, Motl RW, Wison MG, DeJoy DM. Dose relations between goal setting theory-based correlates of goal setting and increases in physical activity during a workplace trial. Health Educ Journal 2010; doi: 10.1093/her/cyp042.

  66. Bull SS, Gillette C, Glasgow RE, Estabrooks P. Work site health promotion research: to what extent can we generalize the results and what is needed to translate research to practice? Health Educ Behav. 2003;30:537–49.

    Article  PubMed  Google Scholar 

  67. Peters DH, Adam T, Alonge O, Agyepong IA, Tran N. Republished research: Implementation research: what is it and how to do it. Br J Sports Med 2014; doi:10.1136/bmj.f6753.

  68. Robroek SJW, van Lenthe FJ, van Empelen P, Burdorf A. Determinants of participation in worksite health promotion programmes: a systematic review. Int J Behav Nutr Phys Act 2009; doi:10.1186/1479-5868-6-26.

  69. Dishman RK, DeJoy DM, Wilson MG, Vandenberg RJ. Move to Improve. A randomized workplace trial to increase physical activity. Am J Prev Med 2009; doi:10.1016/j.amepre.2008.09.038.

  70. Dunstan DW, Wiesner G, Eakin EG, Neuhaus M, Owen N, LaMontagne AD, Moodie M, Winkler EAH, Fjeldsoe BS, Lawler S, Healy GN. Reducing office workers’ sitting time: rationale and study design for the Stand Up Victoria cluster randomized trial. BMC Public Health 2013; doi:10.1186/1471-2458-13-1057.

  71. Healy GN, Eakin EG, Owen N, Lamontagne AD, Moodie M, Winkler EAH, Fjeldsoe BS, Wiesner G, Willenberg L, Dunstan DW. A cluster randomized controlled trial to reduce office workers’ sitting time: effect on activity outcomes. Med Sci Sports Exerc. 2016;48:1787–97.

    Article  PubMed  Google Scholar 

  72. Neuhaus M, Healy GN, Dunstan DW, Owen N, Eakin EG. Workplace sitting and height-adjustable workstations. A randomized controlled trial. Am J Prev Med 2014; doi:10.1016/j.amepre.2013.09.009.

Download references

Acknowledgements

We thank the 12 workplaces for their participation and dedicated efforts during the study. Our sincere thanks also to the regional contacts Harri Appelroth from Tavastia Regional Sports Federation (HLU), Keijo Kylänpää from Lahti Regional Sports Federation (PHLU), and Teemu Ripatti from Southern Savonia Regional Sports Federation (ESLi) for their valuable field work.

Funding

The study was funded by the Finnish Ministry of Education and Culture, grant numbers 249/625/2013, 835/625/2013, 1472/625/2014. The funder had no role in the design of the study or in the collection, analysis and interpretation of data or in writing the manuscript.

Availability of data and materials

The data generated during this study are not publicly available because availability was not included in the study plan approved by the ethics committee and informed consent was not obtained from the participants for the public availability. However, the data are available from the corresponding author on reasonable request.

Authors’ contributions

MA: Conceived and designed the study and its evaluation, participated in the opening and closing-up meetings, planned data storing and analysis and wrote the manuscript. ML: Conceived and designed the study and its evaluation, participated in the opening and closing-up meetings, commented on the manuscript. SL: Conceived and designed the study and its evaluation, participated in the opening and closing-up meetings, commented on the manuscript. AR: Coordinated the field-actions, provided support for regional contacts, collected data and commented on the manuscript. HV-Y: Interpreted accelerometer data, provided accelerometer feedbacks and commented on the manuscript. KT: Performed the statistical analysis and commented on the manuscript. HS: Participated in the study procedure and commented on the manuscript. AM: Participated in the study procedure and commented on the manuscript. TV: Participated in the study procedure and commented on the manuscript. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

The study plan was approved by the Ethics Committee of the Tampere Region, under the auspices of University of Tampere, Human Sciences (http://www.uta.fi/english/research/ethics/review/committee.html, running number 17/2013). The participants gave their consent to participate in the study by agreeing to complete the measurements (questionnaire, diary, accelerometer) after being fully informed in the covering letters about the ethical principles of voluntary participation, avoiding harm and privacy and data protection. Thus, completing the measurements indicated consent to participate in the study. Written informed consent was not obtained because the study did not include collection of sensitive information or intervene physical integrity of the participants in a way that it would have been necessary.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Minna Aittasalo.

Additional files

Additional file 1:

Assessment of physical activity (PA), sedentary behavior (SB) and possibility to break sitting in the baseline and follow-up questionnaire to the employees. The questions used in the study questionnaire to quantify physical activity and sedentary behavior. (DOCX 115 kb)

Additional file 2:

Statistical analysis to test the baseline differences between the employees, who had completed the questionnaire and the accelerometer measurements only at baseline and who had completed the information also at 1-year follow-up. Statistical analysis. (DOCX 181 kb)

Additional file 3

Statistical analysis to test the difference in change in physical activity and sedentary behavior between the workplaces implementing more or fewer actions than on average and between the workplaces implementing actions at all three levels and just one or two levels. Statistical analysis. (DOCX 105 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aittasalo, M., Livson, M., Lusa, S. et al. Moving to business – changes in physical activity and sedentary behavior after multilevel intervention in small and medium-size workplaces. BMC Public Health 17, 319 (2017). https://doi.org/10.1186/s12889-017-4229-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12889-017-4229-4

Keywords