Skip to main content

Feasibility, acceptability and efficacy of a text message-enhanced clinical exercise rehabilitation intervention for increasing ‘whole-of-day’ activity in people living with and beyond cancer

Abstract

Background

Exercise interventions are typically delivered to people with cancer and survivors via supervised clinical rehabilitation. However, motivating and maintaining activity changes outside of the clinic setting remains challenging. This study investigated the feasibility, acceptability and efficacy of an individually-tailored, text message-enhanced intervention that focused on increasing whole-of-day activity both during and beyond a 4-week, supervised clinical exercise rehabilitation program for people with cancer and survivors.

Methods

Participants (n = 36; mean ± SD age 64.8 ± 9.6 years; 44.1 ± 30.8 months since treatment) were randomized 1:1 to receive the text message-enhanced clinical exercise rehabilitation program, or the standard clinical exercise rehabilitation program alone. Activity was assessed at baseline, 4-weeks (end of the standard program) and 12-weeks (end of enhanced program) using both device (activPAL accelerometer; sitting, standing, light-stepping, moderate-stepping) and self-report [Multimedia Activity Recall for Children and Adults (MARCA); sedentary, light, moderate-to-vigorous physical activity (MVPA)] methods. The MARCA also assessed time use domains to provide context to activity changes. Changes and intervention effects were evaluated using linear mixed models, adjusting for baseline values and potential confounders.

Results

The study had high retention (86%) and participants reported high levels of satisfaction [4.3/5 (±0.8)] with the intervention. Over the first 4 weeks, MARCA-assessed MVPA increased [+ 53.2 (95%CI: 2.9, 103.5) min/d] between groups, favoring the text message-enhanced program, but there were no significant intervention effects on sedentary behavior. By 12 weeks, relative to the standard group, participants in the text message-enhanced group sat less [activPAL overall sitting: − 48.2 (− 89.9, − 5.6) min/16 h awake; MARCA: -80.1 (− 156.5, − 3.8) min/d] and were participating in more physical activity [activPAL light stepping: + 7.0 (0.4, 13.6: min/16 h awake; MARCA MVPA: + 67.3 (24.0, 110.6) min/d]. The time-use domains of Quiet Time [− 63.3 (− 110.5, − 16.0) min/d] and Screen Time [− 62.0 (− 109.7, − 14.2) min/d] differed significantly between groups.

Conclusions

Results demonstrate feasibility, acceptability and efficacy of a novel, text message-enhanced clinical exercise rehabilitation program to support changes in whole-of-day activity, including both physical activity and sedentary behavior. Changes were largely seen at 12-week follow-up, indicating potential for the intervention to result in continued improvement and maintenance of behavior change following a supervised exercise intervention.

Trial registration

This trial is registered with the Australian New Zealand Clinical Trials Registry (ACTRN12616000641493; date registered 17/5/16).

Background

Over one million Australians are now living beyond a diagnosis of cancer and ensuing treatment, with an estimated cost to the health care system of AU$1 million per incident case [1]. In this growing population, physical activity has been shown to improve disease- and treatment-related side effects, reduce the risk of cancer recurrence and common co-morbidities (cardio-metabolic diseases), and improve quality of life [2,3,4,5]. Face-to-face clinical exercise rehabilitation is a common mode of delivery for physical activity intervention among people with cancer and survivors [6]. However, despite short-term (i.e. < 12 weeks) increases in muscle strength, exercise capacity, physical function, flexibility and wellbeing, physical activity outside of the supervised clinic-based setting often does not change [7,8,9,10].

Daily activity levels are comprised of time spent sedentary (i.e. sitting, reclining or lying with low energy expenditure [11]), standing and in other light intensity activities (including incidental movements), and moderate-to-vigorous intensity physical activities (MVPA) [12]. Population estimates of physical activity report that 60% of adults spend < 30 min per day in MVPA [13]. Therefore, for most adults the majority of their waking day is spent in non-MVPA activities. There is now considerable evidence that the distribution of time spent between these non-MVPA activities has a significant impact on health, with time accrued in prolonged, unbroken bouts of sitting potentially particularly harmful (e.g. poor cardio-metabolic health markers and musculoskeletal fatigue and pain) [14,15,16,17]. Evidence to this effect has prompted the inclusion of recommendations for sedentary behavior in physical activity guidelines [18]. Moreover, these data suggest that a whole-of-day approach to physical activity promotion should also be considered, where changes in activities across the intensity spectrum are targeted with the focus placed on the time and the context in which activities are accrued [19, 20]. With this approach, the emphasis has moved away from just increasing time in MVPA to also reallocating time away from sedentary to non-sedentary activities (including light and MVPA intensities). However, this whole-of-day approach is yet to be applied in an exercise rehabilitation setting, or to people with cancer or survivors. Their high levels of sedentary time compared to their non-cancer survivor counterparts [21], and increased risk of comorbid chronic disease [22] mean people with cancer and survivors are particularly likely to benefit from this whole-of-day approach.

mHealth (the use of mobile technology to deliver health-related programs) is well suited to supporting this whole-of-day approach to activity promotion. One highly researched and effective mode of mHealth for physical activity promotion is mobile telephone text messages [23,24,25,26,27]. Text messages can: prompt behaviors in real time across the whole day; efficiently deliver tailored, repeated contacts; be delivered to participants to maintain contact over long periods of time; and, maintain two-way communication with an interventionist using minimal resources. Physical activity interventions have previously been delivered successfully via text messages [26,27,28], including in the context of leveraging off more intensive initial contact [29].

Two previous studies have investigated the efficacy of text messages in cancer survivors for increasing and/or maintaining MVPA [30, 31]. Spark and colleagues used text messaging to target maintenance of MVPA, as well as weight loss and dietary behaviors, following a 6-month telephone delivered weight loss intervention [30], while Gell and colleagues used text messaging in conjunction with health coaching and consumer-based activity trackers to try to enhance MVPA levels [31]. While these studies demonstrated potential for the efficacy of text messaging, both studies were limited by single group, pre-post designs and targeted primarily participation in MVPA, rather than addressing activity across the spectrum. Further, neither study considered the context of the changes in MVPA, focusing on average change in MVPA alone. Including appropriate measurement tools that allow for exploration of when the changes occurred, or the type of behaviors that were being modified by the intervention can increase understanding of intervention effects and can highlight opportunities for further intervention.

Therefore, the aim of this randomized controlled trial was to evaluate the feasibility, acceptability and efficacy of a text message-enhanced clinical exercise rehabilitation intervention using a whole-of-day approach to reduce sitting time and increase activity in people living with and beyond cancer. In addition to the magnitude of change in sedentary behavior and physical activity, the context of these changes were also examined using a time-use approach.

Methods

Study design

This randomized controlled trial evaluated a text message-enhanced clinical exercise rehabilitation program compared to the standard clinical exercise rehabilitation program alone. This trial commenced recruitment in February 2016 and data collection was completed in July 2016. This study received ethical clearance from a Medical Research Ethics Committee at The University of Queensland (protocol number: HMS12/1804).

Participant recruitment

Participants were recruited for this study from the 2016 Clinical Exercise Rehabilitation Program for People with Cancer and Survivors at the School of Human Movement and Nutrition Sciences at The University of Queensland. The target sample size of this study was limited by the capacity of the clinic; the program was able to recruit a maximum of 40 participants (primarily limited by staffing and space requirements). All participants enrolled in the program were invited to participate in the current study; prior to the commencement of the program they were forwarded a Participant Information Sheet and Consent Form via mail or email and then contacted by telephone to establish their eligibility and answer any questions. Eligibility criteria included men and women aged > 18 years of age with a previous histologically-confirmed diagnosis of cancer (excluding childhood cancers), who were at least one-month post-surgery and owned a mobile phone. There were no inclusion criteria with respect to primary cancer diagnosis, or time since diagnosis or treatment. All eligible participants were required to have approval to participate from their medical practitioner as well as provide relevant information (e.g. comorbidities, pre-existing injuries, medications) to assist with exercise testing and prescription. Participants were excluded if they had cardiopulmonary or metabolic disorders that would have prevented safe participation in the testing or exercise sessions. Written informed consent was received from all participants prior to being formally included in the study.

Randomization

Once informed consent was obtained, participants underwent baseline assessments and were then randomized to participate in the text message-enhanced clinical exercise rehabilitation program (enhanced clinic) or the clinical exercise rehabilitation program alone (standard clinic). Randomization was conducted using an online tool (www.randomization.com) and allocation was carried out by a trained researcher with no involvement in the current study. As the clinical exercise rehabilitation program was delivered via four groups per week (approximately 10 people per group), participants were block randomized to ensure equal distribution of standard- and enhanced- clinic participants.

Measures

Device-based and self-report activity outcomes, and time spent in self-reported domains of time use were collected at baseline (prior to commencement of the clinic), 4-weeks (immediately after completion of the standard program) and 12-weeks (at completion of the enhanced program). Prior to baseline testing, participant demographic characteristics were determined from the medical history form provided by participants’ medical practitioners. Variables collected included age (years), gender, primary cancer diagnosis, time since diagnosis (months), time since treatment cessation (months) and the number of current medications.

Feasibility and acceptability outcomes

During the clinical exercise rehabilitation program, the supervising Accredited Exercise Physiologist (AEP) recorded session attendance, adverse (i.e. any untoward medical events occurring during the exercise sessions), and serious adverse (i.e. any adverse events requiring medical attention) events. For the text message-enhanced program, the implementation of each intervention component was also monitored through manual recording by the health professional delivering the session (e.g. duration of face-to-face tailoring sessions), or via the automated software platform used to deliver the text messages (e.g. number of text messages sent and received per participant). Participant’s perceived usefulness and satisfaction with the text messages was evaluated at the 12-week assessment via a 5-point Likert scale (1 = not at all satisfied/useful; 5 = extremely satisfied/useful).

Efficacy outcomes

Device-based activity outcomes

Device-based activity outcomes examined were time spent: sitting, prolonged sitting (sitting for ≥30 min continuously), standing, stepping at a light [< 3 Metabolic Equivalents (METs)] and moderate-to-vigorous (≥3 METs) intensity, as well as usual sitting bout duration (× 50%). Lower values of × 50% are desirable as they indicate a tendency to accrue sitting in shorter rather than longer bouts [32]. These activity outcomes were measured at baseline and at the 4-week and 12-week follow-up using the activPAL micro (PAL Technologies Ltd., Glasgow, United Kingdom). The activPAL micro is a small lightweight triaxial accelerometer that provides accurate measures of sitting, standing, stepping and postural transitions [33,34,35]. The acceleration data records (as summed vector magnitudes over a 15-s time window) can be used to estimate METs with acceptable validity that is consistent with other devices [36].

Details of procedures for fitting the monitor, initialising and downloading the device, as well as data reduction are outlined in detail in Additional file 1: Table S1. The procedures are consistent with common practice in the field [37]. Briefly, the protocol asked participants to wear the device 24 h/d for seven continuous days, secured by trained staff to the right thigh via medical adhesive, while reporting sleep, wake and removals longer than 10 min in a diary. Based on the diary, non-wear time, sleep, and invalid days (< 10 h waking wear time, ≥95% of time spent in any one activity, or < 500 strides) were excluded. Participants with ≥1 day of valid data were included. Time spent in the various activities were all reported standardized to time awake wearing the monitor (min/16-h awake).

Self-report activity outcomes and domains of time use

Sedentary behaviour, MVPA and use of time were measured using the adult version of the Multimedia Activity Recall for Children and Adults (MARCA) at baseline, 4-weeks and 12-weeks. The MARCA is a valid and reliable self-report recall instrument with test-retest intraclass correlation coefficients of 0.99–1.00 for a number of key outcomes (including MVPA, screen time and sleep) [38] and convergent validity of rho = 0.72 for MVPA compared to accelerometry [38] and r = 0.77 for sedentary behaviour compared with activPAL [39]. Participants were asked to recall their previous 24 h from midnight to midnight in time slices as small as 5 min, anchored by meal times throughout the day (breakfast, lunch and dinner) and in the sequence in which they occurred. Administered by computer-assisted telephone interview, the interviewer codes each reported activity by choosing from over 500 discrete activities organized under categories. The activities are then linked to a compendium of energy costs in METs, largely based on the Compendium of Physical Activities [40]. In addition, each activity has a unique code indicating body posture (lying, sitting, standing, in locomotion) [39] and is categorised into one of nine ‘super-domains’ of time use: Sleep, Chores, Cultural, Physical Activity, Quiet Time, Screen Time, Self-Care, Social and Work and Study [19]. At each time point, two separate calls were made 1 week apart, during which participants recalled the two previous days. At each time point, participants therefore recalled 4 days of activity, including at least one weekday and one weekend day. Daily time in the outcomes of interest were then determined by summing the time spent in each activity per day, then averaging across the recalled days using a 5:2 weighting for weekdays:weekend days to capture typical activity patterns [19].

Total sedentary time was determined by calculating the time spent in waking activities with either a sitting or lying body position code. MVPA was determined by calculating the time spent in activities with an intensity of ≥3 METs. Use of time was determined by calculating the time (min/d) spent in each of the nine MARCA super-domains of time use. The most frequently reported activities (minutes) were determined by calculating the total duration of time spent in each activity in the compendium for each intervention group, at each time point.

Interventions

Standard clinic

All study participants completed the standard four-week clinical exercise rehabilitation program designed and delivered specifically for people with cancer and survivors. The program consisted of four, 1-h individualized exercise sessions over 4 weeks, supervised by an AEP, with supplementary home-exercise programs detailing prescribed exercises. The specific exercise prescription and design of the program has been described previously [7], and was found to be safe, feasible and efficacious in improving physical and psychosocial health in people with cancer and survivors.

Enhanced clinic

In addition to the four-week clinical exercise rehabilitation program, participants randomly allocated to the enhanced clinic also received 12-weeks of tailored text messaging. The text messaging was designed to improve whole-of-day activity by targeting: reducing overall sedentary time; breaking up prolonged, unbroken periods (30+ mins) of sedentary time with light intensity activities (including standing); and, increasing MVPA behavior (in addition to the MVPA performed during the clinic sessions). The text messages were individually-tailored for frequency, timing, content and wording. Data to tailor the texts were collected during two sessions (tailoring sessions) prior to, and following completion of, the four-week clinical exercise rehabilitation program.

Baseline tailoring session

The face-to-face tailoring sessions were delivered by a ‘coach’, an allied health professional (Physiotherapist or AEP), using a guided script. Each session took approximately 20–30 min. The coaches received a 2 h, study-specific training session and an accompanying 13-page training manual. The baseline tailoring session served to: introduce participants to a whole-of-day approach to activity; provide brief education on the health benefits of reducing time spent sitting and increasing physical activity; build rapport between the participant and coach; and, gather information to tailor the text message content, frequency and timing for the first 4 weeks of the program. During the session, participants were asked to reflect on their activity patterns to identify periods of prolonged sitting (i.e., sitting danger zones) and opportunities for physical activity. To guide this reflection, the coach showed the participant their sitting and physical activity patterns using heat maps (visual display of 24 h body posture and intensity; see example in Additional file 2: Figure S2) from activPAL data; and, temporograms (visual display of 24 h time use; see example in Additional file 3: Figure S3) from the MARCA completed during the baseline assessment. Participant data collected during this tailoring session [i.e., participant name, coach name, sitting danger zones (× 2), strategies to break sitting danger zones (× 2), MVPA goal, MVPA perceived barriers to goal (× 2), MVPA solutions to barriers (× 2)] were entered in a purpose-designed software platform that enables users to create, send and track personally-tailored text messages (Propelo™, www.propelo.com.au).

Text messages

The physical activity text messages in this program were modified from messages previously shown to be effective at supporting MVPA changes in breast cancer survivors and a general adult population [30, 41]. Further modifications were made to the tone and language of the text messages for this population (i.e. no abbreviations, avoided using the terms ‘slip’ or ‘fall behind’ in relation to behavioral relapses). Specific sedentary behavior text messages were developed, given the behavior change techniques targeting MVPA are not always applicable to sedentary behavior [42]. The pervasive and habitual nature of sedentary behavior means that it is well suited to real-time prompting to raise contextual awareness and prompt action. Education on the potential health consequences of sitting and how patterns of sitting time are accrued was considered a key element. Therefore, the text messages in this trial targeted: 1) education (for sedentary behavior and MVPA); 2) real-time behavioral prompting (for sedentary behavior and MVPA); and, 3) goal checking (for MVPA; See Table 1 for examples).

Table 1 Type and frequency of text messages sent to the enhanced clinic group

The frequency of the text messages sent varied among participants depending on personal preference, with a minimum of six text messages per fortnight for the duration of the 12-week intervention. Text messages included a minimum of two educational tips, three real-time prompts and one goal check text per fortnight (see Table 1). All text messages were signed off using the first name of the coach who delivered the baseline tailoring session. Text messages were generated and sent by research staff using the Propelo™ platform, which enabled messages to be tailored and pre-programmed in advance and scheduled to be sent at specific times to encourage real-time behavioral prompting (e.g. Wednesday afternoon at 3:05 pm).

Replies to the goal check text were stored and a tailored response was automatically triggered if the participant replied with the words ‘yes’ or ‘no’ (or accepted variations of these, e.g. yep). If participants replied to a goal check text with additional words (e.g. providing justification as to why they did not reach their weekly goal) or a word not identified as a variant of ‘yes’ or ‘no’, then the program sent an email to research staff. Research staff then manually selected which tailored goal check reply (met goal or did not meet goal) was to be sent to the participant. If required, the research staff contacted the participant’s coach to provide tailored advice depending on what information the participant supplied in their text.

Follow-up tailoring session

At the completion of the four-week supervised clinical rehabilitation program, participants received a second follow-up, face-to-face tailoring session to update their preferences for text message frequency, timing and content for the remaining 8 weeks of the program. The session was led by the same coach as the baseline tailoring session. The follow-up tailoring session was designed to last approximately 20 min and was guided by a script.

Statistical analysis

Statistical analyses were performed in STATA version 14 (StataCorp LLC [US]) with significance set to p < 0.05, two tailed. Participant characteristics, process outcomes and adverse outcomes were reported using descriptive statistics. Mixed models were used to examine change in activity, with outcomes examined as change scores (follow-up- baseline). Models included effects for time (4-weeks/12-weeks), group (clinic/enhanced clinic), group x time interaction (12-weeks-4 weeks), baseline values of the outcome variable (to control regression to the mean) [43] and other potential confounders: age (years); gender (male/female); time since diagnosis (months); and, number of medications currently taken (0 / 1 / ≥2). From the model, we report marginal means, and comparisons of marginal means. Consistent with intention to treat principles, participants were analysed as randomized. Analyses were of evaluable cases, with all participants (100%) followed up at 4 weeks and 86% (n = 31) followed up at 12 weeks.

The primary aim of this study was to assess the feasibility and acceptability of the protocol, and to provide estimates of the effect on daily sitting time (16 h waking day) as the primary outcome. The sample size of the study was limited by the capacity of the exercise rehabilitation program (40 participants). However, allowing for an 80% uptake and 10% drop out [7] (n = 28; n = 14 per group), the study was adequately powered to detect a medium effect (Cohen’s d = 0.5) between groups, with 80% power and 5% alpha.

Results

Participants

Thirty-eight participants were recruited (Fig. 1), with 36 completing baseline testing and being randomly allocated to either the standard or enhanced clinic groups (n = 18, respectively). One participant was excluded due to eligibility criteria (they did not own a mobile phone) and the other was unable to complete baseline testing due to medical complications. Participant demographics are described in Table 2. Participants were, on average, approximately 65 years old, slightly overweight, mostly female and taking two or more medications. Most participants had a primary diagnosis of colorectal cancer. Four (10%) participants were currently undergoing treatment, with an average time since treatment of four years over the whole sample.

Fig. 1
figure 1

Participant flow diagram

Table 2 Participant baseline characteristics

All participants were retained at 4-week follow-up, while five participants withdrew (n = 3 standard clinic; n = 2 enhanced clinic) between 4- and 12-weeks, namely due to lack of time, family bereavement, adverse reaction to activPAL dressings and overseas travel (Fig. 1). Missing data at 12 weeks did not differ significantly (p = 0.603) between the enhanced clinic (6%; n = 1) and standard clinic (17%; n = 3) groups.

Feasibility and acceptability

Thirty-one (out of 36) participants (86%) attended all four supervised exercise sessions. Overall, out of 144 total sessions, attendance was 96% (138/144). One adverse event was recorded; a participant overbalanced and fell during a lunging exercise. No injury occurred and no follow-up treatment was required. All 18 participants in the enhanced clinic group completed both the baseline and follow-up tailoring sessions, and received text messages during the first 4 weeks of the program (whilst attending the clinical exercise rehabilitation program); however, four participants opted out of receiving text messages for the remaining 8 weeks. Reasons for opting out of the texts after the first 4 weeks included: travelling overseas during the 8-week follow-up period (n = 1); sufficiently self-motivated to continue without texts (n = 1); not finding the texts useful (n = 1); and, not liking the directive language in the texts (n = 1). On average, participants opted to receive eight text messages per fortnight (range = 6–12 text messages/fortnight, possible range: 6–17 per fortnight). During the baseline tailoring session, 13/18 opted to receive the minimum of two prompts per fortnight for sedentary behavior and 12/18 opted for the minimum of one prompt per fortnight for the MVPA. The average reply rate to the fortnightly MVPA goal checks was 78%, with 8/18 participants replying to all of the goal checks they received. Of the 83 goal check replies received from participants, eight (9.6%) were screened by the coach for appropriateness of content before sending a reply.

During the initial tailoring session, each participant nominated two ‘sitting danger zones’ to target. The most commonly targeted zones identified were: watching television (12/36 danger zones); computer work (11/36 danger zones); and, reading (8/36 danger zones). Each participant also set an MVPA goal during the initial tailoring session (which may have included more than one type of activity). Here, the most commonly nominated activities were brisk walking (12/18 goals); cycling (5/18 goals); and, going to the gym (4/18 goals).

Seventeen participants in the enhanced clinic completed the satisfaction survey; results are shown in Table 3. Mean (±SD) satisfaction scores were high overall (4.3 ± 0.8) and for both the tailoring sessions and the tailored text messages. Most participants read the text messages, had no difficulty understanding them, and approximately half found the program elements extremely helpful in meeting their physical activity and sitting goals. Of the 14 participants who responded to a request for program suggestions: nine commended the program without raising any suggestion or complaint; two indicated that the texts had not succeeded in prompting a sense of motivation (sometimes prompting guilt); one participant indicated the texts content was incongruent with what was happening when it was received; one participant wanted more texts that required a response; and, one participant suggested access to online exercise prescription.

Table 3 Satisfaction and usefulness of the enhanced clinic intervention (n = 17)

Efficacy

Device-based activity outcomes

Changes in activPAL-assessed activity from baseline to 4-weeks, baseline to 12-weeks and 4-weeks to 12-weeks for both the standard and enhanced clinic groups are presented in Table 4. Over the first 4 weeks, none of the objectively-measured activities outside the clinic changed over time or differed between groups to a large or statistically significant degree. However, at 12-week follow-up, the enhanced clinic group reduced their overall sitting and prolonged sitting time by 40–50 min/16 h awake, with corresponding increases in standing and light-intensity stepping. The standard clinic group made no large or significant changes in objectively measured activities at 12 weeks or between 4- and 12-weeks. Differences between groups reached statistical significance for total sitting and light-intensity stepping, with non-significant tendencies towards better outcomes in the enhanced clinic group than the standard group in prolonged sitting, standing and sitting bout duration. The size of the differences between the 4-week and 12-week assessments indicated that most of the improvements in the enhanced clinic group occurred over this timeframe, with significant improvements seen for overall and prolonged sitting time, standing time, and light-intensity stepping. None of the changes or group differences in activPAL-assessed MVPA observed were statistically significant or large; however, some amounted to half an hour per week or more and confidence intervals could not rule out effects that were of a meaningful magnitude consistent with 1-h or more per week.

Table 4 Changes in out-of-clinic activity (activPAL; min/16 h awake) in the standard clinic (C) and text message-enhanced clinic (EC)

Self-report activity outcomes

Changes in activity according to the MARCA from baseline to 4-weeks, baseline to 12-weeks and from 4- to 12-weeks are presented for both standard and enhanced clinic groups in Table 5. In contrast to the activPAL results, over the first 4 weeks, there were significant differences between groups, with the enhanced clinic group reporting less sleep, more vigorous physical activity, and more MVPA than the standard clinic group. Other outcomes did not differ significantly between the groups. These between-group differences were driven by the standard clinic having significantly changed their reported daily sleep, MVPA and moderate activity while the enhanced clinic group made no significant changes in these reported behaviors and experienced a significant increase in self-report vigorous physical activity.

Table 5 Changes in out-of-clinic time use by intensity (MARCA; min/day) in the standard clinic (C) and text message-enhanced clinic (EC)

At the 12-week follow-up, the enhanced clinic group reported significantly less sleep and more vigorous and MVPA, than the standard clinic group. These effects occurred due to decreased MVPA and moderate physical activity by 50–53 min/day in the standard clinic group, while the enhanced clinic group decreased their reported sitting time by approximately 1 hour per day and increased their vigorous activity and sleep by 20 and 40 min per day, respectively.

Self-report time use domain outcomes

Changes in time use domains according to the MARCA from baseline to 4-weeks, baseline to 12-weeks and from 4- to 12-week follow-up for both the standard and enhanced clinic groups are presented in Table 6. There were no significant differences between the groups in the change in time spent in the use of time domains over the first 4 weeks of the intervention. Both groups either significantly increased time spent in Physical Activity or tended to do so (+ 8–9 min/day). The standard-clinic group significantly decreased their Social time by 40 min/day, whilst there was no significant change in the enhanced clinic group. Difference between the groups were apparent at 12 weeks, however, with the enhanced clinic group reporting significantly lower (by approximately 1 h/day) Screen Time and Quiet Time than the standard-clinic group at 12-weeks. Both groups, to a similar degree, significantly reduced their time spent in cultural activities (− 5 min/day) and increased their time spent in Physical Activity (+ 15–17 min/day). Other differences between groups and changes were not statistically significant, though some were of a meaningful magnitude. For example, the enhanced clinic group spent approximately half an hour per day more in Work and Study than the standard-clinic group at 12 weeks.

Table 6 Changes in out-of-clinic time use by domains (MARCA; min/day) in the standard clinic (C) and the text message-enhanced clinic (EC)

Change in self-reported activities according to the MARCA from baseline to 12-week follow up are presented in Fig. 2 for the enhanced clinic group (baseline to 4-weeks presented in Additional file 4: Figure S4). Consistent with intervention targets, participants reported more time spent walking and in gym-related activities as well as reductions in computer work, watching TV and reading.

Fig. 2
figure 2

Baseline to 12-week follow-up changes in most frequently reported sedentary behavior and moderate-to-vigorous physical activity activities from the MARCA by duration (min/week) in the text message-enhanced clinic. Note: Activities indicated by an asterisk are activities that are reported for the first time at follow-up and those shaded in black are activities that map directly onto commonly chosen intervention targets. MARCA = Multimedia Activity Recall for Children and Adults

Discussion

This study explored the feasibility, acceptability and efficacy of a text message-enhanced clinical exercise rehabilitation program for reducing sitting and increasing activity in people living with and beyond cancer. The major findings of this study were that the enhanced intervention was feasible to deliver, participants reported it to be acceptable and that it resulted in significant reductions in sitting (approximately 1 h/day) and increases in MVPA (approximately 1 h/day) compared to the standard clinic group. This was predominantly due to activity changes after completion of the supervised exercise phase. This study also demonstrated that the specific targets chosen by participants (i.e. their sitting danger zones and MVPA goals), were positively impacted by the enhanced intervention as shown in the MARCA time use domains data.

The changes in MVPA outcomes in the current trial exceed previously reported changes using other digital health interventions in cancer survivors. A recent meta-analysis found that the average increase in physical activity among cancer survivors following a digital health intervention (e.g. websites, mobile applications or text messaging) was 41 min/week (95% CI 12, 71 min) [44]. While there are no systematic reviews of sedentary behavior interventions in cancer survivors, a meta-analysis of general sedentary behavior interventions in otherwise healthy populations estimated a significant reduction in sitting time of 42 min/day (95%CI -78.92, − 4.60) favoring the intervention, with no significant effect for combined physical activity and sedentary behavior interventions for reducing sedentary behavior [45]. Therefore, findings from the current study demonstrate novel evidence for the efficacy of interventions employing a whole-of-day approach, and further, that simultaneous changes in both sitting and physical activity are feasible, and highly acceptable to participants.

The largest changes in both sedentary behavior and physical activity were observed at 12-weeks. This suggests the potential of the enhanced clinic approach to leverage off existing face-to-face, supervised exercise interventions to support participants’ transition to unsupervised, self-directed behaviors. Importantly, these favorable changes in activity were achieved using an intervention with a demonstrated capacity to be delivered at scale [41]. Scalability is important given the growing number of cancer survivors [1] and the challenge of maintaining behavior change following clinical rehabilitation [46]. Nonetheless, while the current study demonstrated maintenance of behavior change with the text message-enhanced intervention beyond the supervised phase, future studies should assess whether changes are maintained once the text message component has also been withdrawn.

A novel feature of this study, compared to other trials in this field, was the use of the MARCA. The MARCA, which was well received among participants, is a high-resolution, 24 h recall tool that collects activity data in 5 min increments. In the context of this study, the MARCA was used as an intervention resource to assist participants in identifying sitting danger zones and for guiding MVPA goal setting. In addition, the MARCA was used as an outcome measure, allowing an in-depth understanding of the context of the changes in sedentary behavior and physical activity that would have otherwise been missed with device- or self-report aggregate-based measures alone. For example, balance and resistance training are commonly prescribed for people with cancer and survivors; however, these activities are not well captured by device-based measures that may misclassify balance and/or resistance training exercises as light or even sedentary time. Indeed, gym based resistance training exercises were commonly reported using the MARCA at follow up. In this study, MARCA data also enabled a deeper examination and understanding of the feasibility of the text message-enhanced intervention. Mapping of the participants’ strategies to break up sedentary behavior danger zones and tailored physical activity goals with their time use results demonstrated close alignment, with several activities mapping directly onto the most commonly reported sedentary behavior danger zones and MVPA goals. These findings provide further evidence of the success of the enhanced intervention.

The current study has a number of strengths. This is the first study to evaluate a text message-enhanced intervention in people with cancer and survivors that uses a whole-of-day approach, both during and following a supervised clinical exercise rehabilitation program. In combination with self-report methods, it used a gold standard, device-based method for assessing activity; the activPAL monitor. Further, employing a randomized controlled trial design allowed comparison to standard care, and incorporating rigorous assessment of feasibility and acceptability enabled a deeper understanding of the potential scalability of the intervention. Finally, despite the average age of almost 70 years, there were high levels of uptake and engagement in the text message-enhanced intervention, with all eligible participants opting into the intervention. Only one participant was excluded as they did not own a mobile phone, and there was a high reply rate to text messages that invited a response (78%), thus demonstrating that technology-based interventions are feasible and acceptable for this population.

There are several limitations of this study. The relatively small sample size provides only preliminary evidence of the efficacy of the enhanced intervention. A larger study, with statistical power to detect changes in both sitting time and physical activity, is required to confirm the effectiveness of the intervention. While powered for medium effect sizes (considered in terms of d = 0.5), confidence intervals around non-significant effects sometimes failed to rule out potentially large differences between groups of an hour or more. Despite the high levels of feasibility and acceptability, four participants (22%) opted out of receiving text messages at the end of the supervised phase (i.e. 4 weeks), indicating that it did not suit all participants. Reasons provided for opting out included: being sufficiently internally motivated to maintain their goals; not finding the text messages useful; and not liking the directive tone of the text messages. In future iterations, participants should be given the flexibility to opt in and out of extended contact interventions as they develop confidence to self-regulate their behavior. Further, the tone of the text message language should also be tailored based on participant preference, where possible. Patients and survivors of colorectal, breast and prostate cancer volunteered to participate in this study. Whilst these are the three most commonly diagnosed cancers in Australia [47], and thus representative of most Australian exercise oncology interventions, whether the results may have differed in other oncological populations is unknown. There were large changes in computer work, however the MARCA does not collect information pertaining to the purpose of the computer work (e.g. social or for paid or unpaid work). Whether there may have been any unintended consequences on social interaction or work productivity from the study should be explored in future iterations. Finally, at baseline, participants who volunteered for this study had high levels of sitting time (> 8 h per day), but on average were already meeting MVPA guidelines [48] and findings may differ in a less active population.

Conclusions

In summary, the findings from this study have demonstrated the feasibility, acceptability and efficacy of a text message-enhanced clinical exercise rehabilitation program with a whole-of-day approach to activity behavior change. This novel approach has the potential to extend clinical contact beyond a supervised exercise rehabilitation program, and at scale, while focusing on a broader range of activities and maintaining contact for a longer period of time. These findings may have great impact on future approaches to clinical practice to support people living with and beyond cancer to improve and maintain their health.

Abbreviations

AEP:

Accredited Exercise Physiologist

CI:

Confidence interval

MARCA:

Multimedia Activity Recall for Children and Adults

METs:

Metabolic equivalents

MVPA:

Moderate-to-vigorous physical activity

TV:

Television

References

  1. Access Economics Pty Ltd CCN (2007). Cost of cancer in NSW 2016. http://www.cancercouncil.com.au/wp-content/uploads/2010/11/costofcancer_summary.pdf. Accessed 3 Mar 2016.

    Google Scholar 

  2. Mishra S, Scherer R, Snyder C, Geigle P, Gotay C. Are exercise programs effective for improving health-related quality of life among cancer survivors? A systematic review and meta-analysis. Oncol Nurs Forum. 2014;41:E326–42.

    Article  Google Scholar 

  3. Meneses-Echavez J, Gonzalez-Jimenez E, Correa-Bautista J, Valle J, Ramirez-Velez R. Effectiveness of physical exercise on fatigue in cancer patients during active treatment: a systematic review and meta-analysis. Cad Saude Publica. 2015;31:667–81.

    Article  Google Scholar 

  4. Sturgeon K, Ky B, Libonati J, Schmitz K. The effects of exercise on cardiovascular outcomes before, during, and after treatment for breast cancer. Breast Cancer Res Treat. 2014;143:219–26.

    Article  CAS  Google Scholar 

  5. Meyerhardt JA, Giovannucci EL, Holmes MD, Chan AT, Chan JA, Colditz GA, Fuchs CS. Physical activity and survival after colorectal cancer diagnosis. J Clin Oncol. 2006;24:3527–34.

    Article  Google Scholar 

  6. Cheema BS, Kilbreath SL, Fahey PP, Delaney GP, Atlantis E. Safety and efficacy of progressive resistance training in breast cancer: a systematic review and meta-analysis. Breast Cancer Res Treat. 2014;148:249–68.

    Article  CAS  Google Scholar 

  7. Skinner T, Peeters GMME, Croci I, Bell KR, Burton NW, Chambers SK, Bolam KA. Impact of a brief exercise program on the physical and psychosocial health of prostate cancer survivors: a pilot study. Asia-Pacific Journal of Clinical Oncol. 2016;12:225–34.

    Article  Google Scholar 

  8. Ayabe M, Brubaker PH, Dobrosielski D, Miller HS, Ishi K, Yahiro T, Kiyonaga A, Shindo M, Tanaka H. The physical activity patterns of cardiac rehabilitation program participants. J Cardpulm Rehabil. 2004;24:80–6.

    Article  Google Scholar 

  9. Egan C, Deering BM, Blake C, Fullen BM, McCormack NM, Spruit MA, Costello RW. Short term and long term effects of pulmonary rehabilitation on physical activity in COPD. Respir Med. 2012;106:1671–9.

    Article  Google Scholar 

  10. Jones NL, Schneider PL, Kaminsky LA, Riggin K, Taylor AM. An assessment of the total amount of physical activity of patients participating in a phase III cardiac rehabilitation program. J Cardiopulm Rehabil Prev. 2007;27:81–5.

    Article  Google Scholar 

  11. Tremblay MS, Aubert S, Barnes JD, Saunders TJ, Carson V, Latimer-Cheung AE, Chastin SFM, Altenburg TM, Chinapaw MJM. Sedentary behavior research network (SBRN) – terminology consensus project process and outcome. Int J Behav Nutr Phys Act. 2017;14:75.

    Article  Google Scholar 

  12. Chastin SFM, Palarea-Albaladejo J, Dontje ML, Skelton DA. Combined effects of time spent in physical activity, sedentary behaviors and sleep on obesity and cardio-metabolic health markers: a novel compositional data analysis approach. PLoS One. 2015;10:e0139984.

    Article  Google Scholar 

  13. Australian Bureau of Statistics: 4364.0.55.004 - Australian Health Survey: Physical Activity, 2011–12. Canberra, Australia. 2013. http://www.abs.gov.au/ausstats/abs@.nsf/Lookup/4364.0.55.004Chapter4002011-12. Accessed 6 July 2018.

  14. Healy G, Winkler EAH, Owen N, Anuradha S, Dunstan DW. Replacing sitting time with standing or stepping: associations with cardio-metabolic markers. Eur Heart J. 2015;36:2643–9.

    Article  Google Scholar 

  15. Thorp A, Kingwell BA, Owen N, Dunstan DW. Breaking up workplace sitting time with intermittent standing bouts improves fatigue and musculoskeletal discomfort in overweight/obese office workers. Occup Environ Med. 2014;71:765–71.

    Article  Google Scholar 

  16. Larsen R, Kingwell BA, Robinson C, Hammond L, Cerin E, Shaw JE, Healy GN, Hamilton MT, Owen N, Dunstan DW. Breaking up of prolonged sitting over three days sustains, but does not enhance, lowering of postprandial plasma glucose and insulin in overweight and obese adults. Clin Sci. 2015;129:117–27.

    Article  CAS  Google Scholar 

  17. Larsen R, Kingwell BA, Sethi P, Cerin E, Owen N, Dunstan DW. Breaking up prolonged sitting reduces resting blood pressure in overweight/obese adults. Nutr Metab Cardiovasc Dis. 2014;24:976–82.

    Article  CAS  Google Scholar 

  18. Australian Government Department of Health. Australia's physical activity and sedentary behaviour guidelines. Canberra: Australian Government Department of Health; 2014. http://www.health.gov.au/internet/main/publishing.nsf/Content/health-pubhlth-strateg-phys-act-guidelines/$File/FS-Adults-18-64-Years.PDF. Accessed 10 June 2018

    Google Scholar 

  19. Gomersall S, Norton K, Maher C, English C, Olds T. In search of lost time: when people undertake a new exercise program, where does the time come from? A randomized controlled trial. J Sci Med Sport. 2015;18:43–8.

    Article  Google Scholar 

  20. Gomersall S, Maher C, English C, Rowlands A, Olds T. Time regained: when people stop a physical activity program, how does their time use change? A randomised controlled trial. PLoS One. 2015;10:e0126665.

    Article  Google Scholar 

  21. Kim RB, Phillips A, Herrick K, Helou M, Rafie C, Anscher MS, Mikkelsen RB, Ning Y. Physical activity and sedentary behavior of Cancer survivors and non-Cancer individuals: results from a National Survey. PLoS One. 2013;8:e57598.

    Article  CAS  Google Scholar 

  22. Lynch B, Dunstan D, Vallance J, Owen N. Don't take cancer sitting down: a new surivorship research agenda. Cancer. 2013;119:1928–35.

    Article  Google Scholar 

  23. Fjeldsoe B, Marshall A, Miller Y. Behaviour change interventions delivered by mobile telephone short-messaging service. Am J Prev Med. 2009;36:165–73.

    Article  Google Scholar 

  24. Cole-Lewis H, Kershaw T. Text messaging as a tool for behaviour change in disease prevention and management. Epidemiol Rev. 2010;32:56–69.

    Article  Google Scholar 

  25. Fanning J, Mullen S, McAuley E. Increasing physical activity with mobile devices: a meta-analysis. J Med Internet Res. 2012;14:e161.

    Article  Google Scholar 

  26. Buchholz S, Wilbur J, Ingram D, Fogg L. Physical activity text messaging interventions in adults: a systematic review. Worldviews Evid Based Nurs. 2013;10:163–73.

    Article  Google Scholar 

  27. Armanasco AA, Miller YD, Fjeldsoe BS, Marshall AL. Adults' sedentary behaviour: determinants and interventions. Am J Prev Med. 2011;52:391–402.

  28. Hall AK, Cole-Lewis H, Bernhardt JM. Mobile text messaging for health: a systematic review of reviews. Annu Rev Public Health. 2015;36:393–415.

    Article  Google Scholar 

  29. Job JR, Fjeldsoe BS, Eakin EG, Reeves MM. Effectiveness of extended contact interventions for weight management delivered via text messaging: a systematic review and meta-analysis. Obes Rev. 2018;19:538–49.

    Article  CAS  Google Scholar 

  30. Spark L, Fjeldsoe B, Eakin E, Reeves M. Efficacy of a text message-delivered extended contact intervention on maintenance of weight loss, physical activity and dietary behaviour change. JMIR mHealth uHealth. 2015;2015:3.

    Google Scholar 

  31. Gell NM, Grover KW, Humble M, Sexton M, Dittus K. Efficacy, feasibility, and acceptability of a novel technology-based intervention to support physical activity in cancer survivors. Support Care Cancer. 2017;25:1291–300.

    Article  Google Scholar 

  32. Chastin SFM, Winkler EAH, Eakin EG, Gardiner PA, Dunstan DW, Owen N, Healy GN. Sensitivity to change of objectively-derived measures of sedentary behavior. Meas Phys Educ Exerc Sci. 2015;19:138–47.

    Article  Google Scholar 

  33. Grant PM, Ryan CG, Tigbe WW, Granat MH. The validation of a novel activity monitor in the measurement of posture and motion during everyday activities. Br J Sports Med. 2006;40:992–7.

    Article  CAS  Google Scholar 

  34. Lyden K, Kozey-Keadle SL, Staudenmayer JW, Freedson PS. Validity of two wearable monitors to estimate breaks from sedentary time. Med Sci Sports Exerc. 2012;44:2243–52.

    Article  Google Scholar 

  35. Ryan CG, Grant PM, Tigbe WW, Granat MH. The validity and reliability of a novel activity monitor as a measure of walking. Br J Sports Med. 2006;40:779–84.

    Article  CAS  Google Scholar 

  36. Powell C, Carson BP, Dowd KP, Donnelly AE. Simultaneous validation of five activity monitors for use in adult populations. Scand J Med Sci Sports. 2017;27:1881–92.

    Article  CAS  Google Scholar 

  37. Edwardson CL, Winkler EAH, Bodicoat DH, Yates T, Davies MJ, Dunstan DW, Healy GN. Considerations when using the activPAL monitor in field-based research with adult populations. J Sport Health Sci. 2017;6:162–78.

    Article  Google Scholar 

  38. Gomersall SR, Olds TS, Ridley K. Development and evaluation of an adult use-of-time instrument with an energy expenditure focus. J Sci Med Sport. 2011;14:143–8.

    Article  Google Scholar 

  39. Gomersall S, Pavey TG, Clark BK, Jasman A, Brown W. Validity of a self-report recall tool for estimating sedentary behaviour in adults. J Phys Act Health. 2015;12:1485–91.

    Article  Google Scholar 

  40. Ainsworth BE, Haskell WL, Herrmann SD, Meckes N, Bassett DRJ, Tudor-Locke C, Greer JL, Vezina J, Whitt-Glover MC, Leon AS. Compendium of physical activities: a second update of codes and MET values. Med Sci Sports Exerc. 2011;43:1575–81.

    Article  Google Scholar 

  41. Fjeldsoe SB, Goode DA, Phongsavan P, Bauman A, Maher G, Winkler E, Eakin GE. Evaluating the maintenance of lifestyle changes in a randomized controlled trial of the ?Get healthy, stay healthy? Program. JMIR mHealth uHealth. 2016;4:e42.

    Article  Google Scholar 

  42. Owen N, Sugiyama T, Eakin EE, Gardiner PA, Tremblay MS, Sallis JF. Adults’ sedentary behavior. Am J Prev Med. 2017;41:189–96.

    Article  Google Scholar 

  43. Barnett AG, van der Pols JC, Dobson AJ. Regression to the mean: what it is and how to deal with it. Int J Epidemiol. 2005;34:215–20.

    Article  Google Scholar 

  44. Roberts AL, Fisher A, Smith L, Heinrich M, Potts HWW. Digital health behaviour change interventions targeting physical activity and diet in cancer survivors: a systematic review and meta-analysis. J Cancer Surviv. 2017;11:704–19.

    Article  Google Scholar 

  45. Martin A, Fitzsimons C, Jepson R, Saunders DH, van der Ploeg HP, Teixeira PJ, Gray CM, Mutrie N. Interventions with potential to reduce sedentary time in adults: systematic review and meta-analysis. Br J Sports Med. 2015;49:1056.

    Article  Google Scholar 

  46. Spark LC, Reeves MM, Fjeldsoe BS, Eakin EG. Physical activity and/or dietary interventions in breast cancer survivors: a systematic review of the maintenance of outcomes. J Cancer Surviv. 2013;7:74–82.

    Article  Google Scholar 

  47. Australian Institute of Health and Welfare. Cancer in Australia 2017. Vol. cat. No. CAN 100, cat. No. CAN 100 edition. Canberra: Australian Institute of Health and Welfare; 2017.

    Google Scholar 

  48. Buffart LM, Galvão DA, Brug J, Chinapaw MJM, Newton RU. Evidence-based physical activity guidelines for cancer survivors: current guidelines, knowledge gaps and future research directions. Cancer Treat Rev. 2014;40:327–40.

    Article  CAS  Google Scholar 

Download references

Acknowledgments

The authors wish to thank the participants for their time and commitment to this study. The authors also wish to thank Professor Tim Olds for permission to use the MARCA temporogram software in this study.

Funding

This study was funded by a Faculty of Health and Behavioural Sciences and Faculty of Medicine Collaborative Grant from The University of Queensland. Dr. Winkler is supported by a NHMRC Centre for Research Excellence Grant on Sitting Time and Chronic Disease Prevention – Measurement, Mechanisms and Interventions [grant number 1057608], on which Prof Eakin and A/Prof Healy are chief investigators. A/Prof Healy is supported by a National Health and Medical Research Council: NHMRC Career Development Fellowship [grant number 108029]. Publication of this article was jointly sponsored by the Faculty of Health and Behavioural Sciences and Faculty of Medicine Collaborative Grant and the School of Health and Rehabilitation Sciences, The University of Queensland.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

About this supplement

This article has been published as part of BMC Public Health Volume 19 Supplement 2, 2019: Application of time use methods to physical activity and behavioural nutrition research. The full contents of the supplement are available online at https://bmcpublichealth.biomedcentral.com/articles/supplements/volume-19-supplement-2

Author information

Authors and Affiliations

Authors

Contributions

SG, TS, BF, GH AND EE contributed to the conceptualisation and design of the study. SG contributed to the acquisition of the data. EW produced the plots used in the intervention and performed the statistical analysis. All authors contributed to interpretation of the results and preparing the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sjaan R. Gomersall.

Ethics declarations

Ethics approval and consent to participate

This study was granted ethical approval from the Medical Research Ethics Committee at The University of Queensland (Ethical approval number: HMS12/1804). Written informed consent was received from all participants prior to being formally included in the study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Table S1. activPAL protocol. Described activPAL protocol in detail. (DOCX 17 kb)

Additional file 2:

Figure S2. Heatmap. Example of a heatmap used during coaching sessions derived from activPAL data. (TIFF 1142 kb)

Additional file 3:

Figure S3. Temporogram used during coaching sessions derived from MARCA data. Example of a temporogram used during coaching sessions derived from MARCA data. (TIFF 1142 kb)

Additional file 4:

Figure S4. Baseline to 4-week follow-up changes in most frequently reported sedentary behavior and moderate-to-vigorous physical activity activities from the MARCA by duration (min/week) in the text message-enhanced clinic. Note: MARCA = Multimedia Activity Recall for Children and Adults. Graph detailing changes in time (min/day) spent in sedentary behaviour and physical activity from baseline to 4-week follow up in the text-message-enhanced clinic. (TIF 1799 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gomersall, S.R., Skinner, T.L., Winkler, E. et al. Feasibility, acceptability and efficacy of a text message-enhanced clinical exercise rehabilitation intervention for increasing ‘whole-of-day’ activity in people living with and beyond cancer. BMC Public Health 19 (Suppl 2), 542 (2019). https://doi.org/10.1186/s12889-019-6767-4

Download citation

  • Published:

  • DOI: https://doi.org/10.1186/s12889-019-6767-4

Keywords