Do physical activity and dietary smartphone applications incorporate evidence-based behaviour change techniques?

Abstract

Background

There has been a recent proliferation in the development of smartphone applications (apps) aimed at modifying various health behaviours. While interventions that incorporate behaviour change techniques (BCTs) have been associated with greater effectiveness, it is not clear to what extent smartphone apps incorporate such techniques. The purpose of this study was to investigate the presence of BCTs in physical activity and dietary apps and determine how reliably the taxonomy checklist can be used to identify BCTs in smartphone apps.

Methods

The top-20 paid and top-20 free physical activity and/or dietary behaviour apps from the New Zealand Apple App Store Health & Fitness category were downloaded to an iPhone. Four independent raters user-tested and coded each app for the presence/absence of BCTs using the taxonomy of behaviour change techniques (26 BCTs in total). The number of BCTs included in the 40 apps was calculated. Krippendorff’s alpha was used to evaluate interrater reliability for each of the 26 BCTs.

Results

Apps included an average of 8.1 (range 2-18) techniques, the number being slightly higher for paid (M = 9.7, range 2-18) than free apps (M = 6.6, range 3-14). The most frequently included BCTs were “provide instruction” (83% of the apps), “set graded tasks” (70%), and “prompt self-monitoring” (60%). Techniques such as “teach to use prompts/cues”, “agree on behavioural contract”, “relapse prevention” and “time management” were not present in the apps reviewed. Interrater reliability coefficients ranged from 0.1 to 0.9 (Mean 0.6, SD = 0.2).

Conclusions

Presence of BCTs varied by app type and price; however, BCTs associated with increased intervention effectiveness were in general more common in paid apps. The taxonomy checklist can be used by independent raters to reliably identify BCTs in physical activity and dietary behaviour smartphone apps.

Background

Lifestyle behaviours, such as diet and physical activity, are modifiable risk factors associated with many non-communicable diseases (NCDs), which account for 63% of deaths worldwide [1]. To date, many intervention programs targeting physical activity and dietary changes have had modest effects and their long-term effectiveness is not well established [2–5]. Thus, public health researchers have begun to examine novel approaches to deliver behaviour change interventions. Mobile and wireless technology (mHealth) is a growing area in the prevention and management of NCDs and holds potential to deliver health-related behaviour change interventions [6–8]. Mobile phone ownership has reached saturation in many developed countries with an increase in smartphone ownership. A 2012 survey in the United States (U.S.) of three thousand adults indicated that 85% owned a mobile phone, 53% of those being smartphones. Moreover, 84% of smartphone owners had downloaded an app to their device and 19% had downloaded an app to specifically manage their health [9].

Despite the recent proliferation of apps to promote positive lifestyle change, there is a dearth of research evidence regarding their effectiveness. Further, content analysis of existing apps have identified gaps between evidence based guidelines and app content relating to smoking cessation [10, 11], weight loss [12, 13], diabetes [14] and exercise [15]. While theoretically grounded mHealth behaviour interventions increase the likelihood of achieving behaviour change, it has been suggested that current theories are inadequate to guide mHealth interventions, which need to be more interactive and dynamic [16]. Abraham and Michie have suggested that there are a number of behaviour change techniques (BCTs) common to many health behaviour theories [17], of which at least five are evident in effective physical activity and dietary interventions (i.e. self-monitoring, intention formation, specific goal setting, review of behavioural goals and feedback on performance) [18]. While studies have been conducted to determine the extent to which behaviour change theory has been applied to app development, none have quantified the extent to which specific BCTs are included.

The present study sought to determine the presence/absence of BCTs in the top 20 free and top 20 paid physical activity and dietary smartphone apps from the New Zealand iTunes Apple App Store Health & Fitness category. A second aim was to determine whether the BCTs taxonomy checklist could be used to reliably identify BCTs in smartphone apps.

Methods

Study design

A comparative assessment of the presence of BCTs within smartphone apps from the New Zealand Apple App Store Health & Fitness category was undertaken by four independent raters. The raters were all trained in health behaviour change; one dietetics and nutrition masters student, two health sciences doctoral students and one health psychologist. The study was conducted in accordance to ethical standards. Human subjects were not recruited and therefore no ethics approval was required.

Sample

The apps were located and downloaded using the software ‘iTunes’ on November 12 of 2012 (available for download at http://www.apple.com/itunes). To be included in the present evaluation apps had to be designed either to promote health or prevent disease, and specifically address physical activity and/or dietary behaviours. Irrelevant and miscategorised apps under the specified Health & Fitness category or apps addressing other health behaviours were excluded. Apps were identified within the Health & Fitness category of iTunes by clicking on the “Top” ordering function button (i.e. “Top Paid iPhone Apps” and “Top Free iPhone Apps”). Their titles and descriptions were initially screened by the first author. Apps that met the inclusion criteria were downloaded until a total of 20 was achieved. This procedure was repeated to retrieve the 20-top paid and 20-top free apps. During screening, nine top-paid and 11 top-free apps were excluded for not meeting the inclusion criteria (see Figure 1 for an overview of the sampling procedure). All apps were downloaded to an iPhone 4 or 5 running iOS 6.0.1 (version of Apple operating system for iPhones).

Figure 1

Selection of sample of apps. Procedure for selection of sample of physical activity and dietary apps.

Measurement

The Taxonomy of Behaviour Change Techniques Used in Interventions and the Coding Manual to Identify Behaviour Change Techniques in Behaviour Change Intervention Descriptions were used for the present evaluation [17]. Abraham and Michie previously developed and demonstrated the feasibility and reliability of using the taxonomy for identifying BCTs in behavioural interventions. The coding manual provides guidelines to detect whether an intervention description includes any or all of the 26 defined BCTs. Mean kappa values of 0.80 and 0.82 (i.e. good reliability) have been observed when applying the taxonomy of BCTs to physical activity and healthy eating intervention descriptions, respectively [17]. Each app was rated for inclusion of each of the 26 BCTs. Formal statistical comparisons in terms of differences in the number of BCTs between apps were not conducted for the following reasons. First this was an exploratory study to determine whether an existing coding system could be used reliably to assess BCTs among mobile phone apps. Second, given the number of apps assessed in this study and the potential for differences in BCTs between apps, the number of comparisons needed would likely result in Type 1 error.

Procedure

For each app, descriptive information was retrieved regarding its popularity (i.e. frequency of downloads relative to other apps within the same category), average rating (i.e. average number of stars the app was rated ranging one to five), total ratings (i.e. number of users who downloaded the app and voluntarily rated it), total "hate it", “don’t like it”, “it’s ok”, “it’s good”, and “it’s great” ratings (i.e. number of times the app was rated with one to five stars, respectively), customer reviews (i.e. number of times the app was reviewed) and price (as shown in Additional file 1). Every app was evaluated by four independent raters using three iPhones between November 2012 and April 2013. The four raters separately tested all apps in detail to become familiar with the interfaces, menus, features, and functionality (e.g. “profile”, “routes”, “workouts”, “friends”, “meals”, “charts”, “analysis”, “my plan”, “nutrition”, “settings”, “tips & tricks”, “FAQ”). The apps were user-tested independently by each rater.

Prior to evaluation, all raters read the BCTs definitions carefully and had the opportunity to clarify and discuss the definitions. Before beginning a coding session raters read each BCT description carefully to ensure clear differentiation between techniques. After using each app, raters reviewed each of the menu functions to rate the presence or absence of BCTs according to the checklist. A dichotomous score of “0” absent or “1” present was applied for each of the 26 BCTs. Disagreements were resolved by consensus discussion.

Statistical analyses

All statistical analyses were conducted using IBM SPSS Statistics version 20.0. Frequencies and percentages of each of the 26 BCTs included in the 40 apps were calculated. Krippendorff’s alpha was used to evaluate interrater reliability for each of the 26 BCTs. This statistic is appropriate because it can be used with any number of observers, sample sizes, and satisfies all criteria for a good measure of reliability [19]. Further, a macro that computes Krippendorff’s alpha is available for statistical software packages, such as SPSS [19].

Results

The majority of the 40 apps reviewed targeted physical activity (30 apps, 75%), followed by dietary behaviour (6 apps, 15%), and combined behaviours (4 apps, 10%). Attributes of the paid and free apps are presented in Table 1. Generally, apps were rated in iTunes customer ratings as good (mean = 4.1 on a scale of one to five stars). Free apps were on average rated more times (193.4) and had more customer reviews (108.7) than paid ones (35.9 and 22.9, respectively). The average rating was slightly higher for paid (4.2) compared to free apps (3.9).Overall, apps included an average of 8.1 (range 2-18) BCTs, with slightly more BCTs present for paid (mean = 9.7, range 2-18) as compared to free apps (mean = 6.6, range 3-14) (see Figure 2). Commonly included BCTs were “provide instruction” (83% of the apps), “set graded tasks” (70%), and “prompt self-monitoring” (60%). “Model/demonstrate the behaviour” (53%), “provide opportunities for social comparison”, “plan social support/social change” and “prompt identification as a role model” were also frequently incorporated (55%). “Prompt barrier identification”, “prompt self-talk”, and “motivational interviewing” were seldom included (3%), and “teach to use prompts/cues”, “agree on behavioural contract”, “relapse prevention” and “time management” were not included. “Prompt intention formation”, “provide general encouragement”, “prompt specific goal setting”, “prompt self-monitoring of behaviour”, and “prompt practice” were techniques included more frequently in paid compared to free apps.

Table 1 Descriptive data of the top-40 apps

Figure 2

Incorporation of BCTs within the top-40 apps. Percentage of apps incorporating each of the 26 BCTs within the Top-40 Applications from the New Zealand Apple App Store Health & Fitness category by cost and overall.

Reliability data are presented in Table 2. Coefficients ranged from 0.1 to 0.9 with a mean of 0.6 (SD = 0.2), indicating moderate reliability. Raters agreed that none of the apps used BCT15 “teach to use prompts/cues”, BCT16 “agree on behavioural contract”, BCT23 “relapse prevention” and BCT26 “time management”. Despite 100% agreement, the calculation of alphas for these techniques was not possible because there was no variation in the reliability data matrix [19]. Of the 22 reliability tests, seven (32%) yielded alphas of more than 0.7, indicating good reliability, and only two (9%) were below 0.4. Inferior reliability was observed for four techniques: BCT2 “provide information on consequences” (0.4), BCT3 “provide information about others’ approval” (0.4), BCT5 “prompt barrier identification” (0.1), which was only observed once, and BCT6 “provide general encouragement” (0.4).

Table 2 Reliability of BCT identification

Discussion

This study identified the presence or absence of BCTs in popular physical activity and/or dietary behaviour apps. There was substantial variation in the numbers of BCTs present, with an average of eight techniques per app. Using a taxonomy and coding manual it was possible to identify BCTs used in smartphone health behaviour change applications. Beyond these general observations, specific issues are outlined below.

Previous research has already highlighted the shortage of theoretical content present in interactive technologies such as web sites [20] and apps [10, 12, 14] designed to promote health behaviour change [11, 13, 15, 21, 22]. Consistent with previous research, our findings demonstrate the relative absence of behaviour change strategies present in physical activity and dietary apps. Moreover, this study highlights the potential to improve future app development by incorporating key strategies known to enhance behaviour change. For example, existing technologies permit real time assessment, feedback, and tailoring, however, in the present study, only 38% and 23% of the apps prompted specific goal setting or prompted review of behavioural goals, respectively.

The five BCTs shown to be commonly associated with greater effectiveness for modifying physical activity and diet in previous studies were present to varying degrees in the apps reviewed here (i.e. self-monitoring – 60% of apps, intention formation – 50%, specific goal setting – 38%, review of behavioural goals – 23% and feedback on performance – 53%). However, these five BCTs were in general more common in paid versus free apps. BCTs such as relapse prevention, which is important for sustained behavioural change [23] was not present in any of the reviewed apps, which questions the value of these apps for changing behaviour in the long-term.

The observed differences in reliability identifying BCTs indicate the need to clarify definitions and/or coding instructions. We evaluated the presence of BCTs using a coding instrument originally developed to identify BCTs from written text in published papers describing an intervention [17]. Perhaps specific coding instructions to apply when assessing the active ingredients of mHealth or interactive technologies such as apps or video games can be developed. The present research included a taxonomy of 26 techniques; however, subsequent taxonomies have been developed [24]. Future content analysis of apps should apply this updated hierarchical version of the BCT taxonomy.

While identifying the active content of health behaviour change interventions is crucial, researchers must be aware of the caveats of ascribing effectiveness to certain BCTs or combinations of BCTs. To do so, researchers should also consider the parameters for effectiveness for each BCT. These are the required characteristics that a translation of a BCT to usable intervention elements must incorporate (i.e., an effective BCT is undermined if not correctly applied) [25]. Additionally, the effectiveness of BCTs is determined by contextual factors such as target population (e.g., sample characteristics), behavioural domain (e.g., physical activity, smoking) and study design factors (e.g., follow-up period, blinding). Further, BCTs frequently co-occur in interventions and they can interact with each other [25]. Hence, conclusions about the behaviour change potential of apps based on incorporation of BCTs should be interpreted cautiously as BCTs are not effective under all conditions. Caution interpreting our findings in terms of differences in the number of BCTs between apps is also warranted as we did not conduct formal statistical comparisons.

The increasing number and diversity of apps available makes its assessment a difficult task for the public and clinicians to differentiate which apps can be useful in promoting behaviour change. Presumably, the value of apps can be enhanced by developers incorporating more features, theory, and BCTs into their apps, which in turn will increase the behaviour change potential of the app. The current study suggests the higher potential quality of paid apps should be a factor to weigh when selecting and using apps for personal use, clinical intervention, or future research. Furthermore, guidelines can be created to influence and help app developers as to which BCTs (and other components) to include that likely will enhance the behaviour change potential of apps.

Despite the proliferation of physical activity and dietary apps, it is not clear whether they are effective at modifying behaviour. At present, there is a dearth of effectiveness data of app-based interventions to promote healthy behaviours [26], and robust, rigorously conducted and adequately powered trials are required to determine their effectiveness. On the other hand, app development proceeds at a rate that far out paces time frames typically observed in trial development and conduct. Thus, more dynamic forms of evaluation methods are required to determine the effectiveness of such technologies [27]. Generally, the effectiveness of mHealth interventions such as text messaging for modifying health behaviours (e.g., smoking cessation) has been established [28]; however the effectiveness of more complex and dynamic mHealth interventions including apps has yet to be determined.

A strength of this study was the use of an established instrument to systematically rate the incorporation of BCTs in the respective apps. However, in the present study, the presence of BCTs was determined by user-testing the apps rather than from text descriptions. Some app features were not explicit during use. For example, reminders, weekly updates, and pop-up feedback, etc, may have occurred for one, but not all raters at any given time. Despite these issues, modest reliability between raters was observed (0.6). Another strength was the use of four raters, with a range of behaviour change experiences, which provided a more comprehensive assessment of the apps and the use of the taxonomy checklist. A major limitation of this study was not including apps from other app stores such as the Google Play Store/Android platform, or app stores from other countries besides NZ, which limits the generalisability of the findings. Nevertheless, we investigated the most popular and commonly downloaded apps of the iTunes Apple App Store Health & Fitness category, which represent a sample of apps that many people are using and therefore increases the study relevance. Of note, apps may exist that incorporate more evidence based BCTs than those included in the study sample as we only rated the most popular apps. Furthermore, technology has a dynamic nature with new apps and updates developed every day, consequently, these evaluations need to be updated periodically.

The advantages of mobile phone (mHealth) solutions compared to other health intervention delivery modes include the persistent interactivity, personalisation and engagement, potential to make healthcare more accessible and scalable, more cost-effective and more equitable [29]. Such characteristics provide significant potential to assist in disease prevention strategies and supporting sustained change in lifestyle behaviours. However, there are too many apps for consumers and professionals to choose from [30]. In addition, the majority within the health & fitness category of the Apple iTunes U.S. store scored less than 40 out of a possible 100 for functionality according to a recent report from the IMS Institute for Healthcare Informatics that concluded apps do little more than providing information [31]. Emerging evidence demonstrates the need for collaboration between health behaviour change experts and app developers to create apps that include effective BCTs. Future research is also needed to better understand how individuals use apps after downloading them, and to investigate features that may impact user acceptability and preference [32].

Conclusions

Presence of BCTs varied by app type and price; however, BCTs associated with increased intervention effectiveness were in general more common in paid apps. The BCTs taxonomy checklist can be used by independent raters to reliably identify BCTs in physical activity and dietary behaviour smartphone apps.