Sustainability of the whole-community project '10,000 Steps': a longitudinal study

Prevention programs represent an important strategy for improving public health, but their effectiveness rests on the extent to which they are evidence-based and can be widely disseminated and implemented [1, 2]. The extent to which an evidence-based health program can deliver its intended benefits over an extended period of time after external (i.e. financial, managerial, or technical) support is terminated has recently been described as 'sustainability', and dimensions of sustainability are situated at individual, organizational and community levels [3]. Dissemination and implementation specific to PA programs have been considered a staged process with sustainability representing a later stage in these programs' life cycle but dependent on success at earlier stages [4, 5]. An alternative model on the temporal aspects of sustainability argues that program implementation and sustainability occur concomitantly rather than in stages [6]. Sustainability is particularly relevant for public health practitioners and policy makers because investments in public PA programs need to be efficient, and lack of sustainability could lead to an investment loss [5, 6].

Research into sustainability is considered a key criteria for determining the generalizability or external validity of PA programs [7, 8]. However, recent systematic reviews of external validity information in studies of community-wide PA interventions concluded that most studies lacked reports of sustainability [9, 10]. Lack of reported sustainability information at both the individual and organisational levels in community-wide studies is problematic because it limits the estimation of the long-term impact of such interventions [9]. Additionally, the extent of sustainability is also underreported, including information about which specific intervention components were retained, which were discontinued, and why [11].

The extent to which intervention components are modified or adapted by organizations to suit community contexts is also considered very relevant for sustainability research [5, 12]. From a dynamic systems perspective, organizations operate in specific contexts where they are engaged in interactions with health programs and the community, and these interactions may adapt to new conditions [12, 13]. This dynamic systems perspective implies that organizations would shape PA programs to their specific (community) context or to changing circumstances [13]. Documenting these adaptations can facilitate the dissemination of best practices and therefore promote sustainability [14].

In response to the dearth of information in dissemination and implementation literature about the sustainability of community-wide PA programs in general and of '10,000 Steps' in particular, the Government of Flanders (Dutch speaking part of Belgium) provided funding to study the sustainability of '10,000 Steps' in the Flemish region. The '10,000 Steps' project in Flanders is a whole-community project including multiple community-based strategies to promote PA in the adult population [15]. Its intervention components constitute a socio-ecological approach [16], including personal (e.g. pedometer sale and loan, dissemination of information), organizational (e.g. organization of community events), community (e.g. street signs, local media campaign) and policy (e.g. partnerships with other stakeholders) components. Efforts to disseminate '10,000 Steps' in Flanders were initiated in the last quarter of 2007. Dissemination strategies focused primarily on the use of media and peer networks of public health and sports services based on the principles of Roger's Diffusion of Innovations [15, 17]. The dissemination and implementation of '10,000 Steps' was studied in 2009, after which funding for project dissemination ended [15]. Using the RE-AIM framework (reach, effectiveness, adoption, implementation, maintenance) [18] for evaluation, a project adoption rate of 36% and a global implementation score of 52% was measured in professional organizations at that time, while effectiveness measures showed that citizens aware of '10,000 Steps' had significantly more leisure time PA than citizens unaware of it. Figure 1 shows that the initial evaluation with the RE-AIM framework provided final results for 4 out of 5 dimensions including reach, effectiveness, adoption and implementation. Further details of the initial evaluation with the RE-AIM framework are provided elsewhere [15]. However, measures of sustainability or the 'maintenance' dimension in the RE-AIM framework were premature at the time of data collection and follow-up measurements for that particular dimension were recommended [15]. Such follow-up measurements could expand the limited knowledge on the potential of whole-community projects like '10,000 steps' to be continued and institutionalized in organisations after dissemination, and to sustain behavioural effects. Therefore, the aim of the current study was to evaluate sustainability-indicators of '10,000 Steps' at organizational and individual levels, two and a half years after project dissemination and more than 1 year after the prior dissemination and implementation study.

Figure 1 illustrates the combined results for the sustainability measures of '10,000 Steps' in the current study and for the initial evaluation with the RE-AIM framework. In this section results of the current sustainability study will be reported in more detail.

This longitudinal study measured the sustainability of the whole-community project '10,000 Steps' in extension of our previous research on this project's dissemination and implementation process in Flanders [15]. It contributes additional findings to the limited research on sustainability of community-wide PA programs available in the literature [9, 10].

Levels of project continuation among organizations were 50% which was 10% lower than the 60% benchmark reported by a review on empirical sustainability studies of health-related programs [11]. However, this comparison should be interpreted with caution because the types of programs reviewed varied considerably and may not be directly comparable to '10,000 Steps'. It has been hypothesized that complex interventions, such as whole-community projects, that call for changes in different socio-ecological levels of dynamic systems (i.e. communities) are more difficult to sustain [12]. Further research on the sustainability of other whole-community projects promoting PA could explore this hypothesis.

Conceptual frameworks argue that sustainability is determined by a combination of factors related to project design, organizational setting, and the broader context [5, 26, 27]. Findings of the present study support these frameworks and suggest several factors particularly important for increasing the sustainability of whole-community projects like '10,000 Steps'. Reported reasons for project continuation indicate the importance of offering ready-for-use and adaptable interventions to public service organisations including project manuals, materials and websites (project design factors), targeting the organization's superior as a project champion (organizational setting), and providing funding or external support from other stakeholders (broader context). A major reported barrier to project continuation was the preference to switch to another PA project. Next to a lack of project compatibility with the perceived needs of reporting organizations, this finding suggests a need for organizations to have a choice in their decision-making about project implementation. Dearing and colleagues [28] argue that increasing project choice is a key principle for disseminating proven approaches to PA promotion. A logical recommendation would be to combine sustainability strategies of '10,000 Steps' with strategies that cluster alternative evidence-based programs [28]. The web portal Cancer Control PLANET (Plan, Link, Act, and Network with Evidence-based Tools) is a practical example of this principle [29]. Considering external validity indicators in the present study, these were promising, with project continuation rates found to be independent of staff size, type of organization and working context. The global implementation score (58%) remained stable and the majority of project components were continued by more than half of organizations. It is encouraging that partnerships in particular were continued by 75% of organizations as partnerships with other stakeholders are argued to have a positive influence on long-term sustainability of projects [11, 14]. Three project components (personal contact, street signs and variants, and posters) were continued by less than half of organizations. These same components also had low implementation rates with negative z-scores in both the current study and the prior dissemination and implementation study. Personal contact with citizens seemed infeasible for a substantial portion of organizations, which is consistent with findings of a smaller scale dissemination study of an evidence-based PA promotion program for the elderly in community organizations [30]. Mixed reasons, including costs, were listed for non-implementation of street signs or variants, implying the need for alternatives. After similar reasons had been reported in the prior dissemination and implementation study, the Flemish Government initiated an intermediate support structure for '10,000 Steps' on the regional level. This intermediate structure consisted of linkage agents (health promotion services) who developed more workable alternatives such as street banners and flags. However, because planning time for both the intermediate structure and alternative materials was underestimated, availability of these materials was delayed by 1 year. The potential of street banners and flags could therefore not be fully assessed. Production delays of other project materials on the regional level were directly reflected in listed reasons for not continuing the use of posters. Several organizations listed dependency of poster production on the regional level which forced them to continue '10,000 Steps' without posters. This underlines the importance of synchronizing project strategies and change across different levels of a social system to optimize sustainability, as also suggested by Schensul [12]. When considering the targeted PA domains, it was clear that leisure time was the organizations' preferred context to continue the promotion of '10,000 Steps', followed by transport, household, and work.

Several organizations changed the project focus of '10,000 Steps' and switched between a whole-community approach and a specific target group (e.g. the elderly). This emphasizes that sustainability is a dynamic process and strategies for achieving it ought to adapt to changing contexts and priorities [5]. In order to facilitate this dynamic process, project manuals should include extensive guidelines for project adaptations that don't jeopardize project fidelity. This includes specific manuals for projects targeting the elderly only, or projects in workplaces. Furthermore, many organizations continuing '10,000 Steps' changed the original campaign image into one that reflected their regional or local identity (e.g. colours, partner logos, and multiple images). As argued by Dearing and colleagues [28], it is important that projects provide organizations more opportunities to raise their profile and to feel project ownership as a strategy to encourage project sustainability. Furthermore, organizations reported alternatives for less sustainable project components, such as exhibition banners or foot stickers in public places instead of regular posters. This kind of information is useful for formulating guidelines on how to reinforce or replace less sustainable components with alternatives that retain the original function of promoting awareness and knowledge about PA. Future sustainability studies of community-wide PA programs could facilitate the spread of best practices for various settings by documenting adaptations and whether these adaptations retain essential behaviour change elements [7, 14].

Institutionalization levels for routines and project saturation indicate that '10,000 Steps' became moderately integrated within the culture of organizations through policies and practice. However, two separate aspects of institutionalization were poorly integrated. Few permanent staff members contributed to project implementation, which could jeopardize sustainability in case of staff turnover. In addition, very few organizations reported formal project evaluation. As evaluation was not explicitly recommended in the initial '10,000 Steps' project findings suggest it is necessary to do so in the future. Wilson and Kurz [31] argue that lack of evaluation is a significant threat to institutionalization. Without evaluation, the project's potential benefits cannot be monitored, which may reduce relevancy of the project within the organization. In order to facilitate project evaluation and sustainability a combination of measures is recommended. First, Wilson and Kurz [31] recommend providing organizations with training and practical methods for continuous evaluation and quality improvement, such as the "Plan-Do-Study-Act" framework [32]. Second, this study also supports Brug and colleagues [33] who argue the importance of better planning, considerable more funding opportunities for evaluation, and the formation of collaborative centers between public health services and academic experts.

Seven out of twelve organizations reported to have sustained '10,000 Steps' by implementing periodically repeated projects rather than ongoing projects. This implies that project sustainability in organizations can also manifest itself in repeated time cycles rather than on a continuous basis only. This alternative approach to the time dimension of sustainability seems more logical and appropriate for projects like '10,000 Steps' which aim to increase citizens' awareness about PA. For example, project components such as street banners may be disregarded by citizens after years of permanent allocation. Such components may grasp interest of citizens in a more sustainable manner when implemented repeatedly in cycles and as part of a comprehensive intervention [34]. As discussed previously, it would be recommendable to offer project choice to organizations in public health. This would allow organizations to alternate cycles of '10,000 Steps' with cycles of other evidence-based PA programs. This also implies that projects like '10,000 Steps' would need to innovate regularly (e.g. by anticipating and facilitating adaptations) to sustain high levels of interest in both adopters and citizens. The optimal duration or schedule for project cycles of '10,000 Steps' is probably different for each community context because sustainability is a dynamic process. However, based on the present study it seemed that periodically repeated projects can be feasible when they have a maximum duration of 1 year and are alternated with varying periods of interruption.

In the prior dissemination and implementation study, significant differences in domain-specific self-reported levels of PA were only found for leisure-time PA, with citizens aware of '10,000 Steps' found to have higher levels than those not aware [15]. No differences were found for intensity-specific self-reported levels of PA at that time. The same comparative analyses for the current sustainability study confirmed maintenance of significantly more leisure time PA in citizens aware of '10,000 Steps'. In addition, in the current study citizens aware of '10,000 Steps' also reported significantly more household-related and moderate intensity PA. Project materials of '10,000 Steps' promoted all PA domains and also the public health guideline of 30 min of moderate to vigorous intensity PA per day, which may have contributed to additional time-delayed effects in favor of citizens aware of '10,000 Steps'. Furthermore, results suggest (although not significant) that citizens that were unaware of '10,000 Steps' had more work-related PA and vigorous intensity PA. The latter finding could indicate higher fitness levels in these citizens. However the testing of fitness levels was beyond the scope of the current study.

Limitations of this study include the use of self-report data and reliance on only one respondent per organization. With more resources these approaches could become more valid by adding objective types of data collection (e.g. on-site observations) and by contacting multiple respondents per organization [11]. While the sample of organizations was relatively small, which may limit external validity of results, the sample itself appears to be generalizable. In the prior study of '10,000 Steps' [15] external validity analyses revealed no significant differences between these same organizations and the larger group (n = 44) of non-adopting organizations on organizational characteristics (e.g., size, type, and context). Moreover, despite their small number, organizations in the current study served substantial populations and could have wide-scale impact on public health. Finally, this study did not address the issue of capacity building and community level changes, which is considered a third indicator of sustainability [3]. For example, we did not explore if a capacity-building strategy for local staff members influenced the sustainability of '10,000 Steps' or the development of other PA programs. Green [35] argued that rather than program continuation and institutionalization the most appropriate goal of sustainability may be that practitioners learn new skills and new program approaches.

Strengths of this study include its contribution to a small available literature on the sustainability of community-wide physical activity (PA) programs in general and of '10,000 Steps' in particular. Sustained implementation of project components and adaptations were also assessed. This provided more insights in less sustainable components and its contextual barriers, which may also be relevant for other sustainability research. Reported adaptations provide additional information to potentially improve project sustainability in various and changing contexts.

This study shows that projects like '10,000 Steps' could remain sustainable and that project sustainability in organizations can also manifest itself in periodically repeated projects (cycles) rather than ongoing projects (continuous). However, several important barriers need consideration. Limited project evaluation and permanent staff appointed to project implementation may jeopardize institutionalization, while insufficient planning time to optimize synchronization between regional and community policy levels may be harmful for sustained implementation of project components. To make whole-community PA projects like '10,000 Steps' more sustainable several strategies are proposed. These include obtaining support of program champions with upper-level managerial authority and 'packaging' projects as ready-for-use products, with ample space and guidelines for adaptation to various and changing contexts and priorities. Furthermore, maintaining funding that also meets practical needs for project evaluation (e.g. training and academic collaborative centres) and a regional (intermediate) support structure may be beneficial. Future dissemination and implementation research should examine the sustainability of other whole-community PA projects to compare with the present study. It would also be interesting to compare alternative dissemination and implementation approaches in which institutionalization efforts are concomitant with implementation [6]. This allows finding alternative whole-community PA projects, which is necessary to optimize project choice for practitioners. Additionally, the current study could be extended by including sustainability at the community level. Researching sustainability of whole-community approaches to PA promotion is worth the investment because they have potential for developing the social and cultural change required for sustained improvements in population PA [36].