Skip to main content

A stitch in time saves nine? A repeated cross-sectional case study on the implementation of the intersectoral community approach Youth At a Healthy Weight

Abstract

Background

The implementation of programs complex in design, such as the intersectoral community approach Youth At a Healthy Weight (JOGG), often deviates from their application as intended. There is limited knowledge of their implementation processes, making it difficult to formulate sound implementation strategies.

Methods

For two years, we performed a repeated cross-sectional case study on the implementation of a JOGG fruit and water campaign targeting children age 0–12. Semi-structured observations, interviews, field notes and professionals’ logs entries were used to evaluate implementation process. Data was analyzed via a framework approach; within-case and cross-case displays were formulated and key determinants identified. Principles from Qualitative Comparative Analysis (QCA) were used to identify causal configurations of determinants per sector and implementation phase.

Results

Implementation completeness differed, but was highest in the educational and health care sector, and higher for key than additional activities. Determinants and causal configurations of determinants were mostly sector- and implementation phase specific. High campaign ownership and possibilities for campaign adaptation were most frequently mentioned as facilitators. A lack of reinforcement strategies, low priority for campaign use and incompatibility of own goals with campaign goals were most often indicated as barriers.

Discussion

We advise multiple ‘stitches in time’; tailoring implementation strategies to specific implementation phases and sectors using both the results from this study and a mutual adaptation strategy in which professionals are involved in the development of implementation strategies.

Conclusion

The results of this study show that the implementation process of IACOs is complex and sustainable implementation is difficult to achieve. Moreover, this study reveals that the implementation process is influenced by predominantly sector and implementation phase specific (causal configurations of) determinants.

Peer Review reports

Background

A worldwide increase in childhood obesity has been reported over the last decades [13]. In the Netherlands, an estimated 14 % of children have been classified as overweight or obese [1, 4]. Obesity often continues during adult life [5] and is linked to numerous adverse health outcomes [69]. As such, childhood obesity poses a major threat to public health [10], increases health care expenditures and as a consequence, constitutes an economic burden on society [11]. Intersectoral community Approaches to address Childhood Obesity (IACO) appear to have great potential to reduce and prevent childhood obesity [1217]. An IACO aims to target the multiple determinants of childhood obesity by involving various stakeholders from within the community [15, 18, 19]. An example of a successful IACO that resulted in a decline of childhood obesity is the French ‘Ensemble Prévenons l’Obésité Des Enfants’ (EPODE) program. The conditions for effectiveness of EPODE are attributed to four center pillars; (a) political and organizational commitment, (b) collaboration between public and private organizations, (c) use of social marketing and (d) the support of scientific evaluation. As a result of its success, several EPODE-derived community approaches were developed [2022]. In the Netherlands, the EPODE-derived JOGG approach (an acronym for Youth At a Healthy Weight, in Dutch) was installed [23, 24].

The innovation process of an IACO can be defined as the iterative cycle of program adoption, implementation and continuation [25]. This process is considered challenging; a translational gap between innovation development and implementation is often reported. Systematic insight into the delivery of innovation activities and the implementation of these activities by the intended user population is needed to develop strategies that have the potential to decrease this translational gap. Ultimately, these strategies can optimize the potential impact of the innovation [2628].

Research on the implementation of interventions often focuses on fidelity: the extent to which an IACO is put into practice [29]. One critical aspect of fidelity is completeness, defined as ‘the proportion of IACO activities prescribed that is being put into practice’ [30]. Next to questions regarding completeness, research should also focus on the elucidation of determinants of completeness. Knowledge on these determinants is necessary to develop innovation strategies that have the potential for real change to occur [28, 3135]. Only a dozen studies have specifically addressed the innovation process of IACOs [17, 36]. Even fewer studies have evaluated these processes longitudinally. Moreover, the quality of studies performed is not always up to par and determinants found to be critical still need to be (dis)confirmed by future research [37].

As part of a larger study [38], we therefore performed a repeated cross-sectional study on the innovation process of the JOGG “fruit- and water campaign”, evaluating both implementation and continued implementation of the campaign. This JOGG campaign took place in a disadvantaged neighborhood in a major city in the Netherlands, and aimed to promote healthy eating and drinking habits in children aged 0–12 years. Campaign strategies, mainly derived from social marketing, consisted of supplying promotional materials and organizing campaign activities such as educational supermarket visits and decorating water cans. Moreover, the campaign aimed issue a positive message to the target population (water and fruit are cool and hip!).

Our research questions were:

  1. 1.

    To what extent were the JOGG fruit- and water campaign activities implemented as intended (completeness) from December 2011 to July 2014?

  2. 2.

    What appeared to be the most critical determinants of the implementation of this campaign?

    1. a.

      Did determinants differ between the sectors involved (healthcare, educational, sports, welfare and private sector) insert supporting information 1, sector categorization)?

    2. b.

      Did determinants of implementation differ in time?

Methods

Design

This study was approved by the ethical committee of the Faculty of Psychology of the University of Leiden, reference number 8259652117. The evaluation was guided by the framework by Saunders et al. [27].

Research took place from the start of the campaign in December 2011 until its ending in July of 2014. As suggested by Saunders et al. [27] (Fig. 1), we first performed an inventory of the campaign’s setup (t0, research phase A) (Fig. 2). A blue print of the campaign design, setup and activities was then formulated. The implementation of the campaign was evaluated (research phase B, t1-t5) in five subsequent waves that coincided with ‘the booster months’ for either the water or fruit theme. For analytical purposes, we considered the first six months of campaigning as initial implementation, followed by mid-way implementation between 7–18 months, and continued implementation between 19–30 months. Thus, if an organization participated in the campaign from the start, initial implementation was assessed during t1, mid-way implementation during t2 and t3, and continued implementation during t4 and t5. A member check was obtained at t6 (research phase C).

Fig. 1
figure1

Overview of the framework by Saunders et al. [27]

Fig. 2
figure2

Overview of the research process

Sample & instruments

All research activities described in the next paragraph were performed iteratively during each research wave. Also, as stated by Saunders et al. [27], research instruments were adjusted before each wave following local developments and results of preliminary data analysis. Adjustments consisted, for example, of the addition of items to our interview topic list enquiring on ‘new’ determinants identified inductively via the preliminary data analysis.

At the start of each wave, all organizations meeting inclusion criteria were listed. Inclusion criteria consisted of (a) being situated within community boundaries, (b) receiving financial aid or materials from JOGG and/or (c) organizing activities within the context of JOGG. Via purposeful sampling [39], a selection of professionals working for the listed organizations was invited to participate in our study. Before participation, informed consent was obtained on audiotape from all participating professionals and transcribed verbatim.

To measure completeness (RQ1), prescribed campaign activities per organization were incorporated in observation checklists. If in vivo observation for certain activities was not possible, completeness was evaluated via the semi-structured interview detailed below. The checklist included items like “were fruit moments installed in your organization?” and “Did you organize the prescribed excursion to the local supermarket?”. Answers could be either yes (=1) or no (=0).

To evaluate determinants of implementation (RQ2), semi-structured interviews were conducted. The framework of Fleuren et al. [31] was applied as a lead for the selection of interview topics [4042]. This framework distinguishes five major categories of determinants: (a) characteristics of the socio-political context, (b) characteristics of the organization, (c) characteristics of the intended user, (d) characteristics of the innovation and (e) characteristics of the innovation strategies. Interviews were held face-to-face and their duration varied from fifteen tot sixty minutes.

Document analysis was performed on planning documents, minutes of campaign team meetings and campaign manager’s log entries. Finally, field notes containing both notes from data collection and prejudgments of the researcher were taken into consideration.

Our study can be considered as ‘semi-action research’; we provided community stakeholders with study results after every wave and encouraged reflectivity. However, we did not advise them how to translate study results into improvements of the campaign. In this way, stakeholders were provided with the opportunity to optimize IACO implementation while keeping the level of data contamination to a minimum.

Analysis

As for completeness (RQ1), all observations checklists were digitalized and transported to Microsoft Excel 2010. The proportion of all prescribed activities that were put into practice was then counted and a standardized score (percentage) per professional was calculated.

Interviews with professionals on the determinants of implementation (RQ2) were transcribed verbatim and transported to Atlas.ti for Windows version 6.2 (Scientific Software development, Berlin). They were then coded separately by two researchers (RK, SA), using a framework approach [43] derived from Fleuren et al. [31]. Data analysis was performed after each wave, and at t5 all previous analyses were re-evaluated. Next, data was further reduced by formulating within-cases and cross-cases [44]. Within-cases consisted of a narrative and a list of the most important facilitating and impeding determinants per professional. The subsequent cross-cases compared facilitating and impeding determinants per wave, sector and implementation phase. A determinant was classified as a ‘key determinant’ if it was indicated as a barrier or facilitator by more than 50 % of the professionals in the concerning cross-case.

Causal configuration analysis

During cross-case analysis, we found that the determinants were not only self-contained, but seemed to be interrelated and occurring in causal configurations (e.g. presence of determinants A + B + C= > outcome X and presence of determinants B + C + D = > outcome Y). We considered using Qualitative Comparative Analysis (QCA) to analyze these configurations as QCA allows for interrelation analysis when different configurations generate the same outcome [4548]. Moreover, this technique was successfully used to analyze similar configurations by Ordanini, Parasuraman and Rubura [47]. Our interviews however were semi-structured; participants did not provide information on exactly the same determinants. We therefore did not have data on the same determinants for all cases. To counter this challenge and at the same time preserve QCA assumptions, we translated QCA principles to a QCA derived causal configuration analysis (Fig. 3). We identified three outcome categories (low, medium and high completeness). Scores one standard deviation (SD) below the mean were categorized as low completeness, between one SD above and below the mean as medium completeness, and one SD above the mean as high completeness. We also determined sector membership and the implementation phase evaluated per professional. We then identified key determinants via cross-case comparison. After, we explored all possible causal configuration to see if the operator ‘or’ or ‘and’ between determinants could be placed (streamlining of conditions). Finally, truth tables were formulated for each possible configuration and a search for conforming and deviant cases was carried out. If contradictory cases were present, we decided that 75 % of professionals needed to confirm the configuration to be indicated as a causal configuration of determinants.

Fig. 3
figure3

QCA derived causal configuration analysis

Results

Research phase A (t0): Campaign inventory & sample

The fruit and water campaign was part of the national JOGG approach and was financially supported by the local municipality. A campaign manager installed by the Municipal Health Services guided the installment of the campaign, in cooperation with a local campaign team. The campaign was setup as a manualized intervention. However, campaign activities were not prescribed via step-by-step instructions. Instead, activities were prescribed via less formalized instructions such as ‘install a water moment’ or ‘organize an excursion to the local supermarket’. Multiple organizations participated within the campaign, including schools, health organizations and private enterprises. All participating organizations were prescribed campaign activities directed towards the promotion of water and fruit consumption in children. However, the amount and specific content of prescribed activities varied per organization and sector, in accordance with their needs and possibilities. For example, supermarkets were not asked to organize any specific activities for children as they had indicated that insufficient time was available to facilitate such activities. Campaign activities could be divided into key- and additional activities for each sector (Table 1). Key activities were hypothesized by campaign management to be crucial for reaching the intended health promotion effect and were intended to be continued by the participating organizations after the end of the campaign. Additional activities were not intended to be continued after the campaign ended.

Table 1 Overview of key and additional campaign activities

Our sample achieved reflects the complex and ever changing nature of the campaign; not all organizations could be included during all waves (Table 2). This was mostly due to organizations not being prescribed campaign activities or organizations declining (further) participation in the study. Limited time available for data collection by the researchers was also a reason for non-participation. During the study, priority was assigned to those organizations that were prescribed the most campaign activities and/or were considered as most critical for the impact of the campaign. To determine which organizations were most critical for impact, campaign management estimated the number of children that was going to be reached by activities, but also reviewed the content of the prescribed activities. For example, an organization providing a daily water moment during afternoon play activities to a hundred children was prioritized for study participation over an organization that performed a theatre play incorporating a water theme for twenty children on a one day event. Also, a larger number of interviews were conducted during t1 at schools B & C in comparison with school A. This difference was caused by the fact that researchers from an aligned college were conducting an evaluation of fruit- and water consumption at schools B&C at the time of our study. As they were also qualified to perform qualitative research and were already visiting these schools, they were asked to interview the teachers enquiring on implementation using our interview topic list next to their own study. Hence, additional interviews were thus acquired during t1 at schools B&C.

Table 2 Sample achieved

Research phase B: Evaluation of implementation (t1-t5)

General findings

Completeness ranged from 0-100 % throughout sectors and implementation phases (Table 3). Overall, completeness of key activities was higher than for the additional activities The highest levels of completeness were observed in both the educational and health care sector. A majority of organizations showed a notable decline in completeness from t2 to t3.

Table 3 Completeness of key- and additional activities

Twenty-four key determinants were identified; ten facilitators and fourteen barriers (Table 4). Overall, high ownership towards campaign goals (feeling psychologically tied or attached to campaign goals [49]) and high compatibility of the campaign with existing working procedures were most cited as facilitators to implementation. Most frequently named barriers were a lack of reinforcement strategies for ongoing use of the campaign (e.g. a training or new promotional materials), a low priority for campaign use, low procedural clarity and incompleteness of campaign materials (e.g. insufficient quantity of campaign materials, campaign lacking classroom teaching materials). Eleven causal configurations were identified across four sectors (Table 5); ten configurations were related to a medium to high level of completeness. For the healthcare as well as the educational sector, we identified a causal configuration related to both medium and high completeness for an identical implementation phase. Across these sectors, the facilitators identified in the medium and high completeness configuration were mostly similar, whereas barriers were halved or not present at all in the high completeness configurations. For the private sector, a low and high completeness configuration was identified for continuing implementation. Barriers identified were identical for both low and high completeness, whereas facilitators were only absent in the low completeness configuration. Details per sector on levels of completeness, determinants and configurations are described below.

Table 4 Key barriers & facilitators per sector per implementation phase
Table 5 Causal configurations of determinants

Implementation per sector

Educational sector

Completeness of key activities in schools varied between 33-75 % during initial implementation. Overall, low completeness in schools during initial implementation was associated with a lack of procedural clarity or unforeseen negative experiences during implementation (for example chaos caused by preschoolers having difficulties making it to the bathroom).

We need to plan extra toilet breaks… look, he (student) just peed in his pants and that is just because he drank a lot of water due to the water campaign. (Teacher school B)

Throughout mid-way implementation, completeness declined to 50 % or less for schools A & C. Teachers from these schools ascribed this decline to the hectic working schedule they followed, which made prioritizing the promotion of a healthy lifestyle difficult.

We have been so busy the last couple of years, at a certain moment you think ‘I don’t even know the name of this student in my class’. So I think.. Yes, our main priorities lie elsewhere, not with the water campaign. (Teacher school A)

In schools B & D, completeness stayed above 50 % during mid-way implementation. This was often attributed to the program’s compatibility with pre-existing practices (such as the school schedule) or to the possibility to adapt non-essential elements of the campaign (such as timing of water moments) to their own needs.

During continued implementation, completeness recovered from 17 to 75 % at school A and remained above 50 % for schools B&D. Recovery of the completeness rate for school A was attributed to the instalment of an coordinator who advised on how to integrate campaign activities in daily routines (such as combining a play-time break with a water moment). Overall, high levels of completeness in continued implementation were associated with high levels of self-efficacy (beliefs about the ability to reach campaign goals).

At school C, completeness stayed below 50 % during continued implementation. Teachers from school C often attributed their low level of completeness to the lack of reinforcement strategies available for campaign use, such as the provision of a training or new promotional materials.

At first, everything was new, they (students) all had their campaign water bottles on their desks and it was very hip and happening! But, yeah, I don’t know, it is just not cool anymore now. (Teacher school C)

As for causal configurations, during both mid-way implementation and continued implementation professionals displaying high completeness indicated the same facilitators as professionals displaying medium completeness. However, professionals displaying high completeness indicated no key barriers (Table 5).

Health care sector

Within the health care sector, completeness of key activities varied from 67–100 %. Professionals stated that compatibility of the campaign with their daily practices facilitated implementation.

It (the campaign) is now part of my job. So I automatically integrate it into my daily work procedures, this makes the execution easier. (CYF, nurse)

Also, the presence of an internal coordinator to assist campaign implementation was named as a facilitator. During continued implementation, incompleteness of campaign materials was named as a key barrier by professionals. For instance, the distribution of campaign materials was often hindered which resulted in too little campaign materials being available.

With regard to configurations, professionals showing medium completeness during continued implementation stated campaign materials were incomplete and the campaign was poorly organized but found that the campaign to be compatible with existing procedures. All professionals displaying high levels of completeness mentioned the campaign to be compatible with existing practices and stated that they felt high ownership towards achieving the campaign goals. However, they cited the campaign lacked reinforcement strategies.

Welfare & sports sector

For the general welfare organization, a significant decline in completeness of key activities was observed from initial to mid-way implementation. After an initial uplift in completeness (67 %), levels declined again to 25 % at the beginning of continued implementation but reached a 100 % at the end of this phase. Recovery of completeness was mostly attributed to the adaptation of non-essential campaign components to local circumstances and the subsequent uptake of these activities in daily routine.

It costs quite a lot of time to organize a water or fruit booster. But because we now implement it (campaign activities) during our regular activities, it is working out fine! … we for example organized a community walk yesterday, and we provided children with a healthy snack. So it (campaign objectives) just became a standard procedure. (Social Welfare Organization, Social worker children)

Causal configurations revealed that professionals displaying medium to high completeness during implementation all reported that the campaign was incomplete or campaign procedures were unclear, but that they felt highly committed towards the goals to be achieved or found the campaign was compatible with existing work procedures.

Sports organizations showed low completeness (0-33 %) during initial implementation, and ceased campaign activities after this phase. This was mostly attributed to the incompatibility of the campaign with existing working procedures and incompleteness of campaign materials. They reported a mismatch between the equipment needed on the sports field (water tanks) and equipment received (water cans). Moreover, they reported that the number of pupils did not equal the promotional materials received and the promotional materials was delivered while the organizations were closed for the winter break.

Private sector

The household appliance stores opted out of the campaign after initial implementation. One supermarket showed a completeness score of 100 % during continued implementation, the other supermarket displayed lower levels of completeness (33 %). Ownership of campaign goals was cited as a key facilitator in all implementation phases. During initial implementation, the incompatibility of the campaign goals with the goals of the organization was identified as a key barrier.

I didn’t understand the campaign method, I thought the mega fruit cup was a hideous thing, that ruined the image of my shop!. … we have to draw a line somewhere, we are a supermarket and not the extension of municipal programs. (Supermarket B, manager)

Not having a feeling of shared commitment with community partners to implement the campaign was cited as a key barrier throughout continued implementation.

As i have experienced it, the campaign is very standalone instead of coming together with multiple partners and discussing ‘what are we going to do about it’? I think this would open a window of opportunities. (Supermarket A, manager)

Causal configurations revealed that professionals displaying low completeness during continued implementation stated the participation of the target population was lacking and that they did not experience a shared commitment for campaign use with community partners. Professionals displaying high completeness in this phase also expressed these barriers but stated they perceived campaign use as a personal obligation.

Discussion

The aim of this study was to evaluate completeness of the activities prescribed for the JOGG fruit- and water campaign and to identify the most critical implementation determinants.

Overall, completeness of activities was highest for the general welfare organization, and the educational and healthcare sector organizations. Moreover, completeness was higher for key activities than for additional activities. A decline in completeness was observed for a majority of sports- and private sector organizations after (initial) implementation, and a general decline in completeness was half way the study period. Key barriers identified varied more than key facilitators. High ownership for campaign goals and high compatibility of the campaign with existing procedures were most often cited as facilitators, whereas a lack of reinforcement strategies, a low priority for campaign use, low procedural clarity and incompleteness of campaign materials were most frequently indicated as a barrier. Eleven causal configurations of determinants were identified across sectors and a majority of configurations was related to medium or high levels of completeness.

Implications of findings

Previous research corroborates our findings that levels of completeness differs greatly between sectors and implementation phases [5052] and that sustainability of IACOs is hard to accomplish [53]. The general decline in completeness observed halfway the study period (t3) could be explained by the temporary incapacitation of the campaign manager, in combination with the set-up of the IACO. The water- and fruit campaign was highly manualized and delivered top-down, which has been associated with lower levels of ownership [54]. Hence, we argue that in particular in such a top-down implementation approach, the lack of campaign managers’ support in combination with this lower levels of ownership could explain the poor IACO sustainability [55]. Lack of the support of campaign management or lack of ownership were however not explicitly reported as barriers by the professionals; they solely reported a less orderly campaign organization and incomplete delivery of the campaign materials at t3.

The framework of Fleuren et al. [32] proved partly inadequate to identify determinants of implementation of IACOs; seven key determinants identified fell out of the scope of this framework. These determinants, such as ‘difficulty to collaborate with community partners’ seem to be more specific to the intersectoral, community-based characteristics of IACOs, and are in line with other studies on the implementation of IACOs [36, 5053, 5667]. Determinants identified were, to a great extent, sector and implementation phase specific. For example, perceiving campaign implementation as a personal duty or obligation was identified only as a facilitating determinant for the private sector, whereas uptake of the campaign in daily working routine was only named a facilitator for the educational sector. We therefore argue that implementation plans and strategies should be tailored to sector and implementation phase specific determinants. In addition, adjustments to implementation plans and strategies should be verified and discussed with professionals throughout the implementation process to ensure an optimal fit with the implementation context. This course of action responds to the need expressed by professionals from four out of five sectors to adjust the campaign and its strategies to local needs. This so called ‘mutual adaptation approach’ provides an opportunity to obtain site-specific feedback from local professionals, and was named in previous studies as a facilitator for institutionalization of health promotion programs [68] and the implementation of complex innovations in cancer care [69].

An interesting distribution of barriers and facilitators was found among the causal configurations identified for the educational and private sector. In the educational sector, the medium and high completeness configurations identified contained mostly identical facilitators. Most facilitators named in these configurations were internal, such ‘self-efficacy’, ‘ownership’ and ‘task orientation’. The distribution of barriers however differed between these configurations; the medium configuration contained mostly external barriers (such as procedural clarity of the campaign), whereas the high configuration contained no barriers at all. This could imply that, although the same facilitators were present, the absence of certain external barriers could be decisive to achieve implementation success in this sector. For the private sector, barriers identified for both the low and high configurations were similar and mostly external, namely ‘low participation of the target population’ and ‘not feeling part of collaboration in community’. However, an internal facilitator was only present in the high configuration, namely ‘perceiving campaign use as personal duty or obligation’. This could imply that, independent of the external barriers present, perceiving the campaign as a personal duty or obligation is a decisive factor for implementation success in the private sector. However, the fact that the barriers named in these configurations were mostly external and the facilitators named were mostly internal could indicate some form of self-serving bias [70] is present in our data. Hence, participants were perhaps inadvertently more prone to erroneously attribute success to internal factors, and failure to external factors.

The casual configurations extracted from our data indicate that a set of determinants can jointly lead to implementation success or failure. We therefore argue ‘the whole to be greater than the sum of the parts’ and that implementation might benefit more from implementation strategies based on all the configuration determinants combined, than of strategies based on single determinants. Further research testing the effect of such implementation strategies integrating causal configurations in its entirety is warranted to investigate this assumption. Finally, it should be noted that the analysis of causal configurations in qualitative research is still in its infancy [71]. Although, in our opinion, the of use an adapted version of QCA was the best choice to systematically analyze these configurations at this moment in time, readers should keep in mind that no golden standard yet exists and only a limited number of cases were studied. Hence, the reported results should be interpreted with caution.

Strengths & weaknesses

To ensure a systematic, theory-based study design, the framework of Saunders et al. [27] was used to guide our process evaluation. This framework allowed for a sharpened focus in data collection as well as the iterative adjustment of research methods in accordance with (preliminary) results. Although (preliminary) study results were used to adjust research methods, we did not use these results to adjust or improve campaign plans and strategies. Instead, the interaction with practice was guided by a semi-action research design. Hence, we presented the study results to stakeholders after every wave, but did not recommend any changes or alterations to campaign implementation. We chose this approach as to enhance stakeholders’ ability to optimize IACO implementation whilst ensuring a minimal level of data contamination. However, although we anticipated that the mere provision of results would encourage practice to optimize implementation plans, due to time limitations and lack of expertise little could be done by campaign management and practioners with the study results provided. We argue that Participatory Action Research (PAR) [72], in which researchers aid practioners with the translation of research findings into implementation strategies, could perhaps enable practice to take optimal advantage of process evaluation data. A review by Cook [73] revealed that PAR led to the translation of research finding into community action in fourteen out of the twenty studies reviewed. The benefits of PAR would therefore, in our opinion, outweigh the possibility of data contamination, which is perhaps partly inevitable when performing IACO process evaluations.

Several other methods were employed to optimize the credibility, objectivity and internal validity of our data [44, 74]. We collected data via in vivo observation, in contrast with most implementation studies who merely rely on self-reports [30]. Furthermore, data was recorded and transcribed verbatim, analytic software and a framework approach were used for data analysis and further data reduction was performed using theoretically approved methods [44]. Also, coding was performed by two researchers and the principal researcher (RK) kept a log about her opinions and prejudgments to increase awareness en reflexivity, reducing moderator bias [75].

One limitation of this study is the selection of participants. Due to the complex and rapidly evolving nature of the campaign investigated, selection of participants was not at random but per opportunity. This makes selection bias possible [76]. Also, we could not evaluate the implementation process of the same individual(s) for every organization at every measurement. This was partly due to ‘research fatigue’ [77]; for example schools stated they already participated in a number of research activities and therefore wanted to spread the ‘burden’ of study participation by alternating study participation among teachers across measurements. But also the complex and dynamic character of community state of affairs influenced participation; for instance supermarkets showed a high turnover of staff which made it impossible to include the same individual throughout measurements. We countered these sampling issues by ensuring that if the persons included were not similar across measurements, the function or role that the included professionals fulfilled per organization was similar. For example, at schools we always included a teacher from elementary- and middle school, and for supermarkets we always included the floor manager.

Conclusion

This study underlines the complexity of process evaluation of IACOs; the research environment is ever changing and research plans need to be constantly adapted following local developments. Moreover, a participatory action research approach should be considered to enable the swift implementation of study results into practice. Results of this study provide some leads for the formulation of implementation strategies and plans, but more research is needed to (dis)confirm these findings and their generalizability. Tailoring of implementation plans and strategies should be based on a combination of the determinants identified in this study within the context of a mutual adaptation strategy. Hence, ‘stitches in time’ are needed to allow professionals to complement and verify the tailored strategies developed throughout the implementation process.

Lessons learned
• Research plans need to be adapted iteratively to local developments;
• The translation of research findings into practice could possibly be optimized by the use of participatory action research (PAR);
• A complete, understandable IACO that is compatible with and considered relevant by practice can facilitate IACO implementation
• As some determinants appeared in configuration per sector and phase, implementation might benefit from consideration of these determinants in unity, rather than considering single determinants;
• Implementation plans and strategies should be tailored to sector- and implementation phase specific (combinations of) determinants, and should be based on a mutual adaptation strategy (“stitches in time”).

References

  1. 1.

    Schönbec Y, van Buuren S. TNO: Factsheet Vijfde Landelijke Groeistudie. Leiden, The Netherlands. 2013. https://www.tno.nl/media/1996/20100608-factsheet-resultaten-vijfde-landelijke-groeistudie1.pdf. Accessed 12 December 2014.

  2. 2.

    Ahrens W, Pigeot I, Pohlabeln H, De Henauw S, Lissner L, Molnár D, et al. Prevalence of overweight and obesity in European children below the age of 10. Int J Obes. 2014;38:S99–107.

    Google Scholar 

  3. 3.

    Ogden CL, Carroll MD, Kit BK, Flegal KM. Prevalence of obesity and trends in body mass index among US children and adolescents, 1999–2010. JAMA. 2012;307(5):483–90.

    PubMed  Google Scholar 

  4. 4.

    de Wilde JA, van DP, Middelkoop BJ, Verkerk PH. Trends in overweight and obesity prevalence in Dutch, Turkish, Moroccan and Surinamese South Asian children in the Netherlands. Arch Dis Child. 2009;94(10):795–800.

    PubMed  Google Scholar 

  5. 5.

    Singh AS, Mulder C, Twisk JW, van MW, Chinapaw MJ. Tracking of childhood overweight into adulthood: a systematic review of the literature. Obes Rev. 2008;9(5):474–88. doi: 10.1111/j.1467-789X.2008.00475.x.

  6. 6.

    Moniz M, Marques T, Cabral M, Nizarali Z, Coelho R, Monteiro A, et al. [Cardiovascular risk factors and childhood obesity]. Acta Med Port. 2011;24(2):327–32.

    PubMed  Google Scholar 

  7. 7.

    Hannon TS, Rao G, Arslanian SA. Childhood obesity and type 2 diabetes mellitus. Pediatrics. 2005;116(2):473–80. doi: 10.1542/peds.2004-2536.

  8. 8.

    Tsiros MD, Olds T, Buckley JD, Grimshaw P, Brennan L, Walkley J et al. Health-related quality of life in obese children and adolescents. Int J Obes (Lond). 2009;33(4):387–400. doi: 10.1038/ijo.2009.42.

  9. 9.

    Keating CL, Moodie ML, Swinburn BA. The health-related quality of life of overweight and obese adolescents--a study measuring body mass index and adolescent-reported perceptions. Int J Pediatr Obes. 2011;6(5–6):434–41. doi:10.3109/17477166.2011.590197.

  10. 10.

    Visscher TL, Seidell JC. The public health impact of obesity. Annu Rev Public Health. 2001;22:355–75. doi:10.1146/annurev.publhealth.22.1.355.

    CAS  PubMed  Google Scholar 

  11. 11.

    Withrow D, Alter DA. The economic burden of obesity worldwide: a systematic review of the direct costs of obesity. Obes Rev. 2011;12(2):131–41. doi: 10.1111/j.1467-789X.2009.00712.x.

  12. 12.

    Summerbell C, Hillier F. Community interventions and initiatives to prevent obesity. In: Crawford D, Jeffery RW, Ball K, Brug J, editors. Obesity Epidemiology: From Aetiology to Public Health. Oxford: Oxford University Press Inc.; 2010. p. 49–56.

    Google Scholar 

  13. 13.

    Waters E, de Silva-Sanigorski A, Hall BJ, Brown T, Campbell KJ, Gao Y et al. Interventions for preventing obesity in children. Cochrane Database Syst Rev. 2011(12):CD001871. doi:10.1002/14651858.CD001871.pub3.

  14. 14.

    Swinburn BA, de Silva-Sanigorski AM. Where to from here for preventing childhood obesity: an international perspective. Obesity (SilverSpring). 2010;18 Suppl 1:S4-S7. doi: 10.1038/oby.2009.424.

  15. 15.

    Economos CD, Irish-Hauser S. Community interventions: a brief overview and their application to the obesity epidemic. J Law Med Ethics. 2007;35(1):131–7. doi:10.1111/j.1748-720X.2007.00117.x.

  16. 16.

    Birch LL, Ventura AK. Preventing childhood obesity: what works? Int J Obes (Lond). 2009;33 Suppl 1:S74-S81. doi: 10.1038/ijo.2009.22.

  17. 17.

    Bemelmans WJ, Wijnhoven TM, Verschuuren M, Breda J. Overview of 71 European community-based initiatives against childhood obesity starting between 2005 and 2011: general characteristics and reported effects. BMC Public Health. 2014;14(1):758.

    PubMed  PubMed Central  Google Scholar 

  18. 18.

    Davison KK, Birch LL. Childhood overweight: a contextual model and recommendations for future research. Obes Rev. 2001;2(3):159–71.

    CAS  PubMed  PubMed Central  Google Scholar 

  19. 19.

    Ang Y, Wee B, Poh B, Ismail M. Multifactorial Influences of Childhood Obesity. Curr Obes Rep. 2013; 2(1):10–22.

  20. 20.

    Van Koperen TM, Jebb SA, Summerbell CD, Visscher TL, Romon M, Borys JM et al. Characterizing the EPODE logic model: unravelling the past and informing the future. Obes Rev. 2013;14(2):162–70. doi:10.1111/j.1467-789X.2012.01057.x.

  21. 21.

    Borys JM, Le BY, Jebb SA, Seidell JC, Summerbell C, Richard D et al. EPODE approach for childhood obesity prevention: methods, progress and international development. Obes Rev. 2012;13(4):299–315. doi:10.1111/j.1467-789X.2011.00950.x.

  22. 22.

    Romon M, Lommez A, Tafflet M, Basdevant A, Oppert JM, Bresson JL et al. Downward trends in the prevalence of childhood overweight in the setting of 12-year school- and community-based programmes. Public Health Nutr. 2009;12(10):1735–42. doi: 10.1017/S1368980008004278.

  23. 23.

    Van Koperen TM, Seidell JC. Overgewichtpreventie, een lokale aanpak naar frans voorbeeld. Praktische Pediatrie. 2010;2:10–4.

    Google Scholar 

  24. 24.

    (JOGG) JoGG. Factsheet JOGG. www.jongerenopgezondgewicht.nl/download/54/2013/februarifactsheet_deelconvenant_jogg.pdf. Accesed 21 may 2015.

  25. 25.

    Rogers EM. Diffusion of innovations. Fourth edition ed. New York: The Free Press; 2003.

    Google Scholar 

  26. 26.

    Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629. doi: 10.1111/j.0887-378X.2004.00325.x.

  27. 27.

    Saunders RP, Evans MH, Joshi P. Developing a process-evaluation plan for assessing health promotion program implementation: a how-to guide. Health Promot Pract. 2005;6(2):134–47. doi: 10.1177/1524839904273387.

  28. 28.

    Helfrich CD, Weiner BJ, McKinney MM, Minasian L. Determinants of implementation effectiveness: adapting a framework for complex innovations. Med Care Res Rev. 2007; 64(3) :279–303. doi: 10.1177/1077558707299887.

  29. 29.

    Peters DH, Tran NT, Adam T. Implementation research in health. a practical guide. 2013;2013.

  30. 30.

    Dusenbury L, Brannigan R, Falco M, Hansen WB. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18(2):237–56.

    PubMed  Google Scholar 

  31. 31.

    Fleuren M, Wiefferink K, Paulussen T. Determinants of innovation within health care organizations: Literature review and Delphi study. Int J Qual Health C. 2004;16(2):107–23.

    Google Scholar 

  32. 32.

    Donnell CL. Defining, Conceptualizing, and Measuring Fidelity of Implementation and Its Relationship to Outcomes in K-12 Curriculum Intervention Research. Rev Educ Res. 2008;78(1):33–84.

    Google Scholar 

  33. 33.

    Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation research: A synthesis of the literature. University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network, Tampa, FL. 2005. http://nirn.fpg.unc.edu/sites/nirn.fpg.unc.edu/files/resources/NIRN-MonographFull-01-2005.pdf. Accessed December 2013.

  34. 34.

    Wensing M, Bosch M, Grol R. Developing and selecting interventions for translating knowledge to action. CMAJ. 2010;182(2):E85-E8. doi: 10.1503/cmaj.081233.

  35. 35.

    Dobson D, Cook TJ. Avoiding type III error in program evaluation: Results from a field experiment. Eval Program Plann. 1980;3(4):269–76.

    Google Scholar 

  36. 36.

    Dreisinger ML, Boland EM, Filler CD, Baker EA, Hessel AS, Brownson RC. Contextual factors influencing readiness for dissemination of obesity prevention programs and policies. Health Educ Res. 2012;27(2):292–306.

    PubMed  Google Scholar 

  37. 37.

    Van der Kleij RM, Coster N, Verbiest M, Van Assema P, Paulussen T, Reis R et al. The implementation of intersectoral community approaches targeting childhood obesity: a systematic review. Obes Rev. 2015; Jun 16(6):454–72. doi: 10.1111/obr.12273.

  38. 38.

    van Koperen MT, van der Kleij RM, Renders CC, Crone MM, Hendriks AMA, Jansen MM, et al. Design of CIAO, a research program to support the development of an integrated approach to prevent overweight and obesity in the Netherlands. BMC Obes. 2014;1(1):5.

    PubMed  PubMed Central  Google Scholar 

  39. 39.

    Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and Policy in Mental Health and Mental Health Services Research. 2013;1–12.

  40. 40.

    Alanen S. V + ñlim + ñki M, Kaila M, Group ftES. NursesGÇÖ experiences of guideline implementation: a focus group study. J Clin Nurs. 2009;18(18):2613–21.

    PubMed  Google Scholar 

  41. 41.

    Rosman AN, Vlemmix F, Beuckens A, Rijnders ME, Opmeer BC, Mol BW, et al. Facilitators and barriers to external cephalic version for breech presentation at term among health care providers in the Netherlands: A quantitative analysis. Midwifery. 2014;30(3):e145–50.

    PubMed  Google Scholar 

  42. 42.

    Bessems KMHH, van Assema P, de Vries NK, Paulussen TWGM. Exploring determinants of completeness of implementation and continuation of a Dutch school-based healthy diet promotion programme. Int J Health Promot Educ; 5/23/2014: Routledge; 2014. p. 1–13.

  43. 43.

    Pope C, Ziebland S, Mays N. Analysing qualitative data. BMJ. 2000;320(7227):114–6.

    CAS  PubMed  PubMed Central  Google Scholar 

  44. 44.

    Miles MB, Huberman AM. Qualitative data analysis: An expanded sourcebook. Thoasand Oakes: Sage Publications; 1994.

    Google Scholar 

  45. 45.

    Schneider CQ, Wagemann C. Standards of good practice in qualitative comparative analysis (QCA) and fuzzy-sets. Int J Comp Sociol. 2010;9(3):397–418.

    Google Scholar 

  46. 46.

    Schneider CQ, Wagemann C. Set-theoretic methods for the social sciences: A guide to qualitative comparative analysis. Cambridge: Cambridge University Press; 2012.

  47. 47.

    Ordanini A, Parasuraman A, Rubera G. When the Recipe Is More Important Than the Ingredients A Qualitative Comparative Analysis (QCA) of Service Innovation Configurations. J Ser Res. 2013;1094670513513337.

  48. 48.

    Berg-Schlosser D, De Meur G, Rihoux B, Ragin CC. Qualitative comparative analysis (QCA) as an approach. Configurational comparative methods. 2009;1–18.

  49. 49.

    Pierce JL, Kostova T, Dirks KT. Toward a theory of psychological ownership in organizations. Acad Manage Rev. 2001;26(2):298–310.

    Google Scholar 

  50. 50.

    Rogers VW, Hart PH, Motyka E, Rines EN, Vine J, Deatrick DA. Impact of Let's Go! 5-2-1-0: A Community-Based, Multisetting Childhood Obesity Prevention Program. J Pediatr Psychol. 2013;38(9):1010–20.

    PubMed  Google Scholar 

  51. 51.

    Rosecrans AM, Gittelsohn J, Ho LS, Harris SB, Naqshbandi M, Sharma S. Process evaluation of a multiinstitutional community-based program for diabetes prevention among First Nations. Health Educ Res. 2008;23(2):272–86.

    CAS  PubMed  Google Scholar 

  52. 52.

    Richards Z, Kostadinov I, Jones M, Richard L, Cargo M. Assessing implementation fidelity and adaptation in a community-based childhood obesity prevention intervention. Health Educ Res. 2014;29(6):918–32.

    PubMed  Google Scholar 

  53. 53.

    Pettigrew S, Borys JM, du Plessis HR, Walter L, Huang TT, Levi J, et al. Process evaluation outcomes from a global child obesity prevention intervention. BMC Public Health. 2014;14(1):757.

    PubMed  PubMed Central  Google Scholar 

  54. 54.

    Trompette J, Kivits J, Minary L, Cambon L, Alla F. Stakeholders' perceptions of transferability criteria for health promotion interventions: a case study. BMC Public Health. 2014;14(1):1134.

    PubMed  PubMed Central  Google Scholar 

  55. 55.

    Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7(12):1–19.

    Google Scholar 

  56. 56.

    Fotu KF, Moodie MM, Mavoa HM, Pomana S, Schultz JT, Swinburn BA. Process evaluation of a community-based adolescent obesity prevention project in Tonga. BMC Public Health. 2011;11(1):284.

    PubMed  PubMed Central  Google Scholar 

  57. 57.

    Mathews LB, Moodie MM, Simmons AM, Swinburn BA. The process evaluation of It's Your Move! an Australian adolescent community-based obesity prevention project. BMC Public Health. 2010;10(1):448.

    PubMed  PubMed Central  Google Scholar 

  58. 58.

    Pate RR, Saunders RP, Ward DS, Felton G, Trost SG, Dowda M. Evaluation of a community-based intervention to promote physical activity in youth: lessons from Active Winners. Am J Health Promot. 2003;17(3):171–82.

    PubMed  Google Scholar 

  59. 59.

    Schwarte L, Samuels SE, Capitman J, Ruwe M, Boyle M, Flores G. The Central California Regional Obesity Prevention Program: changing nutrition and physical activity environments in California's heartland. Am J Public Health. 2010;100(11):2124.

    PubMed  PubMed Central  Google Scholar 

  60. 60.

    Middleton G, Henderson H, Evans D. Implementing a community-based obesity prevention programme: experiences of stakeholders in the north east of England. Health Prom Int. 2013;29(2):201–11.

    Google Scholar 

  61. 61.

    Edvardsson K, Ivarsson A, Garvare R, Eurenius E, Lindkvist M, Mogren I, et al. Improving child health promotion practices in multiple sectors: outcomes of the Swedish Salut Programme. BMC Public Health. 2012;12(1):920.

    PubMed  PubMed Central  Google Scholar 

  62. 62.

    Huberty JL, Dodge T, Peterson K, Balluff M. Activate Omaha: the journey to an active living environment. Am J Prev Med. 2009;37(6):S428–35.

    PubMed  Google Scholar 

  63. 63.

    Davis SM, Sanders SG, FitzGerald C, Keane PC, Canaca GF, Volker-Rector R. CHILE: An Evidence Based Preschool Intervention for Obesity Prevention in Head Start. J Sch Health. 2013;83(3):223–9.

    PubMed  PubMed Central  Google Scholar 

  64. 64.

    Levine E, Olander C, Lefebvre C, Cusick P, Biesiadecki L, McGoldrick D. The Team Nutrition pilot study: lessons learned from implementing a comprehensive school-based intervention. J Nutr Educ Behav. 2002;34(2):109–16.

    PubMed  Google Scholar 

  65. 65.

    Edvardsson K, Garvare R, Ivarsson A, Eurenius E, Mogren I, Nystrøm ME. Sustainable practice change: Professionals' experiences with a multisectoral child health promotion programme in Sweden. BMC Health Serv Res. 2011;11(1):61.

    PubMed  PubMed Central  Google Scholar 

  66. 66.

    Gombosi RL, Olasin RM, Bittle JL. Tioga County Fit for Life: a primary obesity prevention project. Clin Pediatr. 2007;46(7):592–600.

    Google Scholar 

  67. 67.

    Smith A, Coveney J, Carter P, Jolley G, Laris P. The Eat Well SA project: an evaluation-based case study in building capacity for promoting healthy eating. Health Prom Int. 2004;19(3):327–34.

    Google Scholar 

  68. 68.

    Goodman RM, Steckler A. A model for the institutionalization of health promotion programs. Fam Community Health. 1989;11(4):63–78.

    Google Scholar 

  69. 69.

    Urquhart R, Porter GA, Sargeant J, Jackson L, Grunfeld E. Multi-level factors influence the implementation and use of complex innovations in cancer care: a multiple case study of synoptic reporting. Implement Sci. 2014;9:121.

    PubMed  PubMed Central  Google Scholar 

  70. 70.

    Miller DT, Ross M. Self-serving biases in the attribution of causality: Fact or fiction? Psychol Bull. 1975;82(2):213.

    Google Scholar 

  71. 71.

    Green CA, Duan N, Gibbons RD, Hoagwood KE, Palinkas LA, Wisdom JP. Approaches to Mixed Methods Dissemination and Implementation Research: Methods, Strengths, Caveats, and Opportunities. Adm Policy Ment Health. 2014;1–16.

  72. 72.

    Stringer ET. Action research. Sage Publications; 2013.

  73. 73.

    Cook WK. Integrating Research and Action: A Systematic Review of Community-based Participatory Research To Address Health Disparities In Environmental and Occupational Health in the United States. J Epidemiol Community Health. 2008;62(8):668–76. doi:10.1136/jech.2007.067645.

    CAS  PubMed  PubMed Central  Google Scholar 

  74. 74.

    Patton MQ. Qualitative Research. John Wiley & Sons, Ltd: Encyclopedia of Statistics in Behavioral Science; 2005.

    Google Scholar 

  75. 75.

    Alvesson M, Sk + ¦ldberg K. Reflexive methodology: New vistas for qualitative research. Sage; 2009.

  76. 76.

    Collier D, Mahoney J. Insights and pitfalls: Selection bias in qualitative research. World Politics. 1996;49(01):56–91.

    Google Scholar 

  77. 77.

    Clark T. We're Over-Researched Here!'Exploring Accounts of Research Fatigue within Qualitative Research Engagements. Sociology. 2008;42(5):953–70.

    Google Scholar 

Download references

Acknowledgment

The authors would like to acknowledge ZonMw (the Netherlands Organisation of Health Research and Development) for the funding provided for this research project. The authors would also like to acknowledge all professionals who have invested significant time and effort into study participation.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Rianne MJJ van der Kleij.

Additional information

Competing interests

The authors declare that no competing interests are present.

Authors’ contributions

RvK, MC, TP and RR conceived and designed the study. RvK was in charge of the data collection, data entry and analysis. TP, MC and RR supervised all stages of the research process. The manuscript was written by RvK; TP, VvdG, MC and RR drafted and revised the manuscript and approved the final manuscript. All authors read and approved the final manuscript.

Authors’ information

Not applicable.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

van der Kleij, R.M., Crone, M.R., Paulussen, T.G. et al. A stitch in time saves nine? A repeated cross-sectional case study on the implementation of the intersectoral community approach Youth At a Healthy Weight. BMC Public Health 15, 1032 (2015). https://doi.org/10.1186/s12889-015-2306-0

Download citation

Keywords

  • Childhood obesity
  • Intersectoral community approach
  • Implementation
  • Qualitative methods
  • Process evaluation