- Research article
- Open Access
- Open Peer Review
Effectiveness of a systematic approach to promote intersectoral collaboration in comprehensive school health promotion-a multiple-case study using quantitative and qualitative data
BMC Public Healthvolume 15, Article number: 613 (2015)
We report on the longitudinal quantitative and qualitative data resulting from a two-year trajectory (2008–2011) based on the DIagnosis of Sustainable Collaboration (DISC) model. This trajectory aimed to support regional coordinators of comprehensive school health promotion (CSHP) in systematically developing change management and project management to establish intersectoral collaboration.
Multilevel analyses of quantitative data on the determinants of collaborations according to the DISC model were done, with 90 respondents (response 57 %) at pretest and 69 respondents (52 %) at posttest. Nvivo analyses of the qualitative data collected during the trajectory included minutes of monthly/bimonthly personal/telephone interviews (N = 65) with regional coordinators, and documents they produced about their activities.
Quantitative data showed major improvements in change management and project management. There were also improvements in consensus development, commitment formation, formalization of the CSHP, and alignment of policies, although organizational problems within the collaboration increased. Content analyses of qualitative data identified five main management styles, including (1) facilitating active involvement of relevant parties; (2) informing collaborating parties; (3) controlling and (4) supporting their task accomplishment; and (5) coordinating the collaborative processes.
We have contributed to the fundamental understanding of the development of intersectoral collaboration by combining qualitative and quantitative data. Our results support a systematic approach to intersectoral collaboration using the DISC model. They also suggest five main management styles to improve intersectoral collaboration in the initial stage. The outcomes are useful for health professionals involved in similar ventures.
Comprehensive school health promotion (CSHP) is endorsed by the WHO because of its broad perspective on children’s health, involving social and physical environments in and around school, community activity and redesigning health services. Moreover, it has been recognized as an effective means to improve children’s health and well-being [1–4].
The Dutch equivalent of CHSP is the Dutch Healthy School Approach (HSA). Basically, HSA targets demand-driven practices in school health promotion based on the epidemiological data of the school community, a prioritization of school needs, an assessment of important and modifiable determinants, the drafting and implementation of a multi-year school health plan, and its evaluation. All this is done jointly with multiple stakeholders who provide different expertise and fulfill different tasks [5–7]. At school level, the implementation of the HSA is professionally assisted by a ‘health promoting school advisor’, who represents various public services and providers (e.g. from the welfare, health, prevention and safety sectors) in individual contacts with schools. At local and regional levels, the public health services (PHSs) function as a linking pin (coordinator) between the education sector, health authorities and public services stakeholders (PSSs). Their coordinating role derives from a legal responsibility for the implementation of local public health policy and youth health care financed by the municipality .
While the popularity of the HSA and its adoption in Dutch regions is growing , its implementation remains complex [10, 11]. The establishment of a functional structure for the collaboration between the sectors involved poses important challenges in practice [12, 13]. The necessity of partnerships and networking and related challenges are not unique for the Dutch context, but are also acknowledged by studies in other countries in and outside Europe [14–17]. The increasing understanding of public health schools as social adaptive systems stimulates the study of intersectoral collaboration in CSHP. This perspective puts an emphasis on the need of joint work between relevant actors at different levels (individual, organizational, policy level) and from various sectors (health, education and authorities) to successfully implement CSHP [18, 19].
In the study at hand we investigated a systematic approach to the development of intersectoral collaborations based on a thorough analysis of facilitating and impeding factors and the use of appropriate change strategies. We studied the impact of this systematic approach quantitatively in a two-year trajectory supporting health professionals in its implementation. We also aimed to open up the ‘black box’ of collaborative processes, since collaboration research has found that these interactive processes are generally not very well understood [20, 21]. We therefore explored qualitatively if, and if so which, managerial activities were induced by the systematic approach. The benefits of a mixed methods approach (including qualitative and quantitative data) over qualitative or quantitative research alone have been acknowledged in inquiries evaluating health promotion in schools  and studying collaboration in public health . In our case, presenting both types of results in one paper helps to understand whether a systematic approach positively influences intersectoral collaboration in the HSA.
The specific collaborative structure involving relevant stakeholders from the education sector, health authorities, PHSs, and PSSs in HSA has been studied by Leurs et al. . Inspired by the Integrated Care model, which is rooted in organizational theories [24–26] and which advocates a systematic approach to intersectoral collaboration in integrated health care [27, 28], Leurs et al. [12, 13] developed a diagnostic framework for intersectoral collaboration in the HSA, called the DIagnosis of Sustainable Collaboration (DISC) model. Using the DISC questionnaire as an assessment tool and semi-structured interviews to triangulate the results of the DISC questionnaire, Leurs et al.  diagnosed the collaborative structure for HSA in the southern parts of the Netherlands. Their study firstly validated the importance of the DISC factors and secondly suggested that DISC analysis can be leading in the implementation of strategies to strengthen collaboration between the sectors involved. This practice orientation makes the model very promising for the study of a systematic approach to intersectoral collaboration in CSHP.
The DISC model summarizes important factors into six main clusters that impact on the intended change and its sustainability: collaborative support (including the perceptions, intentions, and actions subclusters), change management, project management, context, external factors and sustainability (called ‘CSHP’ in original model). In the DISC model, stakeholders’ collaborative support refers to the extent to which major stakeholders agree on the problems that need to be solved and to the degree of consensus on the actions to be taken, outcomes to be achieved and boundaries to be established between the domains of various partners. In addition, it refers to stakeholders’ level of commitment to and trust in each other’s intentions to effectuate actual change (allocation of resources and manpower, adaptations to ensure sustainability, etc.) within the organization they represent (rather than acting out of self-interest only).
Change management refers to the need to gradually develop a common vision of the innovation which then guides and facilitates the selection of a change perspective and change strategies, and, when necessary, the extension of the network with new actors required for the desired system change. Four perspectives for change and related change strategies have been suggested [30, 31, 27].
In the developmental and implementation phase, a project management structure is needed to steer the actors involved towards the accomplishment of the goals. This structure becomes self-supportive as soon as collaboration becomes formalized in the work routines of the parties involved.
The DISC model further assumes that the context of the stakeholders involved and several external factors influence the collaborative process, while especially the later factor itself is difficult to influence. The context supports the collaborative process when the actors involved have positive experiences with previous collaborations, have enough research power, interest and expertise in the relevant field, and recognize relevant policies which they can also influence. External factors include intersectoral policies, laws, and regulations at different levels (EU, local, regional, and national) relevant to HSA, encouraging attitudes of financing bodies and positive community notions and public interest in CSHP. The sixth cluster, sustainability, addresses the HSA itself and refers to its sustainability, with “sustainability” defined as the gradual establishment of a stable collaboration.
The relation between the DISC constructs is based on preliminary assumptions from Leurs et al.’s  single case study and relevant literature. Which managerial principles induce improvements on specific DISC factors has not yet been established.
Using this conceptual framework, we have studied the effectiveness and managerial processes of a systematic approach based on the DISC model in the two-year DISC-based trajectory.
The DISC-based trajectory
The trajectory started with DISC analyses, offering a thorough analysis of facilitating and impeding factors in five regional HSA collaborations spread across the Netherlands. The analyses showed that stakeholders’ positive perceptions about the HSA needed to be more fully translated into positive intentions and real actions and that change management and project management were hardly applied, even though they were considered crucial for this transition . Based on these diagnoses, we advised the regional coordinators of the five HSA collaborations to focus on the development of effective change management and project management. In addition, and in accordance with the operationalization of the DISC model, we supported them for the period of one year in accomplishing the following steps: (1) bringing the different partners together; (2) developing a common vision; (3) identifying the possibilities and impossibilities for each collaborating partner based on the DISC analysis; (4) determining the desired collaborative structure; (5) developing a task distribution based on the information; (6) discussing the choices at the management and executive levels. Support included monthly personal/telephone contacts with the central facilitator involved in the trajectory. This facilitator was a health educator working at the PHS where the Dutch HSA and the DISC model had been developed and who was specialized in the implementation of the HSA. The facilitator provided support which basically comprised critical reflection on the collaborative process using a standard interview protocol, exchange of expertise between regions, with the facilitator as intermediary between them, and occasional peer-review of produced documents. In addition, one joint training session for regional coordinators was organized which addressed the regional coordinator’s role as process facilitators within the HSA.Footnote 1 After support had ended, the collaborative process was monitored for an additional year using the same standard interview protocol during bi-monthly phone calls with regional coordinators. These calls were made by a researcher affiliated to Maastricht University who had also conducted the DISC analyses in the regions. The monitoring data were later used for the analysis of managerial activities induced by the systematic approach. After the one-year follow-up period, the DISC-based trajectory ended with a second round of DISC analyses which, together with the results of the first round, were used for the purpose of effect evaluation.
Procedure and data collection instruments
Longitudinal quantitative data on the DISC factors and qualitative data on the managerial activities were collected in five of 30 (in 2008) Dutch PHS regions, from 2008 to 2011. Inclusion criterion was the ability (e.g. resources, other priorities) and willingness to start with the DISC-based trajectory at the end of 2008.
For the quantitative data collection, coordinators of the HSA working at the PHSs were asked to identify relevant stakeholders from the education sector, municipal authorities, PHSs and other PSSs. The coordinators introduced the topic of research to these persons and distributed materials for the survey (the questionnaire and a brief summary of the HSA) at the end of 2008. Data collection lasted for a total of eight weeks. Reminders were sent after an additional 4 weeks. The procedure was repeated in 2011 as a posttest.
The longitudinal quantitative data were collected using the DISC questionnaire (adapted from Leurs et al. ) which measures the 26 DISC factors of the 6 clusters (collaborative support [including the perceptions, intentions, and actions subclusters], change management, project management, context, external factors, and sustainability), with 1 to 5 items each, mostly on a 5-point scale (completely disagree: 1 to completely agree: 5) with the additional option of ‘unknown’. Examples of items used in the questionnaire are ‘I think it is important that my organization participates in the HSA’ and ‘I think the HSA is of interest to my own organization’ (collaborative support–perceptions).
In the early months of 2009, the results of the quantitative pretest and the resulting recommendations (i.e. the six predefined steps) were presented to the regional coordinators in a regional report, followed by a personal meeting for further clarification between each of the regional coordinators and the facilitator and researcher. At these meetings, all regional coordinators independently indicated that priority should be given to correctly situating the HSA within their own organization and positioning it together with PSSs in the region. Consequently, managerial activities carried out during the trajectory mainly targeted the representatives of PHSs and PSSs, so only quantitative and qualitative data concerning these stakeholders are included in the current paper.
During the one-year period of support (mid 2009-mid 2010) and the one-year follow-up period (mid 2010-mid 2011), qualitative data on the collaborative process were collected using a standard interview protocol including the following questions: Which activities have been conducted for the purpose of collaboration? Which activities are planned for the purpose of collaboration? Have any documents been produced in relation to the collaboration? Are there any other developments influencing the collaborative process? Minutes were made of these interviews and sent to regional coordinators for approval. Relevant documents such as project proposals and project plans (multiyear plans and work plans) were collected. Nvivo analysis was used to extract managerial activities from the interview minutes and the collected documents, in order to open up the black box of collaborative processes in the DISC-based trajectory.
By the end of 2011, after the second round of DISC analyses, the regional coordinators received a final report with the regional results.
In accordance with Dutch regulations, no ethical approval was required for this study .
Participating PHS regions
Five cases of HSA collaboration were studied, with different characteristics and starting situations. Table 1 lists these for each of the five PHS regions, in ascending order of size. We decided to study intersectoral collaboration in a variety of situations, to be able to collect more data than would have been possible in a single case study, and to enhance the generalizability of the study results.
A total of 158 stakeholders at pretest and 132 potential stakeholders at posttest were approached by the regional coordinators for the DISC questionnaire. Contacted PHS professionals included health promoters, epidemiologists, pediatricians, and youth nurses. PSSs came from the domains of addiction, mental health, social welfare, and security, and other services like educational support services, dietician centers, and sports companies. The five regional coordinators who were followed and interviewed during the advice trajectory all worked at PHSs. All of them were health promoters.
Quantitative data analyses
During data cleaning, the ‘unknown’ option was recoded to ‘1’ (negative value) as it apparently indicated the absence of a particular DISC factor. Missing values for one item in scales with four or more items were replaced by the mean of the other items.
Cronbach’s alphas were calculated for the different scales operationalizing the DISC factors. These appeared to be lower than .60 for three of the 13 multi-item scales: willingness to change, organizational characteristics, and characteristics of CSHP (Table 1). The items of these scales were therefore included separately in the analyses. Descriptive statistics were used to establish the mean values for the DISC factors at both measurements.
Multi-level analyses with linear mixed regression including a random intercept were performed for the five HSA collaborations to analyze changes in staff and organizations that occurred between pretest and posttest. The random intercept model enabled us to include in the analyses both respondents with complete data and respondents who participated at pretest or posttest only. This allowed for a more reliable estimation of the general trend than an analysis based only on complete cases.
To test whether relevant measures had improved significantly at posttest, multi-level analyses (F-test) were conducted with time of measurement (pretest versus posttest) as the fixed factor and one of the DISC factors as dependent variable. We corrected for multiple testing using Holm’s method, by first ranking the p-values of the analyses by magnitude and then multiplying them by their rank number (where the largest p-value gets the lowest rank and the smallest p-value gets the highest rank). Ranking was performed for all factors within a cluster (e.g. change management). In the case of subclusters, ranking was performed for all factors within a subcluster (e.g. stakeholder’s perceptions).
In addition we calculated effect sizes based on the intercept and the error variances for each measure, in order to weight the effects. The effect sizes were interpreted according to Cohen’s  categorization, in which d = 0.20, 0.50 and 0.80 represent small, medium, and large effects, respectively.
Statistical analyses were conducted using SPSS 20.
Qualitative data analyses
Document analysis included 65 min of the standard protocol interviews with the regional coordinators, 39 of which were held by the facilitator and 26 by the researcher, and the project proposals and project plans regarding the HSA from each region. Document analysis with Nvivo 9 was independently performed by two researchers, the researcher who conducted the DISC analyses and a research assistant with expertise on the topic. Ambiguities of coding were resolved by discussion. Data from documents were mainly used to verify regional coordinators’ self-reports on managerial activities. Initial coding focused on the identification of managerial activities. Several iterations of analyses resulted in a classification scheme of managerial activities.
We triangulated our data by studying information from various types of sources (qualitative and quantitative data) and by having the coding of the qualitative data done by two independent researchers .
Of the total of 158 stakeholders who were approached at pretest and 132 at posttest, 90 (57 %) and 69 (52 %), respectively, were included in the analyses. There were various reasons for dropping out. Organizational tasks of collaborating parties changed in response to mergers, reorganizations, government cutbacks and changing policies. Also, collaborative goals became more specific as collaboration progressed, which led to a poorer fit between the agenda of some organizations and the collaborative agenda.
Table 2 indicates that the regional coordinators had set slightly different priorities within the collaboration, since the numbers of representatives from the two sectors who were approached varied between regions.
What are the pretest-posttest differences regarding the DISC factors?
Our results indicate significant differences between pretest and posttest in all six DISC categories (Table 3).
Significant differences between pretest and posttest were found, with medium effect size. At posttest, the parties involved had more shared views about ways to realize the HSA (perception of ideological consensus), had greater intentions to commit to the collaborative goal (intention to commit) and reported more formalization of the HSA within their organizational policies (action of formalization).
Change management improved significantly, with gains of almost medium effect size, specifically regarding the presence of a clear and systematically developed vision, the innovative perspectives pursued and the change strategies employed.
Respondents evaluated the project management more favorably at posttest than at pretest. Gains were small, though almost of medium magnitude. Further exploration of project management showed that it was especially the awareness among stakeholders involved and their responsibilities which had increased.
Significant improvements of medium size were found for the external context. According to the respondents, the HSA appeared to fit in with school health policies and public health policies (policies and regulations).
The internal context for the implementation of the HSA, however, had deteriorated at posttest. The number of problems within their own organization increased, an effect of medium magnitude.
Finally, respondents evaluated CSHP significantly better at posttest than at pretest. At posttest, they perceived the HSA more as practical and less as research (ES >0.50) or as theory (ES <0.20). Furthermore, the HSA was evaluated more as permanent and less as ad-hoc (ES <0.20). In addition, respondents perceived less fragmentation of school health promotion and were more aware of the existence of a single service point at PHSs for health promotion programs and expertise. This effect was almost of medium effect.
How was the DISC-based trajectory operationalized by the regional coordinators?
We identified a great variety of working methods and instruments that the regional coordinators applied in response to the DISC-based advice, which clustered into five main management styles (Fig. 1). Below we summarize how these management styles were used in practice.
Regional coordinators created spaces for negotiation between stakeholders about a common vision, a possible collaborative structure and a suitable division of tasks. They used different working methods and instruments to induce people to think about these aspects and to give everybody a voice in the decision-making process. They involved stakeholders through discussions and brainstorm sessions, and encouraged their involvement by means of prepared agendas, short questionnaires, DISC results or assignments. An example of an assignment used for the systematic development of a common vision is shown in Table 4. Based on the results of the individual assignments that had been completed by each stakeholder, the regional coordinators came up with a proposal for a common vision. Comparable assignments were used to compare financing systems and policy cycles between collaborating parties.
Additionally, they stimulated active participation by making collaborating partners responsible for certain tasks through the formation of steering groups and working groups. They also created moments for feedback on minutes, proposals, predefined formats/scenarios, and plans, to create a common perspective and commitment, and they tailored agreements to the possibilities and impossibilities of collaborating parties to facilitate alignment of policies. For example, when stakeholders were unable to commit to long-term agreements, then short-term agreements about investments and task distributions were made instead.
Regional coordinators shared plans with distal stakeholders (i.e. municipal authorities, PSS managers), raised awareness of HSA among ‘potential’ stakeholders and provided everybody who worked with the HSA (distal and proximal stakeholders) with relevant updates and tools. For example, regional coordinators informed stakeholders at local and regional levels through newsletters and websites. In addition, they networked with relevant informants during conferences and symposia. They held presentations during persuasive and information meetings for potential partners and municipalities. They also assigned stakeholders to be responsible for the distribution of information in their organization and their networks. Finally, they used minutes of meetings as a source of commitment and progress tracking for closely involved stakeholders.
Planning and control systems were set up to inform the collaboration about whether the plans that had been made would bring about the desired effects, whether the resources allocated (e.g. staff, time) were realistic, but also to identify deviations at operational level and to intervene in time. The regional coordinators employed different systems at different levels. For example, at operational level, they used available registration systems to monitor and evaluate activities (e.g. the number of schools visits performed by the health promoting school advisors, the time they dedicated to school visits) and worked with action lists which they used as checklists during meetings. At tactical level, the coordinators used different types of evaluations (e.g. end-of-year evaluations) to check whether the intended outcomes and budgets were met.
The coordinators organized different types of support. At operational level, they used monitoring in combination with active coaching. For the monitoring part, they used information from the registration system, from action lists and different types of evaluations. For active coaching, they used tools from the 2010 practitioner guide , including predefined formats and scenarios for discussions with particular stakeholders, and provided these on a platform that was easily accessible to health professionals. The regional coordinators also encouraged participation in training courses and discussion of group norms to strengthen the team spirit (e.g. supporting each other in the attainment of the goals established). At local level, regional coordinators facilitated demand-driven practices among collaborating parties by establishing appropriate financial and other arrangements with municipalities (and possibly PSSs). At managerial level, the regional coordinators created clearly defined agreements about responsibilities and manpower, to prevent resistance and fears among the representatives involved.
More active forms of control and supportive behaviors were used in collaborations between organizations that fall under one umbrella organization, with clear agreements on resources and manpower and one management (e.g. youth health care and public health officials working at PHSs).
Coordinating activities were central to all other management activities. Coordination included an interactive, integrative process of collecting information, interpreting it, determining knowledge requirements, outlining the next steps and elaborating ways in which results could best be presented to the parties involved (e.g. proposals, predefined formats), followed by formalization of final decisions (e.g. multi-year plans).
The current study had two aims: first to investigate the effectiveness of a systematic approach to the development of intersectoral collaboration in CSHP and second to open up the ‘black box’ of the collaborative processes involved. These aspects were investigated using the data from a two-year DISC-based trajectory. We examined longitudinal quantitative data from the trajectory to find out whether collaboration had improved in terms of the DISC factors after participation in the DISC-based trajectory (research question 1), and studied the longitudinal qualitative data to assess whether managerial activities were employed by regional coordinators in response to the trajectory, and if so which ones (research question 2). Presenting both results, quantitative and qualitative, in one paper provides additional understanding of the origins of the observed effects.
As regards our first research question, the quantitative data showed remarkable improvement (of almost medium effect size) in change management and project management. In addition, gains of medium magnitude were found on the three subclusters of collaborative support (i.e. perceived consensus, intention to commit, and formalization), as well as on one measure of external factors (i.e. perceived alignment between policies), and on four measures of sustainability (i.e. theoretical vs. practical, research vs. practical, ad-hoc vs. systematic, fragmentation vs. single service point). These improvements were found despite an increase in organizational problems reported by stakeholders involved (due to e.g. mergers, reorganization, financial cutbacks). As regards our second research question, our qualitative data showed that regional coordinators undertook different activities in response to the DISC-based trajectory, activities which clustered into five managerial styles and basically addressed (1) involving stakeholders in the decision-making process, (2) informing them about decisions and progress made, (3) controlling and (4) supporting their task accomplishment, (5) and coordinating the collaborative process.
The improvements that we found in terms of change management, project management, and collaborative support seem to be the result of having encouraged regional coordinators to identify common grounds with the stakeholders regarding several aspects. These aspects included the establishment of a common vision, a common collaborative structure, and a suitable task distribution, while considering stakeholders’ individual interests. In this respect our qualitative data provide some indications that regional coordinators have created the necessary opportunities for stakeholders to freely voice their suggestions, doubts, and wishes regarding these aspects (the involving management style). Similar findings have been reported by others, who found that creating spaces for negotiation (e.g. brainstorming, discussion) can enhance the consensus on collaborative goals and necessary actions, and thereby promote the formulation of commitments in collaborations [35, 20, 36].
The same reasoning seems to apply to the increased degree of formalization we have found at posttest. We advised regional coordinators to discuss choices at the management and executive levels. The qualitative data indeed indicate that regions consolidated agreements in documents at different levels (the coordinating management style). Bohlmeijer et al.  reported that written documents can be regarded as the visible results of the negotiations and are an important indicator of formalization. Koelen et al.  showed that such formalization builds employees’ accountability, which acts as an important driving force for action in Dutch health organizations.
As an adverse development, we have observed that the internal context of the parties involved deteriorated. It is conceivable that this result is related to the internal developments (e.g. mergers, reorganizations) that the organizations were going through during the trajectory, in response to government cutbacks due to the financial crisis in the Netherlands. In addition, it could be the consequence of the new four-year public health policy cycle which organizations had entered in 2011 and which obliged them to translate new national health promotion objectives into concrete strategies to improve health at local level. We observed that these dynamics often forced collaborating parties to make choices which were not in interest of the collaboration. Time and staff were mainly dedicated to the reorientation. Legislation changed, as did organizational tasks. Some PSSs left the collaboration, others reduced their investments. The reconciliation of collective and individual interests has been reported by others as a recurring dilemma in collaborations. While stakeholders achieve collaborative goals, they also have to fulfil the individual organization’s mission and respond to organizational developments and problems [20, 21, 37–42]. This finding may explain the state of the collaboration after three years, where collaboration shows no improvements in terms of collaborative actions as yet. Even if agreements were reached, this seemed to require much more time.
Next to the effort to reconcile individual and collective interests, our qualitative and quantitative data strongly suggest that regional coordinators recognized relevant policies for intersectoral collaboration and tried to influence them to promote collaboration. Regional coordinators used the ‘windows of opportunity’  that emerged from the dynamic context for this purpose. They placed the HSA and the collaborative structure on the agenda of meaningful internal and external meetings (the informing management style). They made connections and opportunities visible to relevant actors (the coordinating management style). They developed supporting policies and financial agreements between municipal authorities and PHSs (the supporting management style). Our quantitative data confirm greater alignment between health policies, school policies, and the HSA after the trajectory (external factors). De Leeuw  reported comparable skills by ‘social entrepreneurs’ who were able to influence policy agendas. These social entrepreneurs had the abilities to obtain an overall view of the various perspectives of stakeholders, to broker commitments of stakeholders into networks and to reflect on their own position and that of the stakeholders. They influenced the policy agenda by bringing problems, solutions, and the right stakeholders together.
By contrast, our study shows that directive behaviors, which focus on task performance, such as planning and control systems (the controlling management style) were less commonly employed, and when they were used, it was mostly in organizations that fall under one umbrella organization. This probably indicates the difficulty of employing this type of behavior in collaborations lacking formal authority [35, 41].
Finally, it is encouraging to find that stakeholders expressed more favorable judgments about the sustainability of the collaboration at posttest (i.e. more systematic, more practical, more demand-driven practices). Although the absence of comparable studies makes it difficult to compare our findings, there are studies that support the view that a systematic, step-wise approach to change can give direction and transparency [35, 44–46]. A study involving the provision of professional support for school staff to implement the CSHP acknowledged that professional support can enhance the acquisition of organizational knowledge and its translation into practice . Lastly, the finding that school health promotion developed from fragmentation towards a single service point for health promotion (i.e. demand-driven practices) provides additional evidence that collaborative efforts start to pay off and contribute to collaborative goals.
Based on the above interpretations, it is plausible to postulate that regional coordinators employed managerial activities in response to the DISC-based trajectory, which have contributed to the observed improvements in terms of the DISC factors. In this respect the combination of the qualitative and quantitative data was an important strength of our study. It allowed data triangulation and a combined study of effects and processes. This in turn provided important insights into the causes of the observed effects. This strength partly offsets the weakness of the quantitative results based on a pretest-posttest design, which as such limited the opportunities to draw causal inferences due to possible history and maturation biases and Hawthorne effects . In addition, data source and investigator triangulation, as well as the member check of analyzed minutes contributed to the objectivity (i.e. confirmability) and credibility of our findings. Furthermore, studying multiple cases gave us the opportunity to collect qualitative data from various cases, which differed in several characteristics and staring situations, and thereby increased the transferability of our conclusions. Finally, we tried to enhance readers’ judgments about the dependability of qualitative findings through a thorough description of the inquiry process and interpretation of findings against the context of the studied collaboration [49, 50]. Nevertheless, some limitations to our study should be considered. Despite being based on multiple cases, our quantitative research suffered from a small sample size, due to the small number of stakeholders involved. In addition, drop-out affected our sample size, though we minimized its detrimental effects by applying suitable analysis techniques (i.e. including all available cases in the analysis). Furthermore, selection bias may have affected our data because of the voluntary participation of PHS regions in the DISC-based trajectory and because the stakeholders were not randomly selected. In addition, regional coordinators decided to give priority to a particular sector in the collaboration. The desire to achieve their own organizational goals (e.g. youth health promotion) may have led their choices. Finally, the data gathered for the purpose of support may not necessarily capture all managerial activities that the regional coordinators employed, so our overview might not be exhaustive and might need further elaboration. In addition, the stage of the collaboration limited the number of managerial activities that could be studied, activities which can help manage the transition from formalization to collaborative action. These activities will thus need special attention in follow-up studies.
To conclude, our conclusion study provides preliminary evidence for the effectiveness of a systematic approach to intersectoral collaboration based on the DISC model. In addition, it offers important insights into the black box of managerial processes related to this kind of approach.
Our findings suggest that when collaboration is to be established in a systematic way, a thorough analysis of facilitating and impeding factors can help to formulate an appropriate strategy. In addition, it seems to indicate that each step of this strategy should provide enough space for negotiation and be concluded with formalized agreements, so that employees are made accountable for collaborative actions. Support for this process can probably be provided by partnership managers who can devise creative working methods and instruments to manage the collaboration, and who have the skills to function as social entrepreneurs in dynamic contexts. The byproducts of our study are a strategy to improve the change and project management in intersectoral collaboration and a large repertoire of managerial working methods and instruments. Both might inspire health professionals engaged in comparable partnership endeavors. Assistance on demand might be necessary to support them in the acquisition, assimilation, and application of the new knowledge.
The training was based on a national supportive manual for health professionals involved in the HSA (RIVM Centrum Gezond Leven  and co-organized by the Netherlands School of Public and Occupation Health.
- DISC model:
Diagnosis of sustainable collaboration model
Comprehensive school health promotion
Healthy school approach
Public health services
Public service stakeholders
Lister-Sharp D, Chapman S, Stewart-Brown S, Sowden A. Health promoting schools and health promotion in schools: two systematic reviews. Health Technol Assess. 1999;3 suppl 22:1–207.
IUHPE. Achieving health promoting schools: guidelines for promoting health in schools. 2009. http://www.dhhs.tas.gov.au/__data/assets/pdf_file/0011/115895/guidelines_for_health_promoting_schools1.pdf. Accessed 15 Jan 2015.
Stewart-Brown S. What is the evidence on school health promotion in improving health or preventing disease, and specifically, what is the effectiveness of the health promoting schools approach? WHO Regional Office for Europe: Copenhagen; 2006.
St Leger LH. The opportunities and effectiveness of the health promoting primary school in improving child health-a review of the claims and evidence. Health Education Reserach. 1999;14 suppl 1:51–69.
Buijs GJ. Werkdocument. De gezonde school methode in nederland. (Work document. The healthy school approach in the Netherlands. Woerden: National Instituut voor Gezondheidsbevordering en Ziekte preventie (Netherlands Institute for Health Promotion and Disease Prevention); 2005.
Leurs M, Steenbakkers M, Jansen M. Het schoolSlag-praktijkboek: samen werken aan preventie op maat in het onderwijs (The schoolBeat manual: working together towards tailored prevention in education). GGD Zuid Limburg: Maastricht; 2006.
RIVM Centrum Gezond Leven (National Institute for Public Health and the Environment Centre of Healthy Living). Handleiding Gezonde School (Healthy School Guide). Bilthoven; 2010.
Hirsch Ballin EMH. Wet Publieke Gezondheid (Public Health Act). In: Overheid.nl (Government.nl). 2008. http://wetten.overheid.nl/BWBR0024705/geldigheidsdatum_14-04-2015. Accessed 14 April 2015. Accessed 19 Dec 2014.
Bos V, De Jongh D, Paulussen T. Gezondheidsbevordering en preventie in het onderwijs. Stand van zaken, effectiviteit en ervaringen van GGD'en en scholen (Health promotion and prevention in schools. Current state of affairs, effectiveness and experiences from public health services and schools.). Bilthoven: RIVM; 2010
Boot MWM, van Assema P, Hesdahl B, de Vries NK. Professional assistance in implementing school health policies. Health Educ. 2010;110 suppl 4:294–308.
Boot MWM, van Assema P, Hesdahl B, Leurs M, de Vries NK. Gezondheidsbevordering en voortgezet onderwijs: verstandshuwelijk of echte liefde? (Health promotion and secondary education: marriage of convenience or true love?). Tijdschrift voor Gezondheidswetenschappen. 2010;88(3):127–35.
Leurs M, Mur-Veeman I, Schaalma H, De Vries N. Integrale samenwerking gaat verder dan de samenwerking tussen professionals: introductie van het DISC-model (Integrated collaboration continues beyond professional collaboration: introduction of the DISC-model). TSG. 2003;81(suppl):369–72.
Leurs M, Mur-Veeman I, van der Sar R, Schaalma H, de Vries N. Diagnosis of sustainable collaboration in health promotion-a case study. BMC Public Health. 2008. doi:10.1186/1471-2458-8-382.
Noble C, Toft M. The english national healthy school programme 1999–2011. In: Samdal O, Rowling L, editors. Implementation of health promoting schools: exploring the theories of what, why and how. London: Routledge; 2013. p. 112–6.
Rowling L, Legder E. Australia: The role of intermediary organisations in the development of health promoting schools. In: Samdal O, Rowling L, editors. Implementation of health promoting schools: exploring the theories of what, why and how. London: Routledge; 2013. p. 116–20.
Dadaczynski K, Paulus P. Germany, Anschub.de-Alliance for sustainable school health and education. In: Samdal O, Rowling L, editors. Implementing Health promoting schools: exploring the what, why and how. London: Routledge; 2013. p. 139–42.
Inchley J, Guggelberger L, Young I. From health to education: the journey towards health promoting school implementation in Scotland. In: Samdal O, Rowling L, editors. Implementing health promoting schools: the what, why and how. London: Routledge; 2013. p. 142–6.
Keshavarz N, Nutbeam D, Rowling L, Khavarpour F. Schools as social complex systems: a new way to understand the challenges of introducing the health promoting school concept. Soc Sci Med. 2010;70 suppl 10:1467–74.
Pucher KK, Candel MJJM, Boot NMWM, Van Raak A, De Vries NK. A multiple-case study of intersectoral collaboration in the context of coordinated school health promotion using the DIagnosis of Sustainable Collaboration (DISC). Model Health Educ. 2015;115(suppl 3/4):301–21.
Thomson AM, Perry JL. Collaboration processes: inside the black box. Public Admin Rev. 2006;66(suppl):20–32.
Wood DJ, Gray B. Toward a comprehensive theory of collaboration. J Appl Behav Sci. 1991;27 suppl 2:139–62.
Pommier J, Guével M, Jourdan D. Evaluation of health in schools: a realistic evaluation approach using mixed methods. BMC Public Health. 2010. doi:10.1186/471-2458-10-43.
El Ansari W, Phillips CJ, Hammick M. Collaboration and partnerships: developing the evidence base. Health Soc Care Comm. 2001;9 suppl 4:215–27.
Cummings TG, Worley CG. Organizational development and change. 7th ed. Mason, Ohio: Southern-Western College Publishing; 2001.
Daft RL, Noe RA. Organizational behavior. Orlando: Hartcourt Inc.; 2001.
De Caluwé L, Vermaak H. Learning to change. Thousand Oaks: Sage Publications; 2003.
De Leeuw E. Healthy cities: urban social entrepreneurship for health. Health Promot Int. 1999;14 suppl 3:261–9.
Van Raak A, Mur-Veeman I, Paulus A. Understanding the feasibility of integrated care: a rival viewpoint on the influence of actions and the institutional context. Int J Health Plann Manage. 1999;14 suppl 3:235–48.
The International Journal of Health Planning and ManagementLeurs M. A collaborative approach to tailored whole-school health promotion. The schoolBeat study. Maastricht 2008.
Barnhoorn H, Walda R. De eerste lijn op spitzen. Van Gorcum: Assen; 1992.
Saan H, De Haes W. Gezond effect bevorderen. Het organiseren van effectieve gezondheidsbevordering (Promotion of healthy effects. The organization of effective health promotion.). NIGZ: Woerden; 2005.
Central on research involving human subjects. http://www.ccmo.nl. Accessed 3 June 2015.
Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. New York: Lawrence Erlbaum Associates; 1995.
Yin RK. Case study research: design and methods. Califormia: Sage Publications; 2009.
Bohlmeijer E, Ruland E, Van Raak A, Mur-Veeman IM. Procesmanagement in public health - ontwerp, analyse & verandering (Processmanagement in public health -design, analysis & change). Trimbos: Utrecht; 2005.
Gray B. Collaborating: finding common ground for multiparty problems. San Francisco: Jossey Bass; 1989.
Koelen MA, Vaandrager L, Wagemakers A. The healthy alliances (HALL) framework: prerequisites for success. Fam Pract. 2012;29(1):i132–8. doi:10.1093/fampra/cmr088.
Bardach E. Getting agencies to work together: the practice and theory of managerial craftsmanship. Washington: Brookings Institution Press; 1998.
Van de Ven AH, Emmett DC, Koenig Jr R. Theoretical and conceptual issues in inter-organizational theory. In: Negandhi AR, editor. Inter-organizational theory. Kent, Ohio: Kent State University Press; 1975. p. 19–38.
Warner M, Gould N. Integrating health in all policies at the local level: using network governance to create ‘virtual reorganization by design’. In: Kickbusch I, editor. Policy Innovation for Health. Springer; 2009. p. 125–63.
Vangen S, Huxham C. Enacting leadership for collaborative advantage: dilemmas of ideology and pragmatism in the activities of partnership managers. Br J Manag. 2003;14(suppl):S61–76.
Huxham C, Vangen S. Working together. Int J Pub Sector Manage. 1996. doi:10.1108/09513559610153863.
Kingdon JW. Agendas, alternatives and public policies. 2nd ed. New York: Harper Collins College Publishers; 1995.
Deschesnes M, Martin C, Hill AJ. Comprehensive approaches to school health promotion: how to achieve broader implementation? Health Promot Int. 2003;18 suppl 4:387–96.
Mur-Veeman I, Van Raak A. Inter-organizational networks on the Dutch home health care market. Int J Health Plann Manage. 1994;9 suppl 3:245–58.
Gajda R. Utilizing collaboration theory to evaluate strategic alliances. Am J Eval. 2004;25 suppl 1:65–77.
Deschesnes M, Tessier C, Couturier Y, Martin C. Professional development in the context of Healthy Schools in Quebec. Health Promot Int. 2013;(e-pub ahead of print); doi:10.1093/heapro/dat043.
Isaac S, Michael WB. Handbook in research and evaluation. a collection of principles, methods and strategies useful in the planning, design and evaluation of studies for education and the behavioral sciences. 3rd ed. EdiTS: San Diego, California; 1995.
Anney VN. Ensuring the quality of the findings of qualitative research: looking at Trustworthiness Criteria. Journal of Emerging Trends in Educational Research and Policy Studies. 2014;5 suppl 2:272–81.
Denzin NK, Lincoln YSE. The sage handbook of qualitative research. Thousands Oaks, CA: Sage; 2005.
This study was funded by ZonMw, the Netherlands Organisation for Health.
Research and Development (grant number: 63300019). The authors would like to thank the Regional Public Health Services (GGD Hollands Midden, GGD Hollands Noorden, GGD Limburg-Noord, GGD Drenthe, GGD Zuid-Holland Zuid) for the close collaboration in the DISC-based trajectory.
The authors declare that they have no competing interests.
NKDV was the principle writer of the original proposal. NMWMB and KKP further developed the study design and implemented the DISC-based trajectory. KKP significantly contributed to writing this paper, while MJJM, AK, NKDV and NMWMB contributed to revising it. All authors approved the final manuscript.