Skip to main content
Download PDF

Evaluation of the promising neighbourhoods community program to reduce health inequalities in youth: a protocol of a mixed-methods study

BMC Public Health volume 19, Article number: 555 (2019) Cite this article

Abstract

Background

Reducing socioeconomic health inequalities among youth is a major challenge for governments around the world and reports on successful attempts are scarce. Socioecological and integral approaches with collaborative partnerships and community engagement are recommended but knowledge about the effectiveness and effective and ineffective elements is limited. The Promising Neighbourhoods program employs such an approach aiming to reduce socioeconomic inequalities in health, safety and talent development in youth. We will evaluate the process-implementation, and effectiveness of the Promising Neighbourhoods program.

Methods/design

Core elements of Promising Neighbourhoods are a collaborative community programming approach with stakeholders, data-based priority setting, knowledge-, and theory-based policies and evidence-based interventions. Community stakeholders and key-leaders from the neighbourhoods are engaged in the program. For this evaluation study the program will be implemented in three intervention neighbourhoods. These neighbourhoods will be compared to three control neighbourhoods at baseline in 2018/2019 and at follow-up in 2020/2021 after full implementation of the Promising Neighbourhoods program. Intervention neighbourhoods receive a tailored intervention-package including evidence-based interventions and additional measures by community stakeholders. In control neighbourhoods, no special planning will take place thus interventions are offered as usual. A mixed-methods approach following the stages of the logic model from program is applied for this evaluation. Questionnaires, focus groups, and registration data will be collected among community stakeholders, key-leaders, and youth to evaluate the process-implementation of the program. Indicators of intermediate and ultimate outcomes will be studied among N = 818 children and N = 818 youngsters using difference-in-difference regression analysis to evaluate the effectiveness of the Promising Neighbourhoods program.

Discussion

Hypotheses are that a collaborative community approach with stakeholders leads to clear priority-setting and better tailored interventions of better quality. We further hypothesise a decline in socioeconomic inequalities in intermediate and ultimate outcomes for health, safety and talent development in the intervention neighbourhoods in comparison to control neighbourhoods. The results add knowledge about effective and ineffective elements of collaborative community programming approaches to reduce health inequalities in youth and thus are relevant for local and national public health authorities.

Trial registration

Netherlands National Trial Register number NL7279. Date of registration: 26-Sept-2018.

Peer Review reports

Background

Reducing socioeconomic health inequalities is a major challenge for governments around the world and many investments have been made in developing strategies and programs to reduce inequalities [1,2,3,4]. However, no convincing reductions in health inequalities at population level have been reported and even an increase is mentioned [5]. Neighbourhood quality, parenting, family life, and the bio-socio-emotional development of youth are thought to explain at least part of the association between socioeconomic conditions and health inequalities [5, 6]. Epidemiological research demonstrated health inequalities in (mental) health status, youth physical activity, school performance, safety, and health related behaviours [7]. In the long term adverse socioeconomic conditions could result in vulnerabilities affecting the perspectives of youth.

To promote health at the local level, socioecological, integral approaches, collaborative partnerships and community engagement are recommended by Weiss et al., and the World Health Organization [7, 8]. Local governments are pivotal in targeting health inequalities, because of their responsibility for social and health policies and their involvement in organising the delivery of social and health services. Furthermore, local governments are in a strong position to bring local actors together for collaboration [9]. At present, knowledge is still limited on how local governments, including municipalities, can successfully implement such integral collaborative community programs, developed in collaboration with community stakeholders. Also, knowledge about what factors determine program successes and failures is still limited [8,9,10]. The scoping review by Weiss et al., mentions facilitators for successful implementation of collaborative community programs such as the Rural Mental Health project [8]. The Rural Mental Health Project is a promising program targeting mental health, well-being and employment in two rural communities in Northern Ireland [11,12,13]. The most frequently mentioned facilitating factors are multidisciplinary collaboration, trust between stakeholders, community engagement and inclusion, local planning and action, adequate resources and the use of a dynamic approach [8]. However, most of the studies included in this scoping review are programs targeted at adults, and not much is known about local community programs targeting youth [8].

Two examples of collaborative community programs that are aimed at promoting the health and well-being of youth are Ensemble Prévenons l’Obésité Des Enfants/Together Let’s Prevent Obesity (EPODE) and Communities that Care (CtC) [14,15,16,17,18,19,20,21,22,23,24]. EPODE uses an integrated community-based approach and includes stakeholders at the national and local level [14]. Variations of EPODE are implemented in several countries. Twelve years after initiation of the first EPODE school- and community based program in France, downward trends in overweight prevalence and obesity prevalence were demonstrated [15]. A downward trend was also demonstrated in Belgium but not in Spain [25, 26]. A process evaluation of 18 EPODE programs among several countries demonstrated that good relations between the local project coordinator, program implementers and stakeholders are seen as an important factor for the effectiveness of EPODE [16]. CtC is an integral community prevention coalition aiming to reduce substance abuse and anti-social behaviour among youth and implemented in several countries including the United States of America (USA), United Kingdom and the Netherlands. CtC was evaluated by the Community Youth Development study in seven matched USA states. The participants in CtC communities were more likely to have abstained from cigarettes, alcohol and drugs compared to control communities [18, 23, 24]. In the United Kingdom and the Netherlands, the results were mixed. For the Netherlands disappointing results are explained by shortcomings in internal validity and study design such as lack of tested and effective preventive interventions, contamination and small sample sizes [17, 19, 27,28,29]. Process evaluations demonstrated that CtC communities had a greater adoption of a science-based approach and used more effective interventions compared to control communities [17, 20,21,22]. Even though these collaborative community programs showed mixed results they demonstrated some important facilitators for successful implementation, and more needs to be learned about their effective and less effective elements to be able to design effective and integral policies and implementation strategies on the local level [8, 30].

Rotterdam, the second largest city in the Netherlands, has many deprived inner-city neighbourhoods. Around one in five of the children and youngsters in this city are raised in poverty [31]. Reducing poverty and socioeconomic health inequalities are recognized as a major challenge by the municipality [31]. The Promising Neighbourhoods program was developed with the aim to decrease health inequalities and to increase health, safety and talent development among youth. A collaborative community approach with stakeholders, using data-based priority setting, knowledge-, and theory-based policies and with evidence-based interventions in a package tailored to the needs of each specific neighbourhood are the core of the Promising Neighbourhoods program. A thorough process- implementation and effect evaluation of the Promising Neighbourhoods program is set up to disentangle which factors are important for successful implementation of these types of collaborative community approaches and to determine the effectiveness. The Promising Neighbourhoods program will be implemented in intervention neighbourhoods which will be compared to control neighbourhoods. This paper describes the design and methodology of the evaluation study of Promising Neighbourhoods.

Research questions

The first research question is: how does the process-implementation of the program evolve and what are effective and less effective steps and elements of the Promising Neighbourhoods program? The second research question is: what is the effectiveness of the Promising Neighbourhoods program on reducing socioeconomic inequalities in intermediate outcomes (determinants: protective and risk factors) and ultimate outcomes (health, safety, and talent development) in youth.

Study hypothesis

We hypothesise that a collaborative community programming approach with stakeholders leads to clear priority setting and better tailored interventions of better quality. Further, we hypothesise a decline in socioeconomic inequalities in intermediate and ultimate outcomes for health, safety and talent development in intervention neighbourhoods in comparison to control neighbourhoods. Intermediate outcomes include the following indicators of risk and protective factors targeted by the program Promising Neighbourhoods: family environment, healthy exercise, nutrition behaviours, smoking and substance use, social cohesion, use of facilities and care and bullying. Ultimate outcomes are indicators of health, safety and talent development.

Methods/design

Description of the promising Neighbourhoods program

The Promising Neighbourhoods program is part of the youth policy ‘Rotterdam Grows 2015-2020’ of the municipality of Rotterdam [32]. This youth policy is based on a multi-level socioecological framework of interrelated risk and protective factors in the life course from pregnancy to young adulthood, including socioeconomic factors (see Fig. 1). This framework provides the theoretical foundation for an integral youth policy program to achieve improvements in several interrelated youth policy domains. The Promising Neighbourhoods program uses this framework and partly builds on experience of methodologies like the community-based EPODE program and the CtC approach [20, 33]. Aims of the Promising Neighbourhoods program are to decrease health inequalities and to increase health, safety, and talent development in youth. The Promising Neighbourhoods program focuses on prevention, stimulation of development, capacity building, empowerment of youth and their families, and improvement of neighbourhood quality, using a collaborative community programming approach with stakeholders, data-based priority setting, knowledge-, and theory-based policies and evidence-based interventions.

Fig. 1
figure 1

Socioecological framework of the Rotterdam Youth Policy Framework 2015–2020. This socioecological framework provides an overview of parental/family, child-youth, school and community risk and protective factors that influence the development of children from pregnancy to young adulthood onward

Full size image

The collaborative community programming approach is managed by municipal district advisors. Each municipal district advisor is assigned to a different neighbourhood and will coordinate and monitor the collaborative community programming approach. Every municipal district advisor plans a neighbourhood tailored intervention-package consisting of parenting support, preventive (health) interventions, youth welfare, measures, facilities and activities to improve preventive factors and reduce risk factors for health, safety and talent development among youth. Each neighbourhood receives a tailored intervention-package according to the individual needs of that particular neighbourhood. These needs are assessed using quantitative indicators of the underlying protective and risk factors (corresponding to the theoretical framework) and in consultation with the community stakeholders and key-leaders within the neighbourhood network. The collaborative community programming approach consists of eight steps.

In step one, the needs-assessment of the neighbourhood will be performed using routinely collected data by the municipality of Rotterdam in the so-called Youth Monitor database [34]. This monitor comprises of around 250 quantitative indicators in the areas of health, safety and talent development. The data are collected from various sources, including Statistics Netherlands (CBS), police databases, survey data and routinely collected registration data by health professionals of the Child & Family Centres. These Child & family centres provide basic preventive health services and function as intermediaries for specialized youth care providers. The indicators in the Youth Monitor database correspond to the risk and protective factors in the above-mentioned theoretical framework. The theoretical framework is used by the municipal district advisors to interpret the data and relations between the findings about priorities in the neighbourhood. Based on the outcomes of the needs-assessment, the municipal district advisors suggest what the priorities should be. At the end of this step, the municipal district advisors prepare a first draft of the needs-assessment report. In step two, the draft needs-assessment report of each neighbourhood is discussed with community stakeholders to match the conclusions based on the quantitative data with their daily experiences and to gain local support by setting joint goals. Subsequently, in step three, the needs-assessment report is adapted based on input of the community stakeholders resulting in a final needs-assessment report of the neighbourhood. In step four, the municipal district advisors inventory the currently available measures, interventions, facilities and activities in the neighbourhood and assess their presence in the so-called database Effective Youth Interventions (EYI) of the Netherlands Youth Institute (NYI) [35]. This is a comprehensive database of all nationally available evidence-based interventions for children and youngsters. The municipal district advisors will do this in collaboration with community stakeholders. In step five, outcomes of step four are discussed by the municipal district advisors with community stakeholders to assess which providers and which measures, interventions, facilities and activities are needed to complete the neighbourhood tailored intervention-package. Priority will be given to EYI database of the Netherlands Youth Institute (NYI) [35]. In step six, a detailed neighbourhood intervention-package plan is drawn and all measures, interventions, facilities and activities that will be implemented are described. In step seven, the intervention-package plan including the proposed measures, interventions, facilities and activities is implemented in the neighbourhood. At last, step eight consists of continuous monitoring and revision of the intervention-package performed by municipal district advisors and community stakeholders in the neighbourhood. The monitoring consists of qualitatively and quantitatively evaluating whether intervention-packages in the neighbourhood have the intended results or not, why the results were achieved or not, and what can be done to achieve the previously set goals.

Intervention neighbourhoods

The Promising Neighbourhoods program will be implemented in three intervention neighbourhoods as described above.

Control Neighbourhoods

In three comparable control neighbourhoods the Promising Neighbourhoods program will not be implemented during this evaluation study. Preventive measures, interventions, activities and facilities will take place in the control neighbourhoods as usual but there will be no collaborative community programming with stakeholders no data-based priority setting and no promotion of knowledge-, and theory-based policies and evidence-based interventions.

Evaluation strategy using a logic model

To evaluate the Promising Neighbourhoods program, a logic model is used (see Fig. 2). This logic model is used as overall guiding framework for the evaluation study [36]. The logic model contains five stages: assets, input, output, intermediate outcomes and ultimate outcomes. All elements of the logic model, i.e. all different stages of the Promising Neighbourhoods program, will be studied and evaluated. The assets, input and output correspond with the process-implementation evaluation and the intermediate and ultimate outcomes correspond with the effect evaluation. The ‘assets stage’ consists of existing data and information about the participating neighbourhoods, the support from partners, community stakeholders, sponsors, and target groups in the neighbourhoods, municipal support, financial resources, expertise, knowledge, local collaboration and coordination. The ‘input stage’ includes the neighbourhood-specific needs-assessment of the municipal district advisors together with community stakeholders, the assessment of measures, interventions, facilities and activities, already present the in neighbourhood and the intervention-package and action plan. The ‘output stage’ includes the type, number, frequency, intensity, quality, costs and reach of the measures, interventions, facilities and activities and improved assets such as capacity building and empowerment. Targeted intermediate outcomes are improvements in risk and protective factors of youth health, safety and talent development. The targeted ultimate outcomes are improvements of socioeconomic inequalities in health, safety, and talent development among youth in Rotterdam.

Fig. 2
figure 2

Logic model for the Promising Neighbourhoods evaluation study

Full size image

Study design

The study consists of a process-implementation and effect evaluation of the Promising Neighbourhoods program using a mixed-methods design. The process-implementation evaluation corresponds to the assets, input and output of the logic model. Important elements such as reach, dose delivered, dose received and program fidelity from process-implementation evaluation strategies are incorporated in this evaluation [37, 38]. For the process-implementation evaluation, qualitative and quantitative measurements will be performed at baseline (T0) and at follow-up after implementation (T1) of the Promising Neighbourhoods program. Qualitative measurements will be performed using questionnaires and focus groups. Quantitative measurements will be performed using register-data and questionnaires. The effect evaluation corresponds with the intermediate and ultimate outcomes of the logic model. The effect evaluation is based on two separate cross-sectional samples and routinely collected data at baseline and at follow-up after implementation of the full Promising Neighbourhoods program.

Setting

This study is carried out in six different neighbourhoods in Rotterdam. Neighbourhoods in Rotterdam are categorized as low, middle or high based on the degree of experienced problems. The categorization is based on the percentage of children aged 4 and 12 years old with a high score on the Strengths and Difficulties Questionnaire (SDQ) and the percentage of overweight children in grade two of primary school in the neighbourhoods [34]. The SDQ is a validated questionnaire to measure emotional problems, conduct problems, hyperactivity/inattention, peer problems, prosocial behaviour and total difficulties score [39]. From each category an intervention neighbourhood was selected resulting in three intervention neighbourhoods. Intervention neighbourhoods were matched to control neighbourhoods as much as possible on these two experienced problems and on average socioeconomic status of the neighbourhoods.

Study population

For the process-implementation evaluation of the Promising Neighbourhoods program, three different groups are distinguished for data collection at baseline as well as at follow-up after the implementation. The first group consists of all community stakeholders in the field of youth (such as teachers, police officers, youth -, health, social -, community -, and sports workers) in each intervention neighbourhood and each control neighbourhood. We want to include all community stakeholders per neighbourhood, and we expect that the size of the population will range between 30 and 50. The second group consists of youngsters (aged ≥12–18 years) in the intervention neighbourhoods, and a comparable group of youngsters in the control neighbourhoods. We will include eight to ten youngsters from intervention neighbourhoods and eight to ten youngsters from control neighbourhoods. The third group consists of key-leaders who are participating in the Promising Neighbourhoods program in each intervention neighbourhood and a comparable group of key-leaders in each control neighbourhood. Key-leaders of the community know the neighbourhood from inside and are for example: police officers, school superintendents, heads of social services agencies, persons knowledgeable about prevention efforts in the community and municipal district advisors managing the process-implementation of the Promising Neighbourhoods program. We will include eight to ten key-leaders per neighbourhood.

For the effect evaluation of the Promising Neighbourhoods Program, the study population consists of N = 818 children aged 0 ≤ 11 years old (N = 409 at baseline and N = 409 at follow-up) and N = 818 youngsters aged 12 ≤ 18 years old (N = 409 at baseline and N = 409 at follow-up), living in the selected intervention and control neighbourhoods in Rotterdam.

Data collection/outcomes

The data collection that will be performed for this evaluation study is discussed separately for each stage of the logic model. The baseline measurement takes place from November 2018 to February 2019. Implementation of the Promising Neighbourhoods program in the intervention neighbourhoods will take place from February 2019 onwards, after the baseline measurement. The follow-up measurement will take place from September 2020 to March 2021.

Assets

Data regarding the assets (knowledge and expertise of community stakeholders about the neighbourhood such as existing problems, municipal support, support from partners, sponsors, and target groups, willingness to collaborate in the Promising Neighbourhoods program, and collaboration and coordination in the neighbourhood) will be studied among community stakeholders in all participating neighbourhoods using an anonymous online-questionnaire and focus groups (see Table 1). This online-questionnaire is based on an instrument (Community Board Interview) to collect information about community stakeholders in neighbourhoods and their role in the process-implementation and their characteristics [19]. This instrument was used for evaluation of the CtC approach in the USA, the Netherlands, and Germany [17, 20, 40] and will be adapted for this process-implementation evaluation of Promising Neighbourhoods. The questionnaire will take at most 30 min to fill out and will be administered from November 2018 to February 2019 and from September 2020 to March 2021. Subsequently, focus groups addressing the assets will be organized from November 2018 to February 2019 and from September 2020 to March 2021 in each neighbourhood. In total, there will be six focus groups at baseline and six focus groups at follow-up. Additionally, two focus groups among youngsters (aged 12 ≤ 18 years old) will be organized, both at baseline and at follow-up; one group representing the intervention neighbourhoods and one group representing the control neighbourhoods. The same topics regarding the assets will be addressed. The focus groups among youngsters are specifically used to take the perspective of youngsters about the quality of neighbourhood conditions into account. To assess financial resources, we will use registration data of the municipality of Rotterdam.

Table 1 Overview of evaluation measurements
Full size table

Input

Data regarding the input (effectiveness of neighbourhood networks consisting of community stakeholders in the neighbourhood, and existing measures, plans, interventions, facilities and activities) will be studied using the same online-questionnaire and focus groups in 2018/2019 and 2020/2021 as will be used to study the assets (see Table 1). The anonymous online-questionnaire and focus groups will be held among community stakeholders in all participating intervention and control neighbourhoods. Additionally, in the two focus groups among youngsters representing the intervention neighbourhoods and youngsters representing the control neighbourhoods (aged 12 ≤ 18 years old) the same topics regarding the input will be discussed. To assess existing measures, interventions, facilities and activities in the neighbourhood, we will use registration data of the municipality of Rotterdam and additional surveys of key-leaders.

Output

Data regarding the output (costs, type, number, quality, frequency, intensity, reach and characteristics of the reached groups of the implemented measures, interventions, facilities, and activities, use of a knowledge/science-based approach, use of effective programs and monitoring of implementation) will be studied using a questionnaire (see Table 1). This questionnaire will be administered among key-leaders in November 2018 to February 2019 and from September 2020 to March 2021. This questionnaire is an adapted version of the Key-Informant questionnaire previously used in CtC evaluations in the USA, the Netherlands and Germany [17, 22]. This questionnaire will take at most 60 min to fill out and will be administered via telephone. During the focus groups with community stakeholders in 2020/2021, and the online-questionnaire prior to it, the contribution of assets and input to the realized output will be discussed. To examine the costs, type, number, quality, frequency, intensity, reach and characteristics of the reached groups of the implemented measures, interventions, facilities, and activities we will use registration data of the municipality of Rotterdam. These data are registered by the municipal district advisors and providers of the interventions (see Table 1).

Intermediate outcomes

In this study, the intermediate outcomes are the proximal results of the measures, interventions, facilities and activities that will be implemented in the neighbourhoods. Therefore, we will study indicators of the following targeted risk and protective factors among children and youngsters: family environment (parenting, child-parent relationship, family life, and family conflict), healthy exercise, nutrition behaviours, smoking and substance use, social cohesion, use of facilities and care, and bullying (see Table 1). Data on intermediate outcomes will be collected at two time points, in 2018 at baseline and in 2020/2021 at follow-up, separately for children aged 0 ≤ 11 years old and for youngsters aged 12 ≤ 18 years old.

For the baseline measurement of intermediate outcomes among 0 ≤ 11 years old children, we will use anonymous survey data from the Health survey obtained in 2018 by the municipality of Rotterdam. The Health survey is administered every four years in a random sample drawn from the municipal population register using online parent-questionnaires. The Health survey consists of questionnaires addressing the following topics: general health, nutrition and exercise behaviour, home-environment, emotional and psychological health, neighbourhood-, and school environment, use of care and facilities, smoking and alcohol [39]. Follow-up data will be collected by administering a comparable parent-questionnaire for children aged 0 ≤ 11 years old in 2020/2021. This questionnaire will be based on the Health survey and will include similar questions. We will use the same procedures for sample selection and administration of the follow-up questionnaire, as used by the municipality for the Health survey (the baseline data).

For the baseline and follow-up measurements in the group of 12 ≤ 18 years old youngsters we will use survey-data obtained by the Child and Family Centre Rijnmond in 2018 and 2020/2021 respectively. These survey data consist of the SDQ and the so-called You and Your health questionnaire which includes several validated questionnaires [39, 41,42,43]. The You and Your health questionnaire addresses the following topics: general health, nutrition and exercise behaviour, home-environment, emotional and psychological health, school environment, performing anxiety and learning behaviour, smoking, alcohol, substance use and gaming. These questionnaires are routinely administered by the Child and Family Centre Rijnmond every year for municipal health monitoring. The questionnaires are filled out online at school, the SDQ in first grade and You and your Health in third grade.

Ultimate outcomes

Ultimate outcome measures are: indicators of health (socio-emotional and/or psychological problems, general/physical health, overweight), safety (safety of the home environment, safety of the neighbourhood) and talent development (school performance, truancy) (see Table 1). Ultimate outcomes will be collected at the same two time-points, at baseline and at follow-up after implementation of the Promising Neighbourhoods program. The same instruments as for the intermediate outcomes will be used, separately for children aged 0 ≤ 11 years old and for youngsters aged 12 ≤ 18 years old. Since truancy, school performance and safety in the neighbourhood are not administered by the Health survey, SDQ or You and Your Health questionnaire we will use routinely collected data on the individual, school and neighbourhood level from the municipality of Rotterdam for these outcomes.

Covariates

In addition to the intermediate and ultimate outcomes, demographical data including neighbourhood, age, sex, ethnicity, and socioeconomic status will be collected for the effect evaluation. Socioeconomic status will be measured as the highest educational level obtained either by both parents or the mother for children aged 0 ≤ 11 years old and current educational level of youngsters aged 12 ≤ 18 years old. Educational level will be classified according to the International Standard Classification of Education 2011 [44]. Demographical data will be obtained from the municipality of Rotterdam and the Child and Family Centre Rijnmond.

Power considerations

The size of the data samples needed to determine small-effect sizes (f2 = 0.02) is calculated for the effect evaluation [45]. Analyses will be performed separately for children aged 0 ≤ 11 years old and for youngsters aged 12 ≤ 18 years old as outcome-variables of interest might differ between age groups and are measured in a different way. Based upon 5% two-sided significance tests of the null hypothesis that socioeconomic status-groups in the intervention and control neighbourhoods do not differ on outcome variables at follow-up and a power of 80% allowing for 10 independent variables in the model, we need two samples of 818 individuals, one sample including children aged 0 ≤ 11 years old and one sample including youngsters aged 12 ≤ 18 years old. Half of each sample will consist of individuals at baseline, evenly distributed over the conditions (intervention or control neighbourhood). The other half will consist of individuals at follow-up, evenly distributed over conditions as well. For the baseline measurement of both age groups, we will have sufficient survey data. For the follow-up measurement of youngsters aged 12 ≤ 18 years old sufficient survey data is available because the Child and Family Centre Rijnmond administers their survey every year. However, since the Health survey of the municipality of Rotterdam is administered only once every four years we will not have sufficient follow-up data for children aged of 0 ≤ 11 years old. Therefore, we will gather these data by administering an additional comparable parent-questionnaire. With a predicted response rate of 40% we will need to reach 1025 parents in order to receive 409 follow-up questionnaires.

Data analyses

The process-implementation evaluation consists of the assets, input and output stages in the logic model. All focus groups and questionnaires will be documented and analysed using standardized formats. The qualitative data will be analysed using a set of codes. At the start of the analysis, a codebook will be made with herein a list of pre-set codes on different levels such as concepts or categories. This codebook will be made by two researchers independently coding the first two focus groups. During the analysis new codes may occur. These codes will be added to the codebook. Questionnaire data and register-data per intervention and per neighbourhood about the costs, type, number, quality, frequency, intensity, reach, and characteristics of the reached groups will also be analysed to detect changes between the baseline measurement and the follow-up measurement. Intervention and control neighbourhood results will be compared. Because of the relative small size of the groups in the process-implementation evaluation, T-tests with Bayesian statistics will be used [46, 47]. This will be performed using the program Jags in R. Intervention and control estimates will be compared, each chain with 5000 iterations and each time 500 not used. At the end, the estimates will be made with a total of 4500 estimates.

In the effect evaluation of the Promising Neighbourhoods program, effects on the dependent intermediate outcomes (indicators of risk and protective factors) and ultimate outcomes (indicators of health, safety and talent development) of the logic model will be examined. We will perform Difference-in-Difference regression analyses separately for children aged 0 ≤ 11 years old and for youngsters aged 12 ≤ 18 years old. Predictor variables of interest are time of measurement (baseline/follow-up), condition (intervention/control), socioeconomic status, and their two-way and three-way interactions controlled for neighbourhood conditions. We will also adjust for other sociodemographic covariates. The main variable of interest in our study is the three-way interaction term for time of measurement, condition, and socioeconomic status. A significant parameter (p < 0.05) for this interaction term indicates a change in outcome over time in the intervention neighbourhoods with a socioeconomic status gradient. In case of continuous outcome variables linear regression analysis will be used and for dichotomous outcome variables logic regression analysis will be used. Missing data will be handled using multiple imputation. Level of significance is set at 0.05 for two-tailed analyses.

Discussion

This article describes the design and methodology of a mixed-methods study for evaluation of the process-implementation and effectiveness of the Promising Neighbourhoods program. Promising Neighbourhoods is a program aiming to reduce socioeconomic health inequalities and to increase health, safety and talent development among youth in different neighbourhoods of Rotterdam. The Promising Neighbourhoods program will be gradually implemented in all neighbourhoods in Rotterdam, if proven effective.

We hypothesise that a collaborative community programming approach with stakeholders leads to clear priority setting and better tailored interventions of better quality. Furthermore, we hypothesise a decline in socioeconomic inequalities in intermediate and ultimate outcomes for health, safety and talent development in intervention neighbourhoods in comparison to control neighbourhoods. As we also study the process-implementation of the Promising Neighbourhoods program, we will be able to provide relevant insights on possible facilitators and barriers for future implementation of policy programs using a collaborative community approach, such as Promising Neighbourhoods.

A strength of this study is that we study the process-implementation as well as the effectiveness [48]. Without evaluation of the process-implementation, the black box of why a program is effective or not effective cannot be disentangled [48]. Furthermore, the process-implementation evaluation can also shed light on possible defects or unwanted side effects within the Promising Neighbourhoods program [10]. Adding to this, it may elucidate how to successfully adapt programs and reach specific communities. Therefore, this study is relevant for local settings and collaborative programming and for national governments that depend on successful local implementation of policies. Another strong aspect is that we evaluate the process-implementation and effectiveness in different neighbourhoods increasing the generalisability of our findings.

The limitations that need to be taken into consideration are first that long term effects are not part of our study design. The time of two years between the baseline and follow-up does not allow studying longer term outcomes on especially the intended ultimate outcomes in the current study. Second, there is always a chance for residual confounding even though we will adjust for confounding in our analyses and tried to match intervention neighbourhoods as much as possible based on the degree of experienced problems and on socioeconomic status. We cannot control planned or unplanned implementation of interventions in the neighbourhoods on the initiative of other institutions or in some cases even of collaborating community stakeholders. Moreover, we cannot control for the fact that children, youngsters and their families in control neighbourhoods may receive or take part in interventions or activities provided in the intervention neighbourhoods. Or that they take part in interventions or activities implemented independently of interventions that are part of the Promising Neighbourhoods program. However, to monitor this, we will obtain registration data about implemented measures, interventions, facilities and activities in the neighbourhoods.

In conclusion, this study will provide knowledge about the process-implementation and effectiveness of a collaborative community programming approach implemented together with stakeholders, using data-based priority setting, knowledge-, and theory-based policies and evidence-based interventions. The results following from this study may be used for the design, implementation and transferability of intervention programs aiming to reduce health inequalities among youth using a collaborative community programming approach with stakeholders. Therefore, our study is relevant for local and national public health authorities and for improvement of the health, safety, and talent development among youth.

Abbreviations

CCP:

Collaborative community program

CtC:

Communities that Care

EPODE:

Ensemble Prévenons l’Obésité Des Enfants/Together Lets Prevent Obesity

EYI:

Effective Youth Interventions

ICJME:

International Committee of Medical Journal Editors

NYI:

Netherlands Youth Institute

SDQ:

Strengths and Difficulties Questionnaire

T0:

baseline measurement

T1:

Follow-up measurement

USA:

United States of America

References

  1. Mackenbach JP. Socioeconomic inequalities in health in the Netherlands: impact of a five year research programme. Bmj. 1994;309(6967):1487–91.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Mackenbach JP, Kunst AE, Cavelaars AE, Groenhof F, Geurts JJ. Socioeconomic inequalities in morbidity and mortality in western Europe. The EU Working Group on Socioeconomic Inequalities in Health. Lancet. 1997;349(9066):1655–9.

    Article  CAS  PubMed  Google Scholar 

  3. Mackenbach JP, Stronks K. A strategy for tackling health inequalities in the Netherlands. Bmj. 2002;325(7371):1029–32.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Marmot M. Commission on Social determinants of health: achieving health equity: from root causes to fair outcomes. Lancet. 2007;370(9593):1153–63.

    Article  PubMed  Google Scholar 

  5. Marmot M, Bell R. Fair society, healthy lives. Public Health. 2012;126:S4–S10.

    Article  PubMed  Google Scholar 

  6. Yoshikawa H, Aber JL, Beardslee WR. The effects of poverty on the mental, emotional, and behavioral health of children and youth: implications for prevention. Am Psychol. 2012;67(4):272–84.

    Article  PubMed  Google Scholar 

  7. World Health Organization, Commission on Social Determinants of Health. closing the gap in a generation: health equity through action on the social determinants of health: final report of the commission on Social determinants of health. Geneva: World Health Organization; 2008.

  8. Weiss D, Lillefjell M, Magnus E. Facilitators for the development and implementation of health promoting policy and programs - a scoping review at the local community level. BMC Public Health. 2016;16:140.

    Article  PubMed  PubMed Central  Google Scholar 

  9. World Health Organization. Addressing the social determinants of health: the urban dimension and the role of local government. Copenhagen: World Health Organization Regional Office for Europe; 2012.

    Google Scholar 

  10. Lorenc T, Oliver K. Adverse effects of public health interventions: a conceptual framework. J Epidemiol Community Health. 2014;68(3):288–90.

    Article  PubMed  Google Scholar 

  11. Barry MM. Researching the implementation of community mental health promotion programs. Health Promot J Austr. 2007;18(3):240–6.

    Article  PubMed  Google Scholar 

  12. Barry M. Designing an evaluation framework for community mental health promotion. J of Public Ment Health. 2003;2(4):26–36.

    Article  Google Scholar 

  13. Barry M, Reynolds C, Sheridan A, Egenton R. Implementation of the JOBS programme in Ireland. J of Public Ment Health. 2006;5(4):10–25.

    Article  Google Scholar 

  14. Borys JM, Le Bodo Y, Jebb SA, Seidell JC, Summerbell C, Richard D, De Henauw S, Moreno LA, Romon M, Visscher TL, et al. EPODE approach for childhood obesity prevention: methods, progress and international development. Obes Rev. 2012;13(4):299–315.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Romon M, Lommez A, Tafflet M, Basdevant A, Oppert JM, Bresson JL, Ducimetiere P, Charles MA, Borys JM. Downward trends in the prevalence of childhood overweight in the setting of 12-year school- and community-based programmes. Public Health Nutr. 2009;12(10):1735–42.

    Article  PubMed  Google Scholar 

  16. Pettigrew S, Borys JM, du Plessis HR, Walter L, Huang TT, Levi J, Vinck J. Process evaluation outcomes from a global child obesity prevention intervention. BMC Public Health. 2014;14:757.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Steketee M, Oesterle S, Jonkman H, Hawkins JD, Haggerty KP, Aussems C. Transforming prevention systems in the United States and the Netherlands using communities that care promising prevention in the eyes of Josine Junger-Tas. Eur J Crim Pol Res. 2013;19(2):99–116.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Oesterle S, Kuklinski MR, Hawkins JD, Skinner ML, Guttmannova K, Rhew IC. Long-term effects of the communities that care trial on substance use, antisocial behavior, and violence through age 21 years. Am J Public Health. 2018;108(5):659–65.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Jonkman HB, Haggerty KP, Steketee M, Fagan A, Hanson K, Hawkins JD. Communities that care, Core elements and context: Research of implementation in two countries. Soc dev issues. 2009;30(3):42–57.

    CAS  PubMed  PubMed Central  Google Scholar 

  20. Arthur MW, Hawkins JD, Brown EC, Briney JS, Oesterle S, Social Development Research G, Abbott RD. Implementation of the communities that care prevention system by coalitions in the community youth Development study. J community psychol. 2010;38(2):245–58.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Brown EC, Hawkins JD, Arthur MW, Briney JS, Abbott RD. Effects of communities that care on prevention services systems: findings from the community youth Development study at 1.5 years. Prev Sci. 2007;8(3):180–91.

    Article  PubMed  Google Scholar 

  22. Brown EC, Hawkins JD, Arthur MW, Briney JS, Fagan AA. Prevention service system transformation using communities that care. J Community Psychol. 2011;39(2):183–201.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Hawkins JD, Catalano RF, Arthur MW, Egan E, Brown EC, Abbott RD, Murray DM. Testing communities that care: the rationale, design and behavioral baseline equivalence of the community youth Development study. Prev Sci. 2008;9(3):178–90.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Hawkins JD, Oesterle S, Brown EC, Abbott RD, Catalano RF. Youth problem behaviors 8 years after implementing the communities that care prevention system in a community-randomized trial. JAMA Pediatr. 2014;168(2):122–9.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Gomez SF, Casas Esteve R, Subirana I, Serra-Majem L, Fletas Torrent M, Homs C, Bawaked RA, Estrada L, Fito M, Schroder H. Effect of a community-based childhood obesity intervention program on changes in anthropometric variables, incidence of obesity, and lifestyle choices in Spanish children aged 8 to 10 years. Eur J Pediatr. 2018;177(10):1531–9.

    Article  PubMed  Google Scholar 

  26. Vinck J, Brohet C, Roillet M, Dramaix M, Borys JM, Beysens J, Jacobs N, Jebb S, De Laet C, Neve J. Downward trends in the prevalence of childhood overweight in two pilot towns taking part in the VIASANO community-based programme in Belgium: data from a national school health monitoring system. Pediatr Obes. 2016;11(1):61–7.

    Article  CAS  PubMed  Google Scholar 

  27. Crow I, France A, Hacking S, Hart M. Does communities that care work? An evaluation of a community-based risk prevention programme in three neighbourhoods. York: Joseph Rowntree Foundation; 2004.

    Google Scholar 

  28. Jonkman HB, Junger-Tas J, van Dijk B. From behind dikes and dunes: communities that care in the Netherlands. Child Soc. 2005;19(2):105–16.

    Article  Google Scholar 

  29. Jonkman H, Aussems C, Steketee M, Boutellier H, Cuijpers P. Prevention of problem Behaviours among adolescents: the impact of the communities that care strategy in the Netherlands (2008–2011). Int J Dev Sci. 2015;9(1):37–52.

    Article  Google Scholar 

  30. Merzel C, D'Afflitti J. Reconsidering community-based health promotion: promise, performance, and potential. Am J Public Health. 2003;93(4):557–74.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Mieloo C, de Haan C, Jansen W, Stoorvogel H, Wiering D: Youth Monitor (Staat van de jeugd 2018), Rotterdam: Municipality of Rotterdam; 2018.

  32. Municipality of Rotterdam: Youth Policy 2015–2020 (Beleidskader Jeugd 2015–2020). In. Rotterdam, The Netherlands; 2015.

  33. Borys JM, Richard P, Ruault du Plessis H, Harper P, Levy E. Tackling health inequities and reducing obesity prevalence: the EPODE community-based approach. Ann Nutr Metab. 2016;68(Suppl 2):35–8.

    Article  PubMed  Google Scholar 

  34. Youth Monitor (Staat van de jeugd). https://www.rotterdam.nl/werken-leren/staat-van-de-jeugd. Accessed 20 Nov 2018.

  35. Database effective youth interventions (databank Effectieve Jeugdinterventies). https://www.nji.nl/nl/Databank/Databank-Effectieve-Jeugdinterventies. Accessed 10 Oct 2018.

  36. Cooksy LJ, Gill P, Kelly PA. The program logic model as an integrative framework for a multimethod evaluation. Eval Program Plann. 2001;24(2):119–28.

    Article  Google Scholar 

  37. Saunders RP, Evans MH, Joshi P. Developing a process-evaluation plan for assessing health promotion program implementation: a how-to guide. Health Promot Pract. 2005;6(2):134–47.

    Article  PubMed  Google Scholar 

  38. Nutbeam D. Evaluating health promotion—Progress, problems and solutions. Health Promot Int. 1998;13(1):27–44.

    Article  Google Scholar 

  39. van Widenfelt BM, Goedhart AW, Treffers PD, Goodman R. Dutch version of the strengths and difficulties questionnaire (SDQ). Eur Child Adolesc Psychiatry. 2003;12(6):281–9.

    Article  PubMed  Google Scholar 

  40. Jonkman H. Communities That Care in Niedersachsen. In: Ergebnisse der Implementation von CTC in fünf Standorten (2013–2014). Hannover: LPR; 2015.

    Google Scholar 

  41. Thombs BD, Bernstein DP, Lobbestael J, Arntz A. A validation study of the Dutch childhood trauma questionnaire-short form: factor structure, reliability, and known-groups validity. Child Abuse Negl. 2009;33(8):518–23.

    Article  PubMed  Google Scholar 

  42. Kosters MP, Chinapaw MJ, Zwaanswijk M, van der Wal MF, Koot HM. Structure, reliability, and validity of the revised child anxiety and depression scale (RCADS) in a multi-ethnic urban sample of Dutch children. BMC Psychiatry. 2015;15:132.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Verlinden E, van Meijel EP, Opmeer BC, Beer R, de Roos C, Bicanic IA, Lamers-Winkelman F, Olff M, Boer F, Lindauer RJ. Characteristics of the Children's revised impact of event scale in a clinically referred Dutch sample. J Trauma Stress. 2014;27(3):338–44.

    Article  PubMed  Google Scholar 

  44. Unesco Institute for Statistics. International standard classification of education: ISCED 2011. Montreal: UNESCO Institute for Statistics; 2012.

    Google Scholar 

  45. cohen J. Statistical power analysis for the behavioral sciences second edition. Hillsdale: Lawrence Erlbaum Associates Inc.; 1988.

    Google Scholar 

  46. Cowles MK. Applied Bayesian statistics: with R and OpenBUGS examples, vol. 98. New York: Springer Science & Business Media; 2013.

    Google Scholar 

  47. Johnson W, Branscum A, Hanson TE, Christensen R. Bayesian ideas and data analysis: an introduction for scientists and statisticians. Boca Raton: CRC Press; 2010.

    Google Scholar 

  48. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Int J Nurs Stud. 2013;50(5):587–92.

    Article  PubMed  Google Scholar 

  49. ICJME Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals. http://www.icmje.org/icmje-recommendations.pdf. Accessed 20 Feb 2019.

Download references

Acknowledgements

The authors thank the involved program-managers and municipal district advisors of Promising Neighbourhoods from the city of Rotterdam.

Funding

This work was funded by a research grant (project number: 531001313) from ZonMw, The Netherlands Organization for Health Research and Development. ZonMw has no role in any part of the research, writing and reviewing of the manuscript.

Availability of data and materials

Data availability and sharing is non-applicable to this particular study as datasets are not yet generated nor analysed for this study protocol. Future study results will be communicated according to the International Committee of Medical Journal Editors (ICJME) recommendations for authorship [49] via relevant scientific journals without any publication restrictions. A data management plan and a checklist for processing data on individual persons is set up for this study. The dataset that will be generated and analysed for future publications, the data management plan and a checklist for processing data on idividual persons is available from the corresponding author on reasonable request and with permission of the third parties involved.

Author information

Authors and Affiliations

  1. Department of Public Health, Erasmus University Medical Centre, Rotterdam, the Netherlands

    Mirte Boelens, Dafna A. Windhorst, Hein Raat & Wilma Jansen

  2. Verwey-Jonker Institute, Utrecht, the Netherlands

    Harrie Jonkman

  3. Department of Health Promotion, Maastricht University, Maastricht, the Netherlands

    Clemens M. H. Hosman

  4. Department of Clinical Psychology, Radboud University, Nijmegen, the Netherlands

    Clemens M. H. Hosman

  5. Hosman Prevention and Innovation Consultancy, Berg en Dal, the Netherlands

    Clemens M. H. Hosman

  6. Municipality of Rotterdam, Rotterdam, the Netherlands

    Wilma Jansen

Authors
  1. Mirte Boelens
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dafna A. Windhorst
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Harrie Jonkman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Clemens M. H. Hosman
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hein Raat
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wilma Jansen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

DW, HJ, CH, WJ and HR had the original idea for this study and the study design and were responsible for acquiring the study grant. MB and HJ are responsible for the data collection, data analysis and reporting of the study results. DW, MB, WJ and HR are responsible for the data management of the effect evaluation. HJ is responsible for the data management of the process-implementation evaluation. CH provides expert input in this study. DW, HR and WJ supervise this study. All authors regularly participated in discussing the design drafts of this study. All authors have read and approved this final manuscript.

Corresponding author

Correspondence to Wilma Jansen.

Ethics declarations

Ethics approval and consent to participate

The medical ethics committee of the Erasmus University Medical Center Rotterdam has declared that the Medical Research Involving Human Subjects Act does not apply to this research protocol and issued a declaration of no objection for this study (registration number MEC-2018-1506).Therefore, this study can be carried out without further approval by an accredited medical ethical committee. This study has been entered into the Netherlands National Trial Register (NTR; https://www.trialregister.nl/trial/7279) under catalogue number NL7279 on 26 September 2018. Any important protocol modifications will be reported by the researchers to the funding body and the trial registry.

Written informed consent will be asked from community stakeholders and youngsters for participation in the focus groups that are part of the process and implementation evaluation For youngsters aged 12 ≤ 16 years old the parents will also be asked to additionally give written informed consent before participating in the focus groups. Verbal informed consent will be asked from the key-leaders for the key-informant questionnaire, that is part of the process and implementation evaluation. Verbal informed consent is used among key-leaders as the questionnaire is administered by telephone. For the questionnaires among community stakeholders and all effect evaluation questionnaires informed consent is assumed by returning a completed anonymous questionnaire.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Boelens, M., Windhorst, D.A., Jonkman, H. et al. Evaluation of the promising neighbourhoods community program to reduce health inequalities in youth: a protocol of a mixed-methods study. BMC Public Health 19, 555 (2019). https://doi.org/10.1186/s12889-019-6901-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12889-019-6901-3

Keywords

BMC Public Health

ISSN: 1471-2458

Contact us