- Study protocol
- Open Access
Evaluation of the promising neighbourhoods community program to reduce health inequalities in youth: a protocol of a mixed-methods study
BMC Public Health volume 19, Article number: 555 (2019)
Reducing socioeconomic health inequalities among youth is a major challenge for governments around the world and reports on successful attempts are scarce. Socioecological and integral approaches with collaborative partnerships and community engagement are recommended but knowledge about the effectiveness and effective and ineffective elements is limited. The Promising Neighbourhoods program employs such an approach aiming to reduce socioeconomic inequalities in health, safety and talent development in youth. We will evaluate the process-implementation, and effectiveness of the Promising Neighbourhoods program.
Core elements of Promising Neighbourhoods are a collaborative community programming approach with stakeholders, data-based priority setting, knowledge-, and theory-based policies and evidence-based interventions. Community stakeholders and key-leaders from the neighbourhoods are engaged in the program. For this evaluation study the program will be implemented in three intervention neighbourhoods. These neighbourhoods will be compared to three control neighbourhoods at baseline in 2018/2019 and at follow-up in 2020/2021 after full implementation of the Promising Neighbourhoods program. Intervention neighbourhoods receive a tailored intervention-package including evidence-based interventions and additional measures by community stakeholders. In control neighbourhoods, no special planning will take place thus interventions are offered as usual. A mixed-methods approach following the stages of the logic model from program is applied for this evaluation. Questionnaires, focus groups, and registration data will be collected among community stakeholders, key-leaders, and youth to evaluate the process-implementation of the program. Indicators of intermediate and ultimate outcomes will be studied among N = 818 children and N = 818 youngsters using difference-in-difference regression analysis to evaluate the effectiveness of the Promising Neighbourhoods program.
Hypotheses are that a collaborative community approach with stakeholders leads to clear priority-setting and better tailored interventions of better quality. We further hypothesise a decline in socioeconomic inequalities in intermediate and ultimate outcomes for health, safety and talent development in the intervention neighbourhoods in comparison to control neighbourhoods. The results add knowledge about effective and ineffective elements of collaborative community programming approaches to reduce health inequalities in youth and thus are relevant for local and national public health authorities.
Netherlands National Trial Register number NL7279. Date of registration: 26-Sept-2018.
Reducing socioeconomic health inequalities is a major challenge for governments around the world and many investments have been made in developing strategies and programs to reduce inequalities [1,2,3,4]. However, no convincing reductions in health inequalities at population level have been reported and even an increase is mentioned . Neighbourhood quality, parenting, family life, and the bio-socio-emotional development of youth are thought to explain at least part of the association between socioeconomic conditions and health inequalities [5, 6]. Epidemiological research demonstrated health inequalities in (mental) health status, youth physical activity, school performance, safety, and health related behaviours . In the long term adverse socioeconomic conditions could result in vulnerabilities affecting the perspectives of youth.
To promote health at the local level, socioecological, integral approaches, collaborative partnerships and community engagement are recommended by Weiss et al., and the World Health Organization [7, 8]. Local governments are pivotal in targeting health inequalities, because of their responsibility for social and health policies and their involvement in organising the delivery of social and health services. Furthermore, local governments are in a strong position to bring local actors together for collaboration . At present, knowledge is still limited on how local governments, including municipalities, can successfully implement such integral collaborative community programs, developed in collaboration with community stakeholders. Also, knowledge about what factors determine program successes and failures is still limited [8,9,10]. The scoping review by Weiss et al., mentions facilitators for successful implementation of collaborative community programs such as the Rural Mental Health project . The Rural Mental Health Project is a promising program targeting mental health, well-being and employment in two rural communities in Northern Ireland [11,12,13]. The most frequently mentioned facilitating factors are multidisciplinary collaboration, trust between stakeholders, community engagement and inclusion, local planning and action, adequate resources and the use of a dynamic approach . However, most of the studies included in this scoping review are programs targeted at adults, and not much is known about local community programs targeting youth .
Two examples of collaborative community programs that are aimed at promoting the health and well-being of youth are Ensemble Prévenons l’Obésité Des Enfants/Together Let’s Prevent Obesity (EPODE) and Communities that Care (CtC) [14,15,16,17,18,19,20,21,22,23,24]. EPODE uses an integrated community-based approach and includes stakeholders at the national and local level . Variations of EPODE are implemented in several countries. Twelve years after initiation of the first EPODE school- and community based program in France, downward trends in overweight prevalence and obesity prevalence were demonstrated . A downward trend was also demonstrated in Belgium but not in Spain [25, 26]. A process evaluation of 18 EPODE programs among several countries demonstrated that good relations between the local project coordinator, program implementers and stakeholders are seen as an important factor for the effectiveness of EPODE . CtC is an integral community prevention coalition aiming to reduce substance abuse and anti-social behaviour among youth and implemented in several countries including the United States of America (USA), United Kingdom and the Netherlands. CtC was evaluated by the Community Youth Development study in seven matched USA states. The participants in CtC communities were more likely to have abstained from cigarettes, alcohol and drugs compared to control communities [18, 23, 24]. In the United Kingdom and the Netherlands, the results were mixed. For the Netherlands disappointing results are explained by shortcomings in internal validity and study design such as lack of tested and effective preventive interventions, contamination and small sample sizes [17, 19, 27,28,29]. Process evaluations demonstrated that CtC communities had a greater adoption of a science-based approach and used more effective interventions compared to control communities [17, 20,21,22]. Even though these collaborative community programs showed mixed results they demonstrated some important facilitators for successful implementation, and more needs to be learned about their effective and less effective elements to be able to design effective and integral policies and implementation strategies on the local level [8, 30].
Rotterdam, the second largest city in the Netherlands, has many deprived inner-city neighbourhoods. Around one in five of the children and youngsters in this city are raised in poverty . Reducing poverty and socioeconomic health inequalities are recognized as a major challenge by the municipality . The Promising Neighbourhoods program was developed with the aim to decrease health inequalities and to increase health, safety and talent development among youth. A collaborative community approach with stakeholders, using data-based priority setting, knowledge-, and theory-based policies and with evidence-based interventions in a package tailored to the needs of each specific neighbourhood are the core of the Promising Neighbourhoods program. A thorough process- implementation and effect evaluation of the Promising Neighbourhoods program is set up to disentangle which factors are important for successful implementation of these types of collaborative community approaches and to determine the effectiveness. The Promising Neighbourhoods program will be implemented in intervention neighbourhoods which will be compared to control neighbourhoods. This paper describes the design and methodology of the evaluation study of Promising Neighbourhoods.
The first research question is: how does the process-implementation of the program evolve and what are effective and less effective steps and elements of the Promising Neighbourhoods program? The second research question is: what is the effectiveness of the Promising Neighbourhoods program on reducing socioeconomic inequalities in intermediate outcomes (determinants: protective and risk factors) and ultimate outcomes (health, safety, and talent development) in youth.
We hypothesise that a collaborative community programming approach with stakeholders leads to clear priority setting and better tailored interventions of better quality. Further, we hypothesise a decline in socioeconomic inequalities in intermediate and ultimate outcomes for health, safety and talent development in intervention neighbourhoods in comparison to control neighbourhoods. Intermediate outcomes include the following indicators of risk and protective factors targeted by the program Promising Neighbourhoods: family environment, healthy exercise, nutrition behaviours, smoking and substance use, social cohesion, use of facilities and care and bullying. Ultimate outcomes are indicators of health, safety and talent development.
Description of the promising Neighbourhoods program
The Promising Neighbourhoods program is part of the youth policy ‘Rotterdam Grows 2015-2020’ of the municipality of Rotterdam . This youth policy is based on a multi-level socioecological framework of interrelated risk and protective factors in the life course from pregnancy to young adulthood, including socioeconomic factors (see Fig. 1). This framework provides the theoretical foundation for an integral youth policy program to achieve improvements in several interrelated youth policy domains. The Promising Neighbourhoods program uses this framework and partly builds on experience of methodologies like the community-based EPODE program and the CtC approach [20, 33]. Aims of the Promising Neighbourhoods program are to decrease health inequalities and to increase health, safety, and talent development in youth. The Promising Neighbourhoods program focuses on prevention, stimulation of development, capacity building, empowerment of youth and their families, and improvement of neighbourhood quality, using a collaborative community programming approach with stakeholders, data-based priority setting, knowledge-, and theory-based policies and evidence-based interventions.
The collaborative community programming approach is managed by municipal district advisors. Each municipal district advisor is assigned to a different neighbourhood and will coordinate and monitor the collaborative community programming approach. Every municipal district advisor plans a neighbourhood tailored intervention-package consisting of parenting support, preventive (health) interventions, youth welfare, measures, facilities and activities to improve preventive factors and reduce risk factors for health, safety and talent development among youth. Each neighbourhood receives a tailored intervention-package according to the individual needs of that particular neighbourhood. These needs are assessed using quantitative indicators of the underlying protective and risk factors (corresponding to the theoretical framework) and in consultation with the community stakeholders and key-leaders within the neighbourhood network. The collaborative community programming approach consists of eight steps.
In step one, the needs-assessment of the neighbourhood will be performed using routinely collected data by the municipality of Rotterdam in the so-called Youth Monitor database . This monitor comprises of around 250 quantitative indicators in the areas of health, safety and talent development. The data are collected from various sources, including Statistics Netherlands (CBS), police databases, survey data and routinely collected registration data by health professionals of the Child & Family Centres. These Child & family centres provide basic preventive health services and function as intermediaries for specialized youth care providers. The indicators in the Youth Monitor database correspond to the risk and protective factors in the above-mentioned theoretical framework. The theoretical framework is used by the municipal district advisors to interpret the data and relations between the findings about priorities in the neighbourhood. Based on the outcomes of the needs-assessment, the municipal district advisors suggest what the priorities should be. At the end of this step, the municipal district advisors prepare a first draft of the needs-assessment report. In step two, the draft needs-assessment report of each neighbourhood is discussed with community stakeholders to match the conclusions based on the quantitative data with their daily experiences and to gain local support by setting joint goals. Subsequently, in step three, the needs-assessment report is adapted based on input of the community stakeholders resulting in a final needs-assessment report of the neighbourhood. In step four, the municipal district advisors inventory the currently available measures, interventions, facilities and activities in the neighbourhood and assess their presence in the so-called database Effective Youth Interventions (EYI) of the Netherlands Youth Institute (NYI) . This is a comprehensive database of all nationally available evidence-based interventions for children and youngsters. The municipal district advisors will do this in collaboration with community stakeholders. In step five, outcomes of step four are discussed by the municipal district advisors with community stakeholders to assess which providers and which measures, interventions, facilities and activities are needed to complete the neighbourhood tailored intervention-package. Priority will be given to EYI database of the Netherlands Youth Institute (NYI) . In step six, a detailed neighbourhood intervention-package plan is drawn and all measures, interventions, facilities and activities that will be implemented are described. In step seven, the intervention-package plan including the proposed measures, interventions, facilities and activities is implemented in the neighbourhood. At last, step eight consists of continuous monitoring and revision of the intervention-package performed by municipal district advisors and community stakeholders in the neighbourhood. The monitoring consists of qualitatively and quantitatively evaluating whether intervention-packages in the neighbourhood have the intended results or not, why the results were achieved or not, and what can be done to achieve the previously set goals.
The Promising Neighbourhoods program will be implemented in three intervention neighbourhoods as described above.
In three comparable control neighbourhoods the Promising Neighbourhoods program will not be implemented during this evaluation study. Preventive measures, interventions, activities and facilities will take place in the control neighbourhoods as usual but there will be no collaborative community programming with stakeholders no data-based priority setting and no promotion of knowledge-, and theory-based policies and evidence-based interventions.
Evaluation strategy using a logic model
To evaluate the Promising Neighbourhoods program, a logic model is used (see Fig. 2). This logic model is used as overall guiding framework for the evaluation study . The logic model contains five stages: assets, input, output, intermediate outcomes and ultimate outcomes. All elements of the logic model, i.e. all different stages of the Promising Neighbourhoods program, will be studied and evaluated. The assets, input and output correspond with the process-implementation evaluation and the intermediate and ultimate outcomes correspond with the effect evaluation. The ‘assets stage’ consists of existing data and information about the participating neighbourhoods, the support from partners, community stakeholders, sponsors, and target groups in the neighbourhoods, municipal support, financial resources, expertise, knowledge, local collaboration and coordination. The ‘input stage’ includes the neighbourhood-specific needs-assessment of the municipal district advisors together with community stakeholders, the assessment of measures, interventions, facilities and activities, already present the in neighbourhood and the intervention-package and action plan. The ‘output stage’ includes the type, number, frequency, intensity, quality, costs and reach of the measures, interventions, facilities and activities and improved assets such as capacity building and empowerment. Targeted intermediate outcomes are improvements in risk and protective factors of youth health, safety and talent development. The targeted ultimate outcomes are improvements of socioeconomic inequalities in health, safety, and talent development among youth in Rotterdam.
The study consists of a process-implementation and effect evaluation of the Promising Neighbourhoods program using a mixed-methods design. The process-implementation evaluation corresponds to the assets, input and output of the logic model. Important elements such as reach, dose delivered, dose received and program fidelity from process-implementation evaluation strategies are incorporated in this evaluation [37, 38]. For the process-implementation evaluation, qualitative and quantitative measurements will be performed at baseline (T0) and at follow-up after implementation (T1) of the Promising Neighbourhoods program. Qualitative measurements will be performed using questionnaires and focus groups. Quantitative measurements will be performed using register-data and questionnaires. The effect evaluation corresponds with the intermediate and ultimate outcomes of the logic model. The effect evaluation is based on two separate cross-sectional samples and routinely collected data at baseline and at follow-up after implementation of the full Promising Neighbourhoods program.
This study is carried out in six different neighbourhoods in Rotterdam. Neighbourhoods in Rotterdam are categorized as low, middle or high based on the degree of experienced problems. The categorization is based on the percentage of children aged 4 and 12 years old with a high score on the Strengths and Difficulties Questionnaire (SDQ) and the percentage of overweight children in grade two of primary school in the neighbourhoods . The SDQ is a validated questionnaire to measure emotional problems, conduct problems, hyperactivity/inattention, peer problems, prosocial behaviour and total difficulties score . From each category an intervention neighbourhood was selected resulting in three intervention neighbourhoods. Intervention neighbourhoods were matched to control neighbourhoods as much as possible on these two experienced problems and on average socioeconomic status of the neighbourhoods.
For the process-implementation evaluation of the Promising Neighbourhoods program, three different groups are distinguished for data collection at baseline as well as at follow-up after the implementation. The first group consists of all community stakeholders in the field of youth (such as teachers, police officers, youth -, health, social -, community -, and sports workers) in each intervention neighbourhood and each control neighbourhood. We want to include all community stakeholders per neighbourhood, and we expect that the size of the population will range between 30 and 50. The second group consists of youngsters (aged ≥12–18 years) in the intervention neighbourhoods, and a comparable group of youngsters in the control neighbourhoods. We will include eight to ten youngsters from intervention neighbourhoods and eight to ten youngsters from control neighbourhoods. The third group consists of key-leaders who are participating in the Promising Neighbourhoods program in each intervention neighbourhood and a comparable group of key-leaders in each control neighbourhood. Key-leaders of the community know the neighbourhood from inside and are for example: police officers, school superintendents, heads of social services agencies, persons knowledgeable about prevention efforts in the community and municipal district advisors managing the process-implementation of the Promising Neighbourhoods program. We will include eight to ten key-leaders per neighbourhood.
For the effect evaluation of the Promising Neighbourhoods Program, the study population consists of N = 818 children aged 0 ≤ 11 years old (N = 409 at baseline and N = 409 at follow-up) and N = 818 youngsters aged 12 ≤ 18 years old (N = 409 at baseline and N = 409 at follow-up), living in the selected intervention and control neighbourhoods in Rotterdam.
The data collection that will be performed for this evaluation study is discussed separately for each stage of the logic model. The baseline measurement takes place from November 2018 to February 2019. Implementation of the Promising Neighbourhoods program in the intervention neighbourhoods will take place from February 2019 onwards, after the baseline measurement. The follow-up measurement will take place from September 2020 to March 2021.
Data regarding the assets (knowledge and expertise of community stakeholders about the neighbourhood such as existing problems, municipal support, support from partners, sponsors, and target groups, willingness to collaborate in the Promising Neighbourhoods program, and collaboration and coordination in the neighbourhood) will be studied among community stakeholders in all participating neighbourhoods using an anonymous online-questionnaire and focus groups (see Table 1). This online-questionnaire is based on an instrument (Community Board Interview) to collect information about community stakeholders in neighbourhoods and their role in the process-implementation and their characteristics . This instrument was used for evaluation of the CtC approach in the USA, the Netherlands, and Germany [17, 20, 40] and will be adapted for this process-implementation evaluation of Promising Neighbourhoods. The questionnaire will take at most 30 min to fill out and will be administered from November 2018 to February 2019 and from September 2020 to March 2021. Subsequently, focus groups addressing the assets will be organized from November 2018 to February 2019 and from September 2020 to March 2021 in each neighbourhood. In total, there will be six focus groups at baseline and six focus groups at follow-up. Additionally, two focus groups among youngsters (aged 12 ≤ 18 years old) will be organized, both at baseline and at follow-up; one group representing the intervention neighbourhoods and one group representing the control neighbourhoods. The same topics regarding the assets will be addressed. The focus groups among youngsters are specifically used to take the perspective of youngsters about the quality of neighbourhood conditions into account. To assess financial resources, we will use registration data of the municipality of Rotterdam.
Data regarding the input (effectiveness of neighbourhood networks consisting of community stakeholders in the neighbourhood, and existing measures, plans, interventions, facilities and activities) will be studied using the same online-questionnaire and focus groups in 2018/2019 and 2020/2021 as will be used to study the assets (see Table 1). The anonymous online-questionnaire and focus groups will be held among community stakeholders in all participating intervention and control neighbourhoods. Additionally, in the two focus groups among youngsters representing the intervention neighbourhoods and youngsters representing the control neighbourhoods (aged 12 ≤ 18 years old) the same topics regarding the input will be discussed. To assess existing measures, interventions, facilities and activities in the neighbourhood, we will use registration data of the municipality of Rotterdam and additional surveys of key-leaders.
Data regarding the output (costs, type, number, quality, frequency, intensity, reach and characteristics of the reached groups of the implemented measures, interventions, facilities, and activities, use of a knowledge/science-based approach, use of effective programs and monitoring of implementation) will be studied using a questionnaire (see Table 1). This questionnaire will be administered among key-leaders in November 2018 to February 2019 and from September 2020 to March 2021. This questionnaire is an adapted version of the Key-Informant questionnaire previously used in CtC evaluations in the USA, the Netherlands and Germany [17, 22]. This questionnaire will take at most 60 min to fill out and will be administered via telephone. During the focus groups with community stakeholders in 2020/2021, and the online-questionnaire prior to it, the contribution of assets and input to the realized output will be discussed. To examine the costs, type, number, quality, frequency, intensity, reach and characteristics of the reached groups of the implemented measures, interventions, facilities, and activities we will use registration data of the municipality of Rotterdam. These data are registered by the municipal district advisors and providers of the interventions (see Table 1).
In this study, the intermediate outcomes are the proximal results of the measures, interventions, facilities and activities that will be implemented in the neighbourhoods. Therefore, we will study indicators of the following targeted risk and protective factors among children and youngsters: family environment (parenting, child-parent relationship, family life, and family conflict), healthy exercise, nutrition behaviours, smoking and substance use, social cohesion, use of facilities and care, and bullying (see Table 1). Data on intermediate outcomes will be collected at two time points, in 2018 at baseline and in 2020/2021 at follow-up, separately for children aged 0 ≤ 11 years old and for youngsters aged 12 ≤ 18 years old.
For the baseline measurement of intermediate outcomes among 0 ≤ 11 years old children, we will use anonymous survey data from the Health survey obtained in 2018 by the municipality of Rotterdam. The Health survey is administered every four years in a random sample drawn from the municipal population register using online parent-questionnaires. The Health survey consists of questionnaires addressing the following topics: general health, nutrition and exercise behaviour, home-environment, emotional and psychological health, neighbourhood-, and school environment, use of care and facilities, smoking and alcohol . Follow-up data will be collected by administering a comparable parent-questionnaire for children aged 0 ≤ 11 years old in 2020/2021. This questionnaire will be based on the Health survey and will include similar questions. We will use the same procedures for sample selection and administration of the follow-up questionnaire, as used by the municipality for the Health survey (the baseline data).
For the baseline and follow-up measurements in the group of 12 ≤ 18 years old youngsters we will use survey-data obtained by the Child and Family Centre Rijnmond in 2018 and 2020/2021 respectively. These survey data consist of the SDQ and the so-called You and Your health questionnaire which includes several validated questionnaires [39, 41,42,43]. The You and Your health questionnaire addresses the following topics: general health, nutrition and exercise behaviour, home-environment, emotional and psychological health, school environment, performing anxiety and learning behaviour, smoking, alcohol, substance use and gaming. These questionnaires are routinely administered by the Child and Family Centre Rijnmond every year for municipal health monitoring. The questionnaires are filled out online at school, the SDQ in first grade and You and your Health in third grade.
Ultimate outcome measures are: indicators of health (socio-emotional and/or psychological problems, general/physical health, overweight), safety (safety of the home environment, safety of the neighbourhood) and talent development (school performance, truancy) (see Table 1). Ultimate outcomes will be collected at the same two time-points, at baseline and at follow-up after implementation of the Promising Neighbourhoods program. The same instruments as for the intermediate outcomes will be used, separately for children aged 0 ≤ 11 years old and for youngsters aged 12 ≤ 18 years old. Since truancy, school performance and safety in the neighbourhood are not administered by the Health survey, SDQ or You and Your Health questionnaire we will use routinely collected data on the individual, school and neighbourhood level from the municipality of Rotterdam for these outcomes.
In addition to the intermediate and ultimate outcomes, demographical data including neighbourhood, age, sex, ethnicity, and socioeconomic status will be collected for the effect evaluation. Socioeconomic status will be measured as the highest educational level obtained either by both parents or the mother for children aged 0 ≤ 11 years old and current educational level of youngsters aged 12 ≤ 18 years old. Educational level will be classified according to the International Standard Classification of Education 2011 . Demographical data will be obtained from the municipality of Rotterdam and the Child and Family Centre Rijnmond.
The size of the data samples needed to determine small-effect sizes (f2 = 0.02) is calculated for the effect evaluation . Analyses will be performed separately for children aged 0 ≤ 11 years old and for youngsters aged 12 ≤ 18 years old as outcome-variables of interest might differ between age groups and are measured in a different way. Based upon 5% two-sided significance tests of the null hypothesis that socioeconomic status-groups in the intervention and control neighbourhoods do not differ on outcome variables at follow-up and a power of 80% allowing for 10 independent variables in the model, we need two samples of 818 individuals, one sample including children aged 0 ≤ 11 years old and one sample including youngsters aged 12 ≤ 18 years old. Half of each sample will consist of individuals at baseline, evenly distributed over the conditions (intervention or control neighbourhood). The other half will consist of individuals at follow-up, evenly distributed over conditions as well. For the baseline measurement of both age groups, we will have sufficient survey data. For the follow-up measurement of youngsters aged 12 ≤ 18 years old sufficient survey data is available because the Child and Family Centre Rijnmond administers their survey every year. However, since the Health survey of the municipality of Rotterdam is administered only once every four years we will not have sufficient follow-up data for children aged of 0 ≤ 11 years old. Therefore, we will gather these data by administering an additional comparable parent-questionnaire. With a predicted response rate of 40% we will need to reach 1025 parents in order to receive 409 follow-up questionnaires.
The process-implementation evaluation consists of the assets, input and output stages in the logic model. All focus groups and questionnaires will be documented and analysed using standardized formats. The qualitative data will be analysed using a set of codes. At the start of the analysis, a codebook will be made with herein a list of pre-set codes on different levels such as concepts or categories. This codebook will be made by two researchers independently coding the first two focus groups. During the analysis new codes may occur. These codes will be added to the codebook. Questionnaire data and register-data per intervention and per neighbourhood about the costs, type, number, quality, frequency, intensity, reach, and characteristics of the reached groups will also be analysed to detect changes between the baseline measurement and the follow-up measurement. Intervention and control neighbourhood results will be compared. Because of the relative small size of the groups in the process-implementation evaluation, T-tests with Bayesian statistics will be used [46, 47]. This will be performed using the program Jags in R. Intervention and control estimates will be compared, each chain with 5000 iterations and each time 500 not used. At the end, the estimates will be made with a total of 4500 estimates.
In the effect evaluation of the Promising Neighbourhoods program, effects on the dependent intermediate outcomes (indicators of risk and protective factors) and ultimate outcomes (indicators of health, safety and talent development) of the logic model will be examined. We will perform Difference-in-Difference regression analyses separately for children aged 0 ≤ 11 years old and for youngsters aged 12 ≤ 18 years old. Predictor variables of interest are time of measurement (baseline/follow-up), condition (intervention/control), socioeconomic status, and their two-way and three-way interactions controlled for neighbourhood conditions. We will also adjust for other sociodemographic covariates. The main variable of interest in our study is the three-way interaction term for time of measurement, condition, and socioeconomic status. A significant parameter (p < 0.05) for this interaction term indicates a change in outcome over time in the intervention neighbourhoods with a socioeconomic status gradient. In case of continuous outcome variables linear regression analysis will be used and for dichotomous outcome variables logic regression analysis will be used. Missing data will be handled using multiple imputation. Level of significance is set at 0.05 for two-tailed analyses.
This article describes the design and methodology of a mixed-methods study for evaluation of the process-implementation and effectiveness of the Promising Neighbourhoods program. Promising Neighbourhoods is a program aiming to reduce socioeconomic health inequalities and to increase health, safety and talent development among youth in different neighbourhoods of Rotterdam. The Promising Neighbourhoods program will be gradually implemented in all neighbourhoods in Rotterdam, if proven effective.
We hypothesise that a collaborative community programming approach with stakeholders leads to clear priority setting and better tailored interventions of better quality. Furthermore, we hypothesise a decline in socioeconomic inequalities in intermediate and ultimate outcomes for health, safety and talent development in intervention neighbourhoods in comparison to control neighbourhoods. As we also study the process-implementation of the Promising Neighbourhoods program, we will be able to provide relevant insights on possible facilitators and barriers for future implementation of policy programs using a collaborative community approach, such as Promising Neighbourhoods.
A strength of this study is that we study the process-implementation as well as the effectiveness . Without evaluation of the process-implementation, the black box of why a program is effective or not effective cannot be disentangled . Furthermore, the process-implementation evaluation can also shed light on possible defects or unwanted side effects within the Promising Neighbourhoods program . Adding to this, it may elucidate how to successfully adapt programs and reach specific communities. Therefore, this study is relevant for local settings and collaborative programming and for national governments that depend on successful local implementation of policies. Another strong aspect is that we evaluate the process-implementation and effectiveness in different neighbourhoods increasing the generalisability of our findings.
The limitations that need to be taken into consideration are first that long term effects are not part of our study design. The time of two years between the baseline and follow-up does not allow studying longer term outcomes on especially the intended ultimate outcomes in the current study. Second, there is always a chance for residual confounding even though we will adjust for confounding in our analyses and tried to match intervention neighbourhoods as much as possible based on the degree of experienced problems and on socioeconomic status. We cannot control planned or unplanned implementation of interventions in the neighbourhoods on the initiative of other institutions or in some cases even of collaborating community stakeholders. Moreover, we cannot control for the fact that children, youngsters and their families in control neighbourhoods may receive or take part in interventions or activities provided in the intervention neighbourhoods. Or that they take part in interventions or activities implemented independently of interventions that are part of the Promising Neighbourhoods program. However, to monitor this, we will obtain registration data about implemented measures, interventions, facilities and activities in the neighbourhoods.
In conclusion, this study will provide knowledge about the process-implementation and effectiveness of a collaborative community programming approach implemented together with stakeholders, using data-based priority setting, knowledge-, and theory-based policies and evidence-based interventions. The results following from this study may be used for the design, implementation and transferability of intervention programs aiming to reduce health inequalities among youth using a collaborative community programming approach with stakeholders. Therefore, our study is relevant for local and national public health authorities and for improvement of the health, safety, and talent development among youth.
Collaborative community program
Communities that Care
Ensemble Prévenons l’Obésité Des Enfants/Together Lets Prevent Obesity
Effective Youth Interventions
International Committee of Medical Journal Editors
Netherlands Youth Institute
Strengths and Difficulties Questionnaire
United States of America
Mackenbach JP. Socioeconomic inequalities in health in the Netherlands: impact of a five year research programme. Bmj. 1994;309(6967):1487–91.
Mackenbach JP, Kunst AE, Cavelaars AE, Groenhof F, Geurts JJ. Socioeconomic inequalities in morbidity and mortality in western Europe. The EU Working Group on Socioeconomic Inequalities in Health. Lancet. 1997;349(9066):1655–9.
Mackenbach JP, Stronks K. A strategy for tackling health inequalities in the Netherlands. Bmj. 2002;325(7371):1029–32.
Marmot M. Commission on Social determinants of health: achieving health equity: from root causes to fair outcomes. Lancet. 2007;370(9593):1153–63.
Marmot M, Bell R. Fair society, healthy lives. Public Health. 2012;126:S4–S10.
Yoshikawa H, Aber JL, Beardslee WR. The effects of poverty on the mental, emotional, and behavioral health of children and youth: implications for prevention. Am Psychol. 2012;67(4):272–84.
World Health Organization, Commission on Social Determinants of Health. closing the gap in a generation: health equity through action on the social determinants of health: final report of the commission on Social determinants of health. Geneva: World Health Organization; 2008.
Weiss D, Lillefjell M, Magnus E. Facilitators for the development and implementation of health promoting policy and programs - a scoping review at the local community level. BMC Public Health. 2016;16:140.
World Health Organization. Addressing the social determinants of health: the urban dimension and the role of local government. Copenhagen: World Health Organization Regional Office for Europe; 2012.
Lorenc T, Oliver K. Adverse effects of public health interventions: a conceptual framework. J Epidemiol Community Health. 2014;68(3):288–90.
Barry MM. Researching the implementation of community mental health promotion programs. Health Promot J Austr. 2007;18(3):240–6.
Barry M. Designing an evaluation framework for community mental health promotion. J of Public Ment Health. 2003;2(4):26–36.
Barry M, Reynolds C, Sheridan A, Egenton R. Implementation of the JOBS programme in Ireland. J of Public Ment Health. 2006;5(4):10–25.
Borys JM, Le Bodo Y, Jebb SA, Seidell JC, Summerbell C, Richard D, De Henauw S, Moreno LA, Romon M, Visscher TL, et al. EPODE approach for childhood obesity prevention: methods, progress and international development. Obes Rev. 2012;13(4):299–315.
Romon M, Lommez A, Tafflet M, Basdevant A, Oppert JM, Bresson JL, Ducimetiere P, Charles MA, Borys JM. Downward trends in the prevalence of childhood overweight in the setting of 12-year school- and community-based programmes. Public Health Nutr. 2009;12(10):1735–42.
Pettigrew S, Borys JM, du Plessis HR, Walter L, Huang TT, Levi J, Vinck J. Process evaluation outcomes from a global child obesity prevention intervention. BMC Public Health. 2014;14:757.
Steketee M, Oesterle S, Jonkman H, Hawkins JD, Haggerty KP, Aussems C. Transforming prevention systems in the United States and the Netherlands using communities that care promising prevention in the eyes of Josine Junger-Tas. Eur J Crim Pol Res. 2013;19(2):99–116.
Oesterle S, Kuklinski MR, Hawkins JD, Skinner ML, Guttmannova K, Rhew IC. Long-term effects of the communities that care trial on substance use, antisocial behavior, and violence through age 21 years. Am J Public Health. 2018;108(5):659–65.
Jonkman HB, Haggerty KP, Steketee M, Fagan A, Hanson K, Hawkins JD. Communities that care, Core elements and context: Research of implementation in two countries. Soc dev issues. 2009;30(3):42–57.
Arthur MW, Hawkins JD, Brown EC, Briney JS, Oesterle S, Social Development Research G, Abbott RD. Implementation of the communities that care prevention system by coalitions in the community youth Development study. J community psychol. 2010;38(2):245–58.
Brown EC, Hawkins JD, Arthur MW, Briney JS, Abbott RD. Effects of communities that care on prevention services systems: findings from the community youth Development study at 1.5 years. Prev Sci. 2007;8(3):180–91.
Brown EC, Hawkins JD, Arthur MW, Briney JS, Fagan AA. Prevention service system transformation using communities that care. J Community Psychol. 2011;39(2):183–201.
Hawkins JD, Catalano RF, Arthur MW, Egan E, Brown EC, Abbott RD, Murray DM. Testing communities that care: the rationale, design and behavioral baseline equivalence of the community youth Development study. Prev Sci. 2008;9(3):178–90.
Hawkins JD, Oesterle S, Brown EC, Abbott RD, Catalano RF. Youth problem behaviors 8 years after implementing the communities that care prevention system in a community-randomized trial. JAMA Pediatr. 2014;168(2):122–9.
Gomez SF, Casas Esteve R, Subirana I, Serra-Majem L, Fletas Torrent M, Homs C, Bawaked RA, Estrada L, Fito M, Schroder H. Effect of a community-based childhood obesity intervention program on changes in anthropometric variables, incidence of obesity, and lifestyle choices in Spanish children aged 8 to 10 years. Eur J Pediatr. 2018;177(10):1531–9.
Vinck J, Brohet C, Roillet M, Dramaix M, Borys JM, Beysens J, Jacobs N, Jebb S, De Laet C, Neve J. Downward trends in the prevalence of childhood overweight in two pilot towns taking part in the VIASANO community-based programme in Belgium: data from a national school health monitoring system. Pediatr Obes. 2016;11(1):61–7.
Crow I, France A, Hacking S, Hart M. Does communities that care work? An evaluation of a community-based risk prevention programme in three neighbourhoods. York: Joseph Rowntree Foundation; 2004.
Jonkman HB, Junger-Tas J, van Dijk B. From behind dikes and dunes: communities that care in the Netherlands. Child Soc. 2005;19(2):105–16.
Jonkman H, Aussems C, Steketee M, Boutellier H, Cuijpers P. Prevention of problem Behaviours among adolescents: the impact of the communities that care strategy in the Netherlands (2008–2011). Int J Dev Sci. 2015;9(1):37–52.
Merzel C, D'Afflitti J. Reconsidering community-based health promotion: promise, performance, and potential. Am J Public Health. 2003;93(4):557–74.
Mieloo C, de Haan C, Jansen W, Stoorvogel H, Wiering D: Youth Monitor (Staat van de jeugd 2018), Rotterdam: Municipality of Rotterdam; 2018.
Municipality of Rotterdam: Youth Policy 2015–2020 (Beleidskader Jeugd 2015–2020). In. Rotterdam, The Netherlands; 2015.
Borys JM, Richard P, Ruault du Plessis H, Harper P, Levy E. Tackling health inequities and reducing obesity prevalence: the EPODE community-based approach. Ann Nutr Metab. 2016;68(Suppl 2):35–8.
Youth Monitor (Staat van de jeugd). https://www.rotterdam.nl/werken-leren/staat-van-de-jeugd. Accessed 20 Nov 2018.
Database effective youth interventions (databank Effectieve Jeugdinterventies). https://www.nji.nl/nl/Databank/Databank-Effectieve-Jeugdinterventies. Accessed 10 Oct 2018.
Cooksy LJ, Gill P, Kelly PA. The program logic model as an integrative framework for a multimethod evaluation. Eval Program Plann. 2001;24(2):119–28.
Saunders RP, Evans MH, Joshi P. Developing a process-evaluation plan for assessing health promotion program implementation: a how-to guide. Health Promot Pract. 2005;6(2):134–47.
Nutbeam D. Evaluating health promotion—Progress, problems and solutions. Health Promot Int. 1998;13(1):27–44.
van Widenfelt BM, Goedhart AW, Treffers PD, Goodman R. Dutch version of the strengths and difficulties questionnaire (SDQ). Eur Child Adolesc Psychiatry. 2003;12(6):281–9.
Jonkman H. Communities That Care in Niedersachsen. In: Ergebnisse der Implementation von CTC in fünf Standorten (2013–2014). Hannover: LPR; 2015.
Thombs BD, Bernstein DP, Lobbestael J, Arntz A. A validation study of the Dutch childhood trauma questionnaire-short form: factor structure, reliability, and known-groups validity. Child Abuse Negl. 2009;33(8):518–23.
Kosters MP, Chinapaw MJ, Zwaanswijk M, van der Wal MF, Koot HM. Structure, reliability, and validity of the revised child anxiety and depression scale (RCADS) in a multi-ethnic urban sample of Dutch children. BMC Psychiatry. 2015;15:132.
Verlinden E, van Meijel EP, Opmeer BC, Beer R, de Roos C, Bicanic IA, Lamers-Winkelman F, Olff M, Boer F, Lindauer RJ. Characteristics of the Children's revised impact of event scale in a clinically referred Dutch sample. J Trauma Stress. 2014;27(3):338–44.
Unesco Institute for Statistics. International standard classification of education: ISCED 2011. Montreal: UNESCO Institute for Statistics; 2012.
cohen J. Statistical power analysis for the behavioral sciences second edition. Hillsdale: Lawrence Erlbaum Associates Inc.; 1988.
Cowles MK. Applied Bayesian statistics: with R and OpenBUGS examples, vol. 98. New York: Springer Science & Business Media; 2013.
Johnson W, Branscum A, Hanson TE, Christensen R. Bayesian ideas and data analysis: an introduction for scientists and statisticians. Boca Raton: CRC Press; 2010.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Int J Nurs Stud. 2013;50(5):587–92.
ICJME Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals. http://www.icmje.org/icmje-recommendations.pdf. Accessed 20 Feb 2019.
The authors thank the involved program-managers and municipal district advisors of Promising Neighbourhoods from the city of Rotterdam.
This work was funded by a research grant (project number: 531001313) from ZonMw, The Netherlands Organization for Health Research and Development. ZonMw has no role in any part of the research, writing and reviewing of the manuscript.
Availability of data and materials
Data availability and sharing is non-applicable to this particular study as datasets are not yet generated nor analysed for this study protocol. Future study results will be communicated according to the International Committee of Medical Journal Editors (ICJME) recommendations for authorship  via relevant scientific journals without any publication restrictions. A data management plan and a checklist for processing data on individual persons is set up for this study. The dataset that will be generated and analysed for future publications, the data management plan and a checklist for processing data on idividual persons is available from the corresponding author on reasonable request and with permission of the third parties involved.
Ethics approval and consent to participate
The medical ethics committee of the Erasmus University Medical Center Rotterdam has declared that the Medical Research Involving Human Subjects Act does not apply to this research protocol and issued a declaration of no objection for this study (registration number MEC-2018-1506).Therefore, this study can be carried out without further approval by an accredited medical ethical committee. This study has been entered into the Netherlands National Trial Register (NTR; https://www.trialregister.nl/trial/7279) under catalogue number NL7279 on 26 September 2018. Any important protocol modifications will be reported by the researchers to the funding body and the trial registry.
Written informed consent will be asked from community stakeholders and youngsters for participation in the focus groups that are part of the process and implementation evaluation For youngsters aged 12 ≤ 16 years old the parents will also be asked to additionally give written informed consent before participating in the focus groups. Verbal informed consent will be asked from the key-leaders for the key-informant questionnaire, that is part of the process and implementation evaluation. Verbal informed consent is used among key-leaders as the questionnaire is administered by telephone. For the questionnaires among community stakeholders and all effect evaluation questionnaires informed consent is assumed by returning a completed anonymous questionnaire.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Boelens, M., Windhorst, D.A., Jonkman, H. et al. Evaluation of the promising neighbourhoods community program to reduce health inequalities in youth: a protocol of a mixed-methods study. BMC Public Health 19, 555 (2019). https://doi.org/10.1186/s12889-019-6901-3
- Stakeholder participation
- Socioeconomic prevention