Countdown to 2015 country case studies: systematic tools to address the “black box” of health systems and policy assessment
- Neha S. Singh1Email author,
- Luis Huicho2, 3, 4,
- Hoviyeh Afnan-Holmes5,
- Theopista John6,
- Allisyn C. Moran7,
- Tim Colbourn8,
- Chris Grundy1,
- Zoe Matthews9,
- Blerta Maliqi10,
- Matthews Mathai10,
- Bernadette Daelmans10,
- Jennifer Requejo11,
- Joy E. Lawn1 and
- On behalf of the Countdown to 2015 Health Systems and Policies Technical Working Group
© The Author(s). 2016
Published: 12 September 2016
Evaluating health systems and policy (HSP) change and implementation is critical in understanding reproductive, maternal, newborn and child health (RMNCH) progress within and across countries. Whilst data for health outcomes, coverage and equity have advanced in the last decade, comparable analyses of HSP changes are lacking. We present a set of novel tools developed by Countdown to 2015 (Countdown) to systematically analyse and describe HSP change for RMNCH indicators, enabling multi-country comparisons.
International experts worked with eight country teams to develop HSP tools via mixed methods. These tools assess RMNCH change over time (e.g. 1990–2015) and include: (i) Policy and Programme Timeline Tool (depicting change according to level of policy); (ii) Health Policy Tracer Indicators Dashboard (showing 11 selected RMNCH policies over time); (iii) Health Systems Tracer Indicators Dashboard (showing four selected systems indicators over time); and (iv) Programme implementation assessment. To illustrate these tools, we present results from Tanzania and Peru, two of eight Countdown case studies.
The Policy and Programme Timeline tool shows that Tanzania’s RMNCH environment is complex, with increased funding and programmes for child survival, particularly primary-care implementation. Maternal health was prioritised since mid-1990s, yet with variable programme implementation, mainly targeting facilities. Newborn health only received attention since 2005, yet is rapidly scaling-up interventions at facility- and community-levels. Reproductive health lost momentum, with re-investment since 2010. Contrastingly, Peru moved from standalone to integrated RMNCH programme implementation, combined with multi-sectoral, anti-poverty strategies.
The HSP Tracer Indicators Dashboards show that Peru has adopted nine of 11 policy tracer indicators and Tanzania has adopted seven. Peru costed national RMNCH plans pre-2000, whereas Tanzania developed a national RMNCH plan in 2006 but only costed the reproductive health component. Both countries included all lifesaving RMNCH commodities on their essential medicines lists. Peru has twice the health worker density of Tanzania (15.4 vs. 7.1/10,000 population, respectively), although both are below the 22.8 WHO minimum threshold.
These are the first HSP tools using mixed methods to systematically analyse and describe RMNCH changes within and across countries, important in informing accelerated progress for ending preventable maternal, newborn and child mortality in the post-2015 era.
The Millennium Development Goals (MDGs) ended in 2015, when the 189 signatory countries assessed progress made in the past 15 years. At the heart of the MDGs are MDG4, which called for a reduction of child mortality by two-thirds, and MDG5, which focused on improvement of maternal health through a reduction of maternal mortality by three-quarters and universal access to reproductive health care .
The past decade has seen an encouraging increase in the availability of data and development of methods for evaluating influence coverage levels including patterns of equity, and contextual variables impact on RMNCH. Measuring changes in policy and systems environments, including strength of policy to programme implementation, is also key to understanding which interventions have the greatest impact and to be able to anticipate future health gains [5–12]. Implementation data can also help in evaluating and improving progress toward specific RMNCH outcomes and intervention strategies [7, 10, 13–15]. However, a “black box” remains around how to assess the health systems inputs section of the evaluation framework in a systematic and standardised manner (Fig. 1), as the relationship between policy formulation and implementation, and changes in health systems inputs are unclear with limited scientific evidence. Though advances have been made on defining the stages of policy formulation , there is limited multi-country assessment tracking the policy formulation to implementation pathway, especially from lower and middle-income countries, impeded by a lack of HSP definitions, data and standardised tools. To date, more quantitative approaches have been used to assess HSP changes [16, 17], than qualitative approaches . A recent systematic review on implementation strength concluded that currently there is no consensus on measuring implementation strength of RMNCH interventions with consistent definitions and methodologies .
Policy and Programme Timeline Tool to identify what policies and major systems changes were introduced for RMNCH over time or the lack thereof from 1990 to present;
Health Policy Tracer Indicators Dashboard to track a set of key tracer RMNCH policy indicators as tracked on Countdown country profiles;
Health Systems Tracer Indicators Dashboard to assess key tracer health systems indicators as per Countdown country profiles; and
Programme implementation assessment using geographical mapping with implementation readiness barometers to assess health system readiness to implement RMNCH interventions.
Countdown developed HSP tools to standardise the type of data collected (inputs), methods for assessing (process) and presenting these data (outputs) for each of the countries included in the portfolio of the country case studies, to assess and describe HSP factors that have contributed to change, or lack thereof, in RMNCH in a country, and to enable cross-country comparison of policy and systems changes.
Policy and programme timeline tool
The Policy and Programme Timeline Tool (Additional file 1) aims to assess the agenda setting component of the policy heuristic. This tool was developed to provide an overall view of health policies and programs of a country by examining changes in RMNCH policy, programs, and implementation from 1990 to the current year.
The tool is based on the Timeline tool developed for the Health Policy and Planning supplement on a multi-country evaluation of progress in newborn survival, policy and programmes . The tool was adapted for use in Countdown case studies by broadening its scope to include macro health systems, health system building blocks and high impact policies and research specific to RMNCH.
Process to apply the tool
Standard process for completing the Countdown to 2015 policy and programme timeline
Drafting the policy and programme timeline
Country team to fill in the Policy and Programme Timeline Tool
Led by country teams with technical support as relevant
Country-specific analysis of policy and programme timeline
Country team to approach country partners/stakeholders to share the Policy and Programme Timeline and to use standardised questions as per the tool’s protocol to build consensus on what has changed and what has had the most impact on RMNCH via policy, programs, and implementation in their country
Led by country team, country partners/stakeholders and in some cases, additional support
Synthesising results from policy and programme timeline analysis
▪ Analysis of common themes across countries as relevant
Led by country teams with additional technical and graphic support as relevant
▪ Draft standardised graphics representing analysis results for use in journal articles, policy briefs and/or other dissemination outputs as relevant
In contrast, Fig. 5 shows that Peru started with standalone RMNCH programmes, and has since moved to integrated policies and implementation, to combine multi-sectoral, anti-poverty and RMNCH programmes. The introduction of vertical childbirths and provision of waiting houses for pregnant women are an example of pro-poor and human rights-based approaches to close the health facility coverage gap for rural births.
Health policy tracer indicators dashboard
The Health Policy Tracer Indicators Dashboard (Additional file 3) was developed to document, in a comparable way, 11 RMNCH tracer policy indicators reported in the Countdown country profiles to produce the Countdown Policy Dashboard for the country. This tool was developed to provide a systematic overview of the policy tracer indicators across the RMNCH continuum of care for a country by examining changes in these indicators from 1990 to present. The tool assesses and describes which RMNCH interventions have been translated into national-level policies (i.e. policy formulation component of the heuristic), and its visual output uses a “traffic light” colour coding system to illustrate if a policy has been fully (green) or partially (yellow) adopted, or does not exist (red).
The tracer indicators cover different areas of policies across RMNCH continuum of care. Data sources include the World Health Organisation (WHO) Maternal, Newborn, Child and Adolescent (MNCAH) Systems and Policy Survey, and specific sources e.g. the WHO abortion database and International Labour Organisation (ILO) maternity protection legislation database.
- Family planning for adolescents
- Legal status of abortion
- Midwives authorised for specific tasks
- Maternity protection (Convention 183)
- Maternal deaths notification
- Postnatal home visits in the first week after birth
- Kangaroo Mother Care for low birth weight newborns
- Antenatal corticosteroids as part of management of preterm labour
- International code of marketing of breastmilk substitutes
- Community treatment of pneumonia with antibiotics
- Low osmolality oral rehydration salts (ORS) and zinc for management of diarrhoea
The policy tracer indicators have been selected for global tracking and inclusion on the Countdown country profiles by the Countdown HSP Technical Working Group. These indicators are reviewed periodically and modified according to the latest evidence base. Definitions for the policy tracer indicators, including what constitutes a fully or partially adopted policy, are available in the Additional file 2.
Process to apply the tool
For each of these policy tracer indicators and specific components, respondents, e.g. relevant policy experts, stakeholders and implementers, are requested to review data sources and national policy documents and to select “Yes” or “No” from a dropdown menu if the policy or specific component exists. If the policy exists, respondents are requested to select “Yes” under the period when the policy was endorsed. These data are then presented in the Policy Dashboard, a visual representation of the data, which are reviewed by stakeholders in country in order to build consensus on the documentation of changes in selected RMNCH tracer policy and systems indicators. The tool's protocol with detailed methods is available in Additional file 2.
In contrast, the Health Policy Tracer Indicators Dashboard for Peru (Fig. 6b) shows that it has adopted nine of the 11 policy tracer indicators since 2010 including maternal death notification, antenatal corticosteroids and maternity protection. Peru has partially adopted the remaining two policy tracer indicators, with only one of five circumstances adopted for legal status of abortion, and midwives authorised for five of seven tasks.
Health systems tracer indicators dashboard
The Health Systems Tracer Indicators Dashboard (Additional file 4) was developed to assess and describe, in a comparable way, selected RMNCH systems tracer indicators reported in the Countdown country profiles. This tool assesses and describes key components of the health system necessary to implement RMNCH policies, and focuses on assessing the policy implementation component of the heuristic. The tool was developed to provide a systematic overview of four systems tracer indicators of a country across the RMNCH continuum of care by examining changes in these indicators from 1990 to the current year. The tool’s visual output uses a “traffic light” colour coding system to illustrate if each systems indicators has been fully (green), partially (yellow), or not achieved (red).
These tracer indicators cover key dimensions of the health system essential for RMNCH including the WHO health system building blocks. Data sources include the WHO MNCAH Systems and Policy Survey for costed plans, a number of sources including the "USAID Deliver Project, World Health Organisation, International Consortium for Emergency Contraception" and the Chlorhexidine Working Group for essential commodities, emergency obstetric care (EmOC) surveys by WHO, United Nations Population Fund (UNFPA) and the Averting Maternal Death and Disability Programme (AMDD), and WHO Global Health Observatory for health workforce.
- National strategy/plans of action to improve RMNCH
- Integration of selected lifesaving commodities in essential medicines and supplies list
- Density of health professionals
- National availability of EmOC services
The systems tracer indicators have been selected for global tracking and inclusion on the Countdown country profiles by the Countdown HSP Technical Working Group, and are reviewed periodically and modified according to the latest evidence base. Definitions and data sources for these indicators are available in the Additional file 2.
Process to apply the tool
To use the Tool, respondents answer specific questions and collate the data within the tool. Data reported for “Density of health professionals” and “National availability of emergency obstetric care services” are then compared to globally agreed benchmarks. Lastly, all data within the tool are reviewed by stakeholders in country for validation purposes and to build consensus on their interpretation of changes in selected RMNCH tracer systems indicators. This tool's protocol with detailed methods is available in Additional file 2.
Programme implementation assessment
Using the WHO health system building blocks as a foundation, we have developed a health system implementation readiness barometer, to be overlaid over a heat map of a country showing subnational variation in RMNCH outcomes. Its aim is to identify good and bad performing districts or regions, and to try to understand why there is subnational variation in implementation of RMNCH interventions, with a focus on service availability and service readiness.
Tool - Implementation readiness barometer
The implementation readiness barometer uses categorical data for four health systems building blocks — health financing, workforce, commodities, and facilities — pre-allocated to red, orange, yellow and green categories of strength based on international or national benchmarks as relevant. The circle shape of the barometer demonstrates the ‘wholeness’ of health system strength, i.e. all four health system components are interlinked to achieve readiness to implement a RMNCH intervention.
Barometers are overlaid on heat maps of health outcomes, which can be generated using software such as Arc GIS and Quantum GIS.
Process to apply the tool
Implementation readiness barometers can be constructed at the district, regional or zonal level of a country depending on availability of subnational data. As an example, we used Arc GIS 10.3 software to construct district-level maps in Tanzania with implementation readiness barometers to show variations in reproductive and child health outcome using DTP3 coverage and demand satisfied by modern methods of contraception as proxy indicators. The barometer was constructed using data from Tanzania’s Health Management Information System (HMIS) from Quarter 4 of 2014 [25, 26], the 2010 Tanzania Demographic and Health Survey (DHS) , the Human Resources for Health Country Profile (2012/13) , the 2012 Census , PMO-RALG Local Government Financial Report , and the Tanzanian Service Provision Assessment (SPA) Survey 2014  to present regional-level variations for indicators based on the following four WHO health systems building blocks : (i) workforce: skilled health workforce density per 10,000 population; (ii) commodities: availability of tracer drugs at health facilities; (iii) financing: per capita recurrent expenditure; and (iv) infrastructure: number of health facilities per 10,000 population.
The barometers show mixed implementation readiness by region in Tanzania (Fig. 8a and b). None of the regions meet all of the benchmarks for the four health systems building blocks, and the north-western regions and Dar es Salaam region have the weakest implementation readiness compared to other regions. Across Tanzania, health workforce and availability of tracer drugs in health facilities do not exceed 50 % of the required threshold. Figures 8a and b show a contrasting picture of coverage of child and family planning interventions in Tanzania. DTP3 coverage in Tanzania is relatively high with a range of 61-100 % (Fig. 8b), whereas the proportion of demand satisfied by modern methods of contraception is lower with a range of 21-63 % (Fig. 8a).
This paper addresses the “black box” of HSP assessment by presenting a comprehensive set of standardised tools to use mixed methods to systematically analyse and describe progress of policy formulation to implementation to discuss how and why RMNCH changes took place within and across countries. Although these tools do not address the cross-cutting governance, power and partnerships dimension component of policy change; they focus on assessing if there are signs that RMNCH issues are being prioritised for consideration in policy agendas, being considered in policy formulation, and being moved towards implementation – to prompt discussion with a view to making further progress. This evidence is important in informing accelerated progress for ending preventable maternal, newborn and child mortality in the post-2015 era. The study represents a step forward in relation to previous efforts to document the role of health policy and systems on the evolution of RMNCH at country and sub-national levels, which used qualitative and quantitative approaches [34, 35], but did not include specific tools and minimum standards to track progress.
The Policy and Programme Timeline tool uses a chronological timeline to document and analyse what policies, programmes, and implementation strategies, and pivotal moments have changed for RMNCH in a country. This tool builds on the Policy and Programme Timeline tool developed by Saving Newborn Lives to capture changes in key policy, programme and research achievements influencing newborn survival at national level . The policy and programme timeline tool developed by Countdown assesses variation for policies and strategies at the national level across the continuum of care, highlighting periods of policy intensity for RMNCH at the facility and community level. Outputs from this tool can be used to identify RMNCH policy gaps, target areas of policy to programme implementation for a more detailed assessment, and to compare policy changes with available data on trends in coverage, equity, and financing.
The Health Policy and Systems Tracer Indicators Dashboards are a standardised assessment of 11 RMNCH policy tracer indicators and four health systems tracer indicators reported on the Countdown country profiles and defined by the Countdown HSP Technical Working Group [36, 37]. These two dashboards chronologically assess the adoption of RMNCH policies and supporting health system building blocks at the national level in four time intervals from 1990 onward. More recently, a similar approach has been used by Saving Newborn Lives to develop a Benchmark Achievement Tool to assess readiness to scale up interventions for newborn survival , and by the Global Fund to assess progress toward programme objectives since 2004 . The two Health Policy and Systems Tracer Indicators Dashboards assess progress in policy formulation and key components of the health system across the continuum of care rather than benchmarks for newborn survival, and are key to understanding how RMNCH policy and system environments have changed within and across countries.
The implementation readiness barometer is a novel way of visualising subnational variations in RMNCH outcomes with health service readiness and availability using the WHO health system building blocks. A recent review shows that geographical mapping is increasingly being used to visualise national and subnational data for women and children’s health , for example mapping health facilities with availability of RMNCH commodities using SARA data , mapping subnational variation in health workforce density, for example the density of Health Surveillance Assistants by population and access to health facilities in Malawi , and mapping big data for global health, for example by the Institute for Health Metrics and Evaluation for the Global Burden of Disease study . The implementation readiness barometer presented in this paper builds on previous work by assessing more than one component of a health system against a RMNCH outcome. The construction of barometers using indicators for health financing, workforce, commodities, and infrastructure indicators paints a more comprehensive picture of health system strength, and can give clear indications of which components of the health system are succeeding or failing at the subnational level.
In addition to geographical mapping, bottleneck analysis using qualitative methods is another method that has been used to measure implementation strength. Dickson et al. built on analyses and evidence published previously in The Lancet Every Newborn Series , to adapt and apply the UNICEF Marginal Budgeting for Bottlenecks (MBB) tool in 12 Asian and African countries as part of the Every Newborn Action Plan process to synthesise bottlenecks hindering the scale up of maternal-newborn intervention packages across six health system building blocks . This tool analysed bottleneck by health system building blocks for each maternal-newborn intervention with tracer indicators for each intervention, and used large focus group assessments to finalise results.
Implementation strength has also been measured previously using quantitative methods such as scores or rating systems based on mixed method data collection using interviews or focus groups. Bergh et al. developed and tested a monitoring model with quantitative indicators or progress markers to measure the progress of individual hospitals in the implementation of Kangaroo Mother Care in South Africa , and have since used the model for a multi-country assessment in four African countries . This implementation assessment method showed variation in the quality of implementation of Kangaroo Mother Care between health facilities and across countries, and identified important factors for implementation of this intervention at the facility-level.
Benchmarks of health system readiness to implement integrated community case management of childhood illness (iCCM) have also been widely used . These benchmarks have been complemented by indicators of iCCM implementation strength developed by a consortium under leadership of the Institute of International Programmes (IIP) and Johns Hopkins University . The benchmarks and the indicators are based on the WHO health system building blocks and their application is widely promoted by WHO, UNICEF and partners in planning, management and review of maternal, newborn and child health programs. Nevertheless, collection of high-quality real time data remains a challenge.
Additional approaches to HSP analysis include Systems Thinking and Realist Review, which rely heavily on qualitative information [47, 48], take into account people, power and partnerships, and focus on the need to know at health system level not only what works but also for whom and under what circumstances, as well as to understand intended and unintended consequences resulting from the implementation of complex policy interventions. None of them include the use of specific tools, suggesting these HSP tools can be used to complement other analytic approaches.
Strengths, limitations and future research
These are the first HSP tools developed for application across a range of lower and middle-income countries in Africa, Asia and Latin America, and can be used in other settings to analyse and describe HSP dynamics and changes in a country as related to implementation of RMNCH interventions, enabling cross-country comparisons. However, limited data availability both at national and subnational level remain a barrier to analysing HSP changes and implementation. These tools make progress in linking changes in impact indicators to changes in policies and programmes; however, there is a need to build on these tools to further develop mixed methods to identify key HSP factors and also attributable fractions of mortality decline for individual RMNCH policies and related programmes.
These HSP tools can also be used to prompt national policy dialogue. For example, the HSP tools were used as part of the Countdown Tanzania case study analysis, with results used to inform accelerated focused action towards the end of the MDGs and contributing to ending preventable maternal, newborn, and child deaths by the end of the Sustainable Development Goals in 2030 . A policy brief with key messages on who is being left behind and where to focus efforts informed the evidence based Sharpened One Plan (2014 – 2015) . The policy brief and the Plan were disseminated at a high level event attended by His Excellency President JM Kikwete of Tanzania.
The policy heuristic is the theoretical framework used to underpin these HSP tools. However, they are limited in assessing the implementation stage of the heuristic, signalling the need for the development of standardised metrics for strength of programme implementation. Additionally, these HSP tools do not capture information on power, partnerships, leadership and champions and the important role they play in influencing all stages of the policy heuristic, i.e. agenda setting, policy formulation, implementation and evaluation, to understand why and how policy change took place [51–54].
Furthermore, the policy heuristic framework has been critiqued by Sabatier et al. for presuming a linearity to the public policy process that does not exist in reality, for postulating neat demarcations between stages that are blurred in practice, and for offering no propositions on causality . For example, the Countdown HSP tools document when, but not why changes in policies and programmes took place within and across countries, which raises the question of whether a linear approach to understanding policy adoption is appropriate, or if it should be combined with a political economy approach that recommends that policy adoption and decision making involves more than just evaluation of scientific evidence. Nevertheless, Walt et al. defend the heuristic as it offers a useful and simple way of thinking about the entire public policy process, and helps researchers situate their research within a wider framework .
When developing tools and metrics for HSP analysis, we also need to consider context-specific issues when setting-up reference standards, e.g. when considering facility-based versus community-based RMNCH interventions. For example, the latter may be privileged by some countries while the former may be considered a priority in health systems like in Peru.
There is a need for the development and validation of globally-agreed standards for WHO health systems building blocks to inform evidence-based benchmarking. We constructed benchmarks for health financing and commodities indicators due to lack of international standards. Varying benchmarks exist for health workforce and facilities, with little agreement on global standards, e.g. WHO’s minimum density threshold is 23 skilled health professionals per 10,000 population(34), whereas the International Labour Organisation recommendations vary from 35  to 41 health workers per 10,000 population . Grading data into traffic light categories can be reductive; for example, even the “green lighted” regions do not necessarily have the adequate amount of human resources, financing, RMNCH commodities or health facilities. However, the traffic light system of health system readiness is useful for national and subnational planning and assessment, and complements the new RMNCH scorecards in Tanzania and globally [58, 59].
None of the approaches used to date to assess RMNCH intervention implementation have been assessed comparatively or synthesised to perform an in-depth assessment of policy formulation to programme implementation pathways. However, with national and subnational-level data becoming increasingly available via establishment of routine data collections systems, there is a need to develop analysis methods to address this “black box” of HSP science. At the subnational level, data analysis methods need to be developed and applied to identify “stronger” and “weaker” performing districts on RMNCH outcomes and health systems indicators, and to evaluate subnational variation in implementation of the RMNCH interventions, with a focus on service availability and readiness. These novel subnational data analysis methods should link implementation strength to health outcomes, and be complemented with analyses of governance, power and partnerships to fully understand how and why policy change took place; thus allowing for assessment of national policy formulation to implementation within and across countries.
1. “Black box” of health system and policy (HSP) assessment: This paper presents a standard set of tools to systematically describe policy formulation for reproductive, maternal, newborn and child health and assess changes in programmes and implementation within and across countries, and over time. We adapted the tools from the ‘stages heuristic’ of the policy process (i.e. agenda setting, policy formulation, policy implementation and evaluation).
2. National and subnational change: To date, these tools have been mainly applied at national level to assess HSP change over time and by programme, but HSP assessment would be even more valuable and needs further development at subnational level, particularly to better understand variation in health outcomes.
3. Implementation strength: Metrics to track strength of implementation need more work to develop, particularly to be comparable between programmes and across geographies. These metrics would be especially valuable if linked to GIS data and considering finance, human resources and domains for service readiness.
4. Research gaps: Presentation of data using consistent visualisations may help with interpretation of complex policy changes, but further assessment of perceptions and use of such visualisations would be valuable. Future research should also include analyses of power, partnerships and governance to complement the HSP tools’ outputs by providing a deeper understanding of how and why policy change took place.
AMDD, Averting death and disability programme; EmOC, emergency obstetric care; HMIS, health management information system; HSP, health systems and policy; ILO, International Labour Organisation; MBB, marginal budgeting for bottlenecks; ORS, oral rehydration salts; RMNCH, reproductive, maternal, newborn and child health; SARA, service availability and readiness assessment; SPA, service provision assessment; UNCoLSC, United Nations commission for life saving commodities for women and children; UNFPA, United Nations population fund; UNICEF, United Nations children’s fund; WHO, World Health Organisation
We would like to thank the Countdown to 2015 country case study teams from Afghanistan, China, Ethiopia, Kenya, Malawi, Pakistan, Peru, and Tanzania for pilot testing the HSP tools and providing invaluable feedback. We also thank Evidence for Action’s MamaYe team for graphical support with Figs. 6, 7, and 8. We acknowledge Professor Lucy Gilson and Professor Nicholas Mays for their insightful and thoughtful review of the manuscript, and Dr. Helen Owen for assistance with Fig. 2 and coordinating all stages of the review process.
The costs for the writing and publication of this paper were provided through a sub-grant from the U.S. Fund for UNICEF under the Countdown to 2015 for Maternal, Newborn and Child Survival grant from the Bill & Melinda Gates Foundation, and from the Government of Canada, Foreign Affairs, Trade and Development.
This article has been published as part of BMC Public Health Volume 16 Supplement 2, 2016: Countdown to 2015 country case studies: analysing progress towards maternal and child survival in the Millennium Development Goal era. The full contents of the supplement are available online at http://bmcpublichealth.biomedcentral.com/articles/supplements/volume-16-supplement-2.
Availability of data and materials
Data supporting the conclusions of this article are included within the article.
NSS was responsible for the conception, analysis and writing process with oversight from JEL. TJ, HAH and LH contributed country data and experiences. All named authors contributed to the text and approved the final manuscript.
The authors declare they have no competing interests. The authors alone are responsible for the views expressed in this article and they do not necessarily represent the views of the organisations listed.
Consent for publication
Ethics approval and consent to participate
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Millennium Development Goals 2015 report. http://www.un.org/millenniumgoals/2015_MDG_Report/pdf/MDG%202015%20rev%20(July%201).pdf; Accessed 24 Mar 2016.
- Requejo JH, Bhutta ZA. The post-2015 agenda: staying the course in maternal and child survival. Arch Dis Child. 2015;100 Suppl 1:S76–81.View ArticlePubMedGoogle Scholar
- Requejo JH, Bryce J, Barros AJ, Berman P, Bhutta Z, Chopra M, Daelmans B, de Francisco A, Lawn J, Maliqi B, et al. Countdown to 2015 and beyond: fulfilling the health agenda for women and children. Lancet. 2014;385(9966):466–76.View ArticlePubMedGoogle Scholar
- Moucheraud C, Owen H, Singh NS, Ng CK, Requejo J, Lawn JE, Berman P, Countdown to 2015 Case Study Collaboration G. Countdown to 2015 country case studies: what have we learned about processes and progress toward MDGs 4 and 5? BMC Public Health. 2016; doi:10.1186/s12889-016-3401-6.
- Bryce J, Amouzou A, Hazel E, Miller N, Johns B, Gilroy K, Munos M, Victora C. Measuring the strength of implementation of community case management of childhood illness within the Catalytic Initiative to Save a Million Lives. Available at: http://www.jhsph.edu/research/centers-and-institutes/institute-for-international-programs/_documents/rapid_scaleup/wp-implementation-strength.pdf; Johns Hopkins Bloomberg School of Public Health. Accessed 24 Aug 2015.
- Dane AV, Schneider BH. Program integrity in primary and early secondary prevention: are implementation effects out of control? Clin Psychol Rev. 1998;18(1):23–45.View ArticlePubMedGoogle Scholar
- Durlak JA. Why program implementation is important? J Prev Interv Community. 1998;17:5–18.View ArticleGoogle Scholar
- Dusenbury L, Brannigan R, Falco M, Hansen WB. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18(2):237–56.View ArticlePubMedGoogle Scholar
- Hall E, Rosenthal J, Wade J. How to make reengineering really work. McKinsey Q. 1994;2:107–28.Google Scholar
- Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76.View ArticlePubMedGoogle Scholar
- Victora CG, Black RE, Boerma JT, Bryce J. Measuring impact in the Millennium Development Goal era and beyond: a new approach to large-scale effectiveness evaluations. Lancet. 2011;377(9759):85–95.View ArticlePubMedGoogle Scholar
- Berlan D, Buse K, Shiffman J, Tanaka S. The bit in the middle: a synthesis of global health literature on policy formulation and adoption. Health Policy Plan. 2014;29(3):iii23–34.View ArticlePubMedGoogle Scholar
- Damschroder LJ, Hagedorn HJ. A guiding framework and approach for implementation research in substance use disorders treatment. Psychol Addict Behav. 2011;25(2):194–205.View ArticlePubMedGoogle Scholar
- McGraw S, Sellers D, Stone E, Resnicow K, Kuester S, Fridinger F, Wechsler C, Wechsler H. Measuring implementation of school programs and policies to promote healthy eating and physical activity among youth. Preventative Med. 2000;31:S86–97.View ArticleGoogle Scholar
- Mowbray CT, Holter MC, Teague GB, Bybee D. Fidelity Criteria: Development, Measurement, and Validation. Am J Eval, 2003;24:315–40.View ArticleGoogle Scholar
- Lawn JE, Kinney MV, Black RE, Pitt C, Cousens S, Kerber K, Corbett E, Moran AC, Morrissey CS, Oestergaard MZ. Newborn survival: a multi-country analysis of a decade of change. Health Policy Plan. 2012;27(3):iii6–28.PubMedGoogle Scholar
- Dickson KE, Simen-Kapeu A, Kinney MV, Huicho L, Vesel L, Lackritz E, de Graft JJ, von Xylander S, Rafique N, Sylla M, et al. Every Newborn: health-systems bottlenecks and strategies to accelerate scale-up in countries. Lancet. 2014;384(9941):438–54.View ArticlePubMedGoogle Scholar
- Moran AC, Kerber K, Pfitzer A, Morrissey CS, Marsh DR, Oot DA, Sitrin D, Guenther T, Gamache N, Lawn JE, et al. Benchmarks to measure readiness to integrate and scale up newborn survival interventions. Health Policy Plan. 2012;27(3):iii29–39.PubMedGoogle Scholar
- IDEAS project. Measuring Implementation Strength: Literature review draft report 2012. Available at: http://ideas.lshtm.ac.uk/sites/ideas.lshtm.ac.uk/files/Report_implementation_strength_Final_0.pdf. London School of Hygiene & Tropical Medicine; 2012. Accessed 24 Aug 2015.
- Brewer GD, DeLeon P. The foundations of policy analysis. Homewood: Dorsey Press; 1983.Google Scholar
- C183—Maternity Protection Convention 2000 (No. 183). Convention concerning the revision of the Maternity Protection Convention (Revised), 1952 (Entry into force: 07 Feb 2002). Adoption: Geneva, 88th ILC session (15 Jun 2000). Geneva: International Law Commission; 2000.Google Scholar
- World Health Organisation. The World Health Report 2006 - Working Together for Health. Geneva: WHO; 2006.Google Scholar
- Instituto Nacional de Estadística e Informática. Encuesta a Establecimientos de Salud con Funciones Obstétricas y Neonatales. ENESA 2009–2012. Available at: http://www.unfpa.org.pe/InfoEstadistica/2013/EncuestaESFOyN/libro.pdf. Accessed 24 Aug 2015.
- Instituto Nacional de Estadística e Informática. Instituto Nacional de Estadística e Informática. Establecimientos de Salud con Funciones Obstétricas y Neonatales 2013. Available at:\ http://www.inei.gob.pe/media/MenuRecursivo/publicaciones_digitales/Est/Lib1161/libro.pdf. Accessed 24 Aug 2015.
- Tanzanian Reproductive, Maternal, Newborn and Child Health Scorecard. Available at: http://www.mamaye.or.tz/en/evidence/tanzanian-reproductive-maternal-newborn-and-child-health-scorecard. Accessed 7 Feb 2016.
- Government of United Republic of Tanzania. Tanzanian reproductive, maternal, newborn and child health scorecard: Quarter 4. 2014.Google Scholar
- National Bureau of Statistics of Tanzania, ICF Macro. Tanzania Demographic and Health Survey 2010. Calverton: ICF Macro; 2011.Google Scholar
- United Republic of Tanzania Ministry of Health and Social Welfare. Human Resources for Health Country Profile 2012/13. Dar es Salaam: Ministry of Health and Social Welfare; 2013.Google Scholar
- United Republic of Tanzania. 2012 Population and Housing Census. Basic demographic and socio-economic profile—key findings. Available at: http://nbs.go.tz/nbs/takwimu/census2012/Basic_Demographic_and_Socio-Economic_Profile_PopularVersion-KeyFindings_2012_PHC_EnglishVersion.pdf; 2014. Accessed 7 Dec 2015.
- United Republic of Tanzania Prime Minister's Office & Regional Administration and Local Government - PMO-RALG. Financial reports. Available at: http://www.pmoralg.go.tz/lginformation/monitor.php; 2015. Accessed 7 Dec 2015.
- The DHS Program. Tanzania: Servise Provision Assessment (SPA) 2014–2015. Available at: https://dhsprogram.com/pubs/pdf/SPA22/SPA22.pdf; 2015. Accessed 7 Dec 2015.
- World Health Organisation. Monitoring the Building Blocks of Health Systems: A handbook of Indicators and their Measurement Strategies. Geneva: WHO; 2010.Google Scholar
- World Health Organisation. Service Availability and Readiness Assessment: A methodology for measuring health systems strengthening. Available at: http://www.who.int/healthinfo/systems/sara_introduction/en/. Accessed 7 Feb 2016.
- Bryce J, Victora CG, Habicht JP, Black RE, Scherpbier RW, Advisors M-IT. Programmatic pathways to child survival: results of a multi-country evaluation of Integrated Management of Childhood Illness. Health Policy Plan. 2005;20 Suppl 1:i5–i17.View ArticlePubMedGoogle Scholar
- Huicho L, Davila M, Campos M, Drasbek C, Bryce J, Victora CG. Scaling up integrated management of childhood illness to the national level: achievements and challenges in Peru. Health Policy Plan. 2005;20(1):14–24.View ArticlePubMedGoogle Scholar
- Requejo J, Bryce J, Victora C, The Countdown to 2015 writing team. A decade of tracking progress for maternal, newborn, and child survival: The 2015 report. New York: UNICEF and WHO. 2014: 1–220.Google Scholar
- Countdown Health System and Policy Technical Working Group, Cavagnero E, Daelmans B, Gupta N, Scherpbier R, Shankar A. Assessment of the health system and policy environment as a critical complement to tracking intervention coverage for maternal, newborn, and child health. Lancet. 2008;371(9620):1284–93.View ArticleGoogle Scholar
- The Global Fund. Monitoring & Evaluation: Guideline notes and tools. In: Available at: http://www.theglobalfund.org/en/me/. 2015. Accessed 7 Dec 2015.
- Ebener S, Guerra-Arias M, Campbell J, Tatem AJ, Moran AC, Amoako Johnson F, Fogstad H, Stenberg K, Neal S, Bailey P, et al. The geography of maternal and newborn health: the state of the art. Int J Health Geogr. 2015;14:19.View ArticlePubMedPubMed CentralGoogle Scholar
- The Republic of Uganda Ministry of Health, World Health Organisation. Service Availability Mapping (SAM). Available at: http://www.who.int/healthinfo/systems/samdocs/en/. 2006. Accessed 7 Jan 2016.
- Zimba E, Kinney MV, Kachale F, Waltensperger KZ, Blencowe H, Colbourn T, George J, Mwansambo C, Joshua M, Chanza H, et al. Newborn survival in Malawi: a decade of change and future implications. Health Policy Plan. 2012;27(3):iii88–103.PubMedGoogle Scholar
- Global Burden of Disease Study C. Global, regional, and national incidence, prevalence, and years lived with disability for 301 acute and chronic diseases and injuries in 188 countries, 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet. 2015;386(9995):743–800.View ArticleGoogle Scholar
- Dickson KE, Kinney MV, Moxon S, Ashton J, Zaka N, Simen-Kapeu A, Sharma G, Kerber K, Daelmans B, Gulmezoglu A, et al. Scaling up quality care for mothers and newborns around the time of birth: an overview of methods and analyses of intervention-specific bottlenecks and solutions. BMC Pregnancy Childbirth. 2015;15(2):S1.View ArticlePubMedPubMed CentralGoogle Scholar
- Bergh AM, Arsalo I, Malan AF, Patrick M, Pattinson RC, Phillips N. Measuring implementation progress in kangaroo mother care. Acta Paediatrica. 2005;94(8):1102–8.View ArticlePubMedGoogle Scholar
- Bergh AM, Kerber K, Abwao S, de-Graft Johnson J, Aliganyira P, Davy K, Gamache N, Kante M, Ligowe R, Luhanga R, et al. Implementing facility-based kangaroo mother care services: lessons from a multi-country study in Africa. BMC Health Serv Res. 2014;14:293.View ArticlePubMedPubMed CentralGoogle Scholar
- CCM Central. The iCCM benchmarks framework. Available at: http://www.jhsph.edu/research/centers-and-institutes/institute-for-international-programs/_documents/rapid_scaleup/wp-implementation-strength.pdf. 2015. Accessed 7 Dec 2015.
- Atun R. Health systems, systems thinking and innovation. Health Policy Plan. 2012;27(4):iv4–8.PubMedGoogle Scholar
- Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review--a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10 Suppl 1:21–34.View ArticlePubMedGoogle Scholar
- Afnan-Holmes H, Magoma M, John T, Levira F, Msemo G, Armstrong CE, Martinez-Alvarez M, Kerber K, Kihinga C, Makuwani A, et al. Tanzania's Countdown to 2015: an analysis of two decades of progress and gaps for reproductive, maternal, newborn, and child health, to inform priorities for post-2015. Lancet Glob Health. 2015;3(7):e396–409.View ArticlePubMedGoogle Scholar
- Tanzanian Ministry of Health and Social Welfare. Women and Children First: Countdown to ending preventable maternal, newborn and child deaths. Published by Countdown to 2015: Available at: http://www.countdown2015mnch.org. May 2014.
- Tantivess S, Walt G. The role of state and non-state actors in the policy process: the contribution of policy networks to the scale-up of antiretroviral therapy in Thailand. Health Policy Plan. 2008;23(5):328–38.View ArticlePubMedGoogle Scholar
- Walt G, Shiffman J, Schneider H, Murray SF, Brugha R, Gilson L. 'Doing' health policy analysis: methodological and conceptual reflections and challenges. Health Policy Plan. 2008;23(5):308–17.View ArticlePubMedPubMed CentralGoogle Scholar
- Shiffman J. Network advocacy and the emergence of global attention to newborn survival. Health Policy Plan. 2015;31(Suppl 1):i60-i73.Google Scholar
- Shiffman J. Knowledge, moral claims and the exercise of power in global health. Int J Health Policy Manag. 2014;3(6):297–9.View ArticlePubMedPubMed CentralGoogle Scholar
- Sabatier PA. Theories of the policy process. 2nd ed. Boulder: Westview Press; 2007.Google Scholar
- International Labour Organisation. Global Extension of Social Security. Vol. 20 August 2015. Geneva. Available at: http://www.ilo.org/global/topics/social-security/lang--en/index.htm; 2013.
- International Labour Organization. World Social Protection Report 2014/2015: Building economic recovery, inclusive development and social justice. 2015.Google Scholar
- ALMA National Scorecards. http://alma2030.org/scorecards-and-reports/map. Accessed 5 Jan 2016.
- Government of Tanzania. Tanzania RMNCH Quarterly Scorecard. Edited by Ministry of Health and Social Welfare. Dar es Salaam; 2015.Google Scholar
- Huicho L, Segura ER, Huayanay-Espinoza CA, de Guzman JN, Restrepo-Mendez MC, Tam Y, Barros AJ, Victora CG, Peru Countdown Country Case Study Working G. Child health and nutrition in Peru within an antipoverty political agenda: a Countdown to 2015 country case study. Lancet Glob Health. 2016;4(6):e414–426.View ArticlePubMedGoogle Scholar