- Research article
- Open Access
LiST modelling with monitoring data to estimate impact on child mortality of an ORS and zinc programme with public sector providers in Bihar, India
BMC Public Health volume 18, Article number: 103 (2018)
Many interventions have attempted to increase vulnerable and remote populations’ access to ORS and zinc to reduce child mortality from diarrhoea. However, the impact of these interventions is difficult to measure. From 2010 to 15, Micronutrient Initiative (MI), worked with the public sector in Bihar, India to enable community health workers to treat and report uncomplicated child diarrhoea with ORS and zinc. We describe how we estimated programme’s impact on child mortality with Lives Saved Tool (LiST) modelling and data from MI’s management information system (MIS). This study demonstrates that using LiST modelling and MIS data are viable options for evaluating programmes to reduce child mortality.
We used MI’s programme monitoring data to estimate coverage rates and LiST modelling software to estimate programme impact on child mortality. Four scenarios estimated the effects of different rates of programme scale-up and programme coverage on estimated child mortality by measuring children’s lives saved.
The programme saved an estimated 806–975 children under-5 who had diarrhoea during five-year project phase. Increasing ORS and zinc coverage rates to 19.8% & 18.3% respectively under public sector coverage with effective treatment would have increased the programme’s impact on child mortality and could have achieved the project goal of saving 4200 children’s lives during the five-year programme.
Programme monitoring data can be used with LiST modelling software to estimate coverage rates and programme impact on child mortality. This modelling approach may cost less and yield estimates sooner than directly measuring programme impact with population-based surveys. However, users must be cautious about relying on modelled estimates of impact and ensure that the programme monitoring data used is complete and precise about the programme aspects that are modelled. Otherwise, LiST may mis-estimate impact on child mortality. Further, LiST software may require modifications to its built-in assumptions to capture programmatic inputs. LiST assumes that mortality rates and cause of death structure change only in response to changes in programme coverage. In Bihar, overall child mortality has decreased and diarrhoea seems to be less lethal than previously, but at present LiST does not adjust its estimates for these sorts of changes.
In 2013 alone more than 0.57 million children under five years of age died from diarrhoea . Nearly one-fourth of these child deaths due to diarrhoea occurred in India . As per international guidelines, in 2006, the India Academy of Paediatrics adopted the recommendations for treating diarrhoea with low osmolarity Oral Rehydration Salts (ORS) and zinc but implementation lagged behind . In fact, diarrhoea still accounts for 10.5% of deaths among children under-5 in Bihar .
From 2010 to 2015, Micronutrient Initiative (MI), now Nutrition International conducted the Childhood Diarrhoea Management Programme (CDMP), also called the Diarrhoea Alleviation through Zinc and ORS Therapy (DAZT) project, in Bihar, Gujarat, and Uttar Pradesh. Children’s Investment Fund Foundation (CIFF) supported the Bihar programme . The goal of the programme in Bihar was to reduce child morbidity and mortality related to diarrhoeal disease among children under-5 through improvements to public sector delivery of ORS and zinc for the treatment of childhood diarrhoea. MI implemented the programme in 15 demonstration districts and 23 scale-up districts of Bihar between August 2011 and August 2015. A team led by Johns Hopkins Bloomberg School of Public Health (JHSPH) and the Society for Applied Studies (SAS), New Delhi conducted a baseline survey in April – May 2011 and a follow-up survey in September – December 2013 . JHSPH also used the LiST [7, 8] modelling software, to estimate how many children’s lives had been saved in the DAZT programme and its future potential for saving lives.
Over the years LiST has evolved to model more types of interventions and outcomes . Studies have shown that the utility of LiST modelling to measure the potential effectiveness of an intervention or set of interventions related to mother and child health under scale up scenarios by estimating the number of lives saved [10,11,12]. It has been used to guide the strategic planning process in resource-poor settings with limited data inputs . More recently, LiST has been used to identify high impact health interventions to end preventable deaths in mothers, newborns and stillbirths and to estimate their associated costs . Furthermore, Walker and Walker  detail how LiST estimates deaths averted or lives saved by public sector interventions to reduce diarrhoeal deaths. Their findings demonstrate that LiST can be used as an alternative to large and expensive mortality impact studies.
Along with using LiST software, MI supported the state government to manage a robust management information system (MIS) to estimate the cases of diarrhoea and forecast the supply of ORS and zinc needed at different levels of service provision. Our study incorporates MIS data in LiST modelling to estimate the number of lives saved due to the ORS and zinc programme coverage. There are currently no studies of MIS data being utilized in LiST modelling, therefore our innovative approach has implications for scaling up cost-effective and impactful programmes.
This paper describes how we used the Lives Saved Tool (LiST) modelling software and MIS data to estimate the Bihar programme’s effect on child mortality. Through this study, we further highlight the possibilities and challenges of using LiST and MIS data to measure programmes.
We used LiST version 5.31 to estimate the number of lives saved due to the specific levels of ORS and zinc coverage that the programme achieved, based on MI’s monitoring data. LiST requires three sets of inputs to project the impact of interventions on mortality: (1) measures of population-level health status including mortality and causes of death (available by default in LiST); (2) effect sizes [16, 17] of interventions and affected fractions of the population, and (3) intervention coverage (Table 1).
Rather than use primary data for both numerator (cases covered) and denominator (population affected) as the 2013 JHSPH LiST modelling did, our 2015 LiST modelling exercise used available data from MI’s monitoring data to estimate cases covered at the district level, and used other data sources to estimate the population affected.
Population at risk
The target population of the intervention and the modelling is children aged 2–59 months from 15 demonstration districts.
Incidence of diarrhoea
The lower bound of diarrhoea incidence was estimated as 1.81 (diarrhoea incidence of 1.81 episodes/child/year) and a higher bound of 2.20 .
Public sector programme coverage
In our case the programme intervention for children with diarrhoea was those who received both ORS and zinc from a public-sector service provider. In the previous modelling exercise, JHSPH conducted two cross-sectional studies – baseline (2011) and a follow-up (2013) survey – and used coverage information from these two surveys to estimate the number of lives saved (or deaths averted) due to programme impact.
To estimate public sector coverage, we used three data sources i) MI’s MIS data, ii) JHSPH’s 2011 baseline and 2013 follow-up coverage data for ORS and zinc treatment from public health sector functionaries, and iii) the 2011 India Census, for estimates of population affected by childhood diarrhoea. We estimated the coverage rate numerator from MI’s MIS data and from JHSPH’s 2013 follow-up data, and the denominator from the 2011 India Census.
MI’s intervention was completed solely through public health functionaries in the intervention districts: Accredited Social Health Activists (ASHAs), Anganwadi workers (AWWs), Auxiliary Nurse Midwives (ANMs), Primary Health Centres and government hospitals. Front line health workers (ASHAs and AWWs) at the village level reported each diarrhoeal case and those treated with ORS and zinc within their respective geographies. Auxiliary Nurse Midwives reported cases and treatment at the sub-centre level. Monthly consolidated data was shared at a district and state level.
In 2011 and 2013, JHSPH and SAS surveyed caregivers of children who had had a diarrhoeal episode in the 14 days before the survey. The caregivers who reported having given their children ORS and/or zinc to treat the diarrhoea were asked about the source of the ORS and/or zinc. In addition to coverage estimates obtained from surveys, we also used coverage estimates inputs obtained from MIS. MIS coverage estimates are bound to overestimate since they are not based on cross-sectional data. To nullify such error, we used a multiplication factor (0.68) that was obtained by dividing coverage rates obtained from the JHSPH’s follow-up survey (2013) with the coverage estimate obtained from MIS in the same year. This adjustment was carried out for the coverage estimates calculated from MIS in subsequent years and inputted into the LiST modelling as listed in Table 2.
Similar adjustments were made while calculating zinc coverages.
The denominator of the coverage rate was estimated based on the number of children (2–59 months) who are affected with diarrhoea in the intervention areas. This was estimated based on the total number of children in that age group multiplied by the estimated incidence of diarrhoea in the age group. The total number of children aged 2–59 months was projected from a base population in 2011 using Census population  and adjusted with the annual exponential growth rate for these 15 demonstration districts. The current modelling used LiST’s default values for intervention efficacies and population affected fractions.
We developed four scenarios to establish appropriate mixes of coverage information. The following Table 3 and Fig. 1 provide the details of these scenarios.
During the programme design phase, in 2010, CIFF estimated that the programme could save at least 4200 children’s lives. Scenario 3 of the 2015 modelling is a hypothetical as we estimated what programme coverage would have been necessary to reach this level of impact. This was worked out “backwards” by attempting several iterations in LiST to achieve the target of 4200 deaths averted cumulatively.
Scenarios 1 and 2 – Actual programme coverage
Using the lower estimate of annual diarrhoeal incidence, LiST estimated that a total of between 965 (Scenario 1) and 975 (Scenario 2) additional lives were saved in children under 5, as a result of scaling up ORS and zinc treatment for diarrhoea through public sector providers (Fig. 2). The estimated additional number of lives saved in children under-5 age relative to impact year in 2010–11 are not significantly different.
Using the higher estimate of annual diarrhoeal incidence, LiST estimated that a total of between 806 and 821 additional lives were saved among children under-5, as a result of scaling up ORS and zinc treatment for diarrhoea through public sector providers (Fig. 3). As with the lower estimate of annual diarrhoeal incidence, the LiST results for treatment scenarios 1 and 2 are not very different.
Scenario 3 -- hypothetical model – Backwards coverage estimation to achieve design-stage impact target
To achieve a cumulative additional 4200 lives saved from expanding treatment of ORS and zinc in the public sector, our results LIST found that public sector coverage of ORS and zinc treatment would have had to expand from less than 2% in 2010–11 to nearly 20% in 2014–15 (Table 4).
Scenario 4 – Hypothetical models -- 40% and 60% increased coverage (Fig. 4)
We constructed the final two scenarios to understand the worst possible diarrhoeal incidences (at 2.20 episodes/child/year) combined with the best possible programme coverage (i.e., 40% and 60% more public sector coverage than the coverage rates calculated from 2012 to 13 of MIS data in the year 2014–15, Table 5). This provides an opportunity to measure the impact of the programme intervention if the ORS and zinc coverage rates are peaked at these levels.
Public sector coverage of ORS and zinc would have had to have increased dramatically during the programme to avert more than 2000 deaths.
JHSPH’s 2013 projections based on LiST modelling found that at increased coverage levels– from 19.7% (from both the public and private sectors) in 2010 to 25.9% for ORS and from 3.7% in 2010 to 14.3% in 2015 for zinc – by 2015, more than 3400 diarrhoea related deaths among under-5 children in 15 demonstrated districts of Bihar would have been averted.
The 2015 LiST modelling results reported above, based on Scenarios 1–4, focus on public sector service coverage and are consistent with the 2013 JHU LiST modelling results. To have greater impact on child mortality, public sector coverage levels would have needed to increase quite ambitiously and remain increased as compared to the 2010–11 baseline levels. Had the programme been able to increase public sector coverage by 40%, more than 2100 children’s lives would have been saved, and had public sector coverage increased by 60%, more than 2300 lives would have been saved. To save 4200 children’s lives, the intervention would have had to increase its public sector ORS coverage from 1.8% in 2010–11 to 19.8% in 2014–15, and also its public sector zinc coverage from 1.4% in 2010–11 to 18.3% by 2014–15. These modelled coverage rates for ORS and zinc are ambitious given the field realities such as persistent stock outs of zinc and ORS in Bihar’s public sector made this sustained increase unfeasible .
Throughout our study, the LiST module was suitable for use at a sub-state level such as district in Bihar. However, the current LiST software present several limitations to our study as well.
Firstly, LiST does not provide any guidelines or modules on using MIS data to compute coverage rates for intervention programmes or extracting coverage rates separately from MIS data. Secondly, LiST does not include a joint coverage of both ORS and zinc though it is the recommended treatment for diarrhoea. Since LiST allows inputs for coverage rates of “ORS” (only) or “Zinc” (only) we had to accommodate the ORS + Zinc data into the LiST analysis by clubbing information pertaining to ORS + Zinc with ORS (only) and made two categories – ORS and zinc.
Thirdly, for the zinc (only) category, the field reports suggest that when health workers had no stock of ORS, they generally advised caregivers to give children oral rehydration therapies (ORTs) along with zinc, or get ORS from other sources. The potential impact of ORTs in combination with zinc is not captured in the model results, which may imply some level of additional effectiveness in treating diarrhoea cases.
Lastly, the overarching assumption in LiST is that any changes in the mortality rates and the cause of death structure is solely in response to changes in intervention coverages . This assumption restricts the use of LiST model to short-term projection purposes not suitable for long-term projection as child mortality rates and cause of death distribution are rapidly changing.
This article reports on the authors’ experience with using LiST software to model a programme’s impact on deaths from diarrhoea among children under-5 in Bihar. Our modelling assessed the impact of MI’s public sector ORS and zinc programme in 15 demonstration districts in Bihar using a population survey and programme MIS reporting data. We demonstrated that even with the limited data available in resource-poor settings, LiST software and utilization of MIS data can be a useful tool for programme evaluators and planners to estimate a programme’s impact on child mortality. For example, CIFF used LiST modelling to estimate the programme’s potential for impact on child mortality. The modelled impact results reported here (806 to 975) fell short of the 2010 CIFF projections which may be attributable to the challenges of the LiST software, programme execution, and data quality.
Our study also highlights the need to improve the LiST software. For example, LiST developers can provide appropriate guidelines on how to use MIS data to compute coverage rates for different intervention programmes and extract coverage rates from MIS data. The developers can also incorporate data collection for joint interventions such as treatment with both ORS and zinc. Finally, the software could expand the use of LiST model to long-term projection purposes as child mortality rates and cause of death distribution are rapidly changing.
Despite the improvements that need to be made to LiST, the software and utilization of MIS data should be seen as a complement rather than a replacement for household surveys. LiST and MIS data are viable options for estimating programme impact without the time and expense required to measure programme impact with a household coverage survey.
Auxiliary Nurse Midwife
Accredited Social Health Activist
Children’s Investment Fund Foundation
- LiST :
Lives Saved Tool
Walker CFL, Rudan I, Liu L, Nair H, Theodoratou E, Bhutta ZA, O'Brien KL, Campbell H, Black RE. Global burden of childhood pneumonia and diarrhoea. Lancet. 2013;381:1405–16.
Liu L, Oza S, Hogan D, Perin J, Rudan I, Lawn JE, Cousen S, Colin M, Black RE. Global, regional, and national causes of child mortality in 2000-13, with projections to inform post-2015 priorities: an updated systematic analysis. Lancet. 2015;385:430–40.
Bhatnagar S, Lodha R, Choudhury P, Sachdev HP, Shah N, Narayan S, et al. IAP guidelines 2006 on management of acute diarrhea. Indian Pediatr. 2007;44:380–9. 17536143
Office of the Registrar General & Commissioner, India. Sample Registration System. Report on Medical Certification of Cause of Death Statistics 2010–13. Ministry of Home Affairs, Government of India. 2013.
Children's Investment Fund Foundation. Oral rehydration salts & zinc treaments to reduce childhood diarrhoea through public-sector provision of oral rehydration salts (ORS) and zinc in Bihar. https://ciff.org/grant-portfolio/oral-rehydration-salts-zinc-treatments/. Accessed 24 Dec 2017.
Walker CFL, Taneja S, Lamberti LM, Black RE, Mazumder S. Public sector scale–up of zinc and ORS improves coverage in selected districts in Bihar, India. Journal of Global Health. 2015;5(2) https://doi.org/10.7189/jogh.05.020408.
LiST: The Lives Saved Tool. An evidence-based tool for estimating intervention impact [http://www.jhsph.edu/research/centers-and-institutes/institute-for-international-programs/current-projects/lives-saved-tool/]. Accessed 20 July 2015.
Development and use of the Lives Saved Tool (LiST): A Model to estimate the impact of scaling up proven interventions on maternal, neonatal and child mortality. In Special issue of International Journal of Epidemiology. 2010;39 Suppl 1.
Walker N, Tam Y, Friberg IK. Overview of the lives saved tool (LiST). BMC Public Health. 2013;13(Suppl 3):S1. https://doi.org/10.1186/1471-2458-13-S3-S1.
Jones G, Stekettee RW, Black RE, Bhutta ZA, Morris SS. Bellagio child survival study group: how many chid deaths can we prevent this year? Lancet. 2003;362:65–71.
Stover J, McKinnon R, Winfrey B. Spectrum: a model platform for linking maternal and child survival interventions with AIDS, family planning and demographic projects. Int J Epidemiol. 2010;39(S1):i7–i10.
Ricca J, Kureshy N, LeBan K, Prosnitz D, Ryan L. Community-based intervention packages facilitated by NGOs demonstrate plausible evidence for child mortality impact. Health Policy Plan. 2013:1–13.
Bryce J, Frieberg IK, Kraushaar D, Nsona H, Afenyadu GY, Nare N, Kyei-Faried S, Walker N. LiST as a catalyst in program planning: experiences from Burkina Faso, Ghana and Malawi. Int J Epidemiol. 2010;39(Suppl 1):i40–7.
Bhutta ZA, Das JK, Bahl R, Lawn JE, Salam RA, Paul VK, Sankar MJ, Blencowe H, Rizvi A, Chou VB, Walker N. Lancet newborn interventions review group and the lancet every newborn study group: can available interventions end preventable deaths in mother, newborn babies, and stillbirths, and at what cost? Lancet. 2014;384:347–70.
Walker CLF, Walker N. The lives saved tool (LiST) as a model for diarrhea mortality reduction. BMC Med. 2014;12:70.
Munos M, Walker CFL, Black RE. The effect of oral rehydration solution and recommended home fluids on diarrhea mortality. Int J Epidemiol. 2010;39(Suppl 1):i75–87. http://www.ncbi.nlm.nih.gov/pubmed/20348131.
Walker CLF, Black RE. Zinc for the treatment of diarrhea: effect on diarrhea morbidity, mortality and incidence of future episodes. Int J Epidemiol. 2010;39(Suppl 1):i63–9. http://www.ncbi.nlm.nih.gov/pubmed/20348128.
Office of the Registrar General & Commissioner, India. Annual Health Survey, 2010–11. Ministry of Home Affairs, Government of India. http://www.censusindia.gov.in/2011-Common/AHSurvey.html. Accessed Dec 2015.
In the Key Performance Indicators (KPIs) MI used diarrhoea incidence as 1.81 episodes/child/year (lower side) and 2.2 (on higher side). There was no reliable reference available for CDMP districts. Therefore, in the beginning MI had used incidence 1.71 based on GOI report Burden of Childhood diarrhoea (Estimation of the burden of diarrhoeal diseases in India, NCMH, GOI 2005, page 184). Incidence of diarrhoea varies as per the prevalence and in different surveys there is great variation in the 2 weeks child diarrhoea prevalence. Based on WHO methodology MI computed incidence for 15 CDMP districts. The lowest was based on AHS 1.8 and from two sources DLHS and average prevalence of different surveys in CDMP districts it came to 2.2.
Office of the Registrar General & Census Commissioner, India. Ministry of Home Affairs, Government of India. http://censusindia.gov.in/. Accessed Aug 2015.
Kumar S, Roy R, Dutta S. Scaling–up public sector childhood diarrhea management program: lessons from Indian states of Gujarat, Uttar Pradesh and Bihar. Journal of Global Health. 2015;5(2) https://doi.org/10.7189/jogh.05.020414.
Bhutta ZA, Das JK, Rizvi A, Gaffey MF, Walker N, Horton S, Webb P, Lartey A, Black RE, The Lancet Nutrition Interventions Review Group, and the Maternal and Child Nutrition Study Group. Evidence-based interventions for improvement of maternal and child nutrition: what can be done and at what cost? Lancet. 2013;382:452–77.
The authors wish to thank Dr. Saul Morris, Director, Nutrition at CIFF, UK for his timely advice to complete the modelling work and Ms. Yunyu Liang, Intern with Evidence, Measurement and Evaluation, CIFF, UK for her comments on the initial draft. We would also like to thank Ms. Biva Rajbhandari, Analyst at CIFF, New Delhi, for her significant contributions to the final draft of the manuscript. We also wish to thank MI’s programme staff at New Delhi and two peer reviewers for their valuable comments on draft of the manuscript.
Research for this article was funded by The Children’s Investment Fund Foundation (UK) (“CIFF”). The views, opinions, assumptions or any other information set out in this article are solely those of the author[s] and should not be attributed to CIFF.
Availability of data and materials
The datasets analysed during the current study are not publicly available due to programme MIS managed by MI (now Nutrition International), New Delhi but available from the authors on reasonable request with the permissions of MI and CIFF. Spectrum (.pjnz) files are available with the first author.
JA is now with Avenir Health (India based) and previously worked as an independent consultant hired by CIFF, UK to work on LiST modelling during July, 2015 through December, 2015. CH works for CIFF, UK. SK was previously associated with Micronutrient Initiative, New Delhi till March, 2016 and now with Digital Green, India and MS is with CIFF, India.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Ayyanat, J.A., Harbour, C., Kumar, S. et al. LiST modelling with monitoring data to estimate impact on child mortality of an ORS and zinc programme with public sector providers in Bihar, India. BMC Public Health 18, 103 (2018). https://doi.org/10.1186/s12889-017-5008-y
- LiST modelling
- ORS and zinc
- Under-5 mortality