Correction to: Optima nutrition: an allocative efficiency tool to reduce childhood stunting by better targeting of nutrition-related interventions

It has been highlighted that the original manuscript [1] contains a typesetting error in the name of Meera Shekar. This had been incorrectly captured as Meera Shekhar in the original article which has since been updated.


Background
Stunting, or reduced linear growth, affects around 200 million children younger than 5 years of agemostly in Asia and Africa [1,2]. Stunting is a predictor of poorer cognitive and physical development, which has been shown to reduce productive capacity and economic output [3][4][5][6][7][8]. Sustainable Development Goal 2 calls for an end to malnutrition by 2030 and significant reductions in stunting by 2025. Research suggests that this goal can be partly met through a set of evidence-based interventions [9] in the period from conception to 5 years of age [10].
There are numerous proven ways in which undernutrition and stunting can be reduced. Correct feeding practices, which include exclusive breastfeeding for the first 6 months and continued breastfeeding, combined with appropriate complementary foods until the age of 24 months, not only provide a key source of nutrients but also protection against gastrointestinal and other infections [11]. Sub-optimal breastfeeding is associated with poorer child growth outcomes because of replacement with less nutritious foods and greater risk of infection [12,13]. For children aged 6 months to 5 years, providing high quality and nutritionally diverse complementary foods, and supplementing diets with micro-nutrients including vitamin A and zinc, if needed, can also reduce the risk of stunting [14][15][16]. In addition to nutritional interventions, the risk of stunting can be reduced by addressing household, environmental, socioeconomic and cultural factors [17,18]. The quality and use of health, social protection, and sanitation service delivery systems are further elements affecting nutrition outcomes.
Since there are only scarce resources from domestic sources or from international donors for nutritionrelated interventions [19], it is imperative that available budgets are used in a manner which can achieve the greatest possible outcome. There is evidence from diverse settings that better health resource allocation decisions result in substantial improvements in population health [20][21][22]. Quantitative tools can be useful to integrate available knowledge and data into a logical framework and provide analytical evidence for improving resource allocation decisions. This study introduces a new tool, Optima Nutrition, specially designed and developed to inform allocative efficiency decisions related to stunting.
Existing modeling tools provide an effective base for enabling policy makers to estimate the epidemiological impact of nutritional interventions on population outcomes. These include the Lives Saved Tool (LiST) [23,24], PROFILES [25], World Breastfeeding Costing Initiative (WBCi) [26], CMAM Costing Tool [27] and MINIMOD [28]. LiST calculates the expected impact and cost of scaling up numerous maternal and child health interventions including nutritional interventions; PROFILES demonstrates the effect of nutritional interventions on health and economic development; and WBCi estimates the cost and impact of scaling up promotion and support services for breastfeeding and correct infant and young child feeding (IYCF) practices. The CMAM Costing Tool estimates the cost of establishing, maintaining or expanding services for community management of acute malnutrition (CMAM), and is offered as a part of FANTA (Food and Nutrition Technical Assistance) Project [29] a USAID-funded project for strengthening food security and nutrition policies. MINIMOD finds the mix of delivery systems that maximizes effective coverage of a micronutrient supplementation. Some of these tools consider the benefits of single interventions, whereas others incorporate the effects of multiple interventions simultaneously. Currently, no tool is capable of estimating the allocation of a given budget across a range of nutritional interventions that gets as close as possible to attaining certain objectives. The Optima Nutrition tool aims to fill this gap by building on the strengths of the existing nutrition modeling field and expertise in allocative efficiency tools from other fields.
A consortium of academic institutions and development partners has produced a suite of modeling tools with central focus on improving allocative efficiency. Optima tools have been applied in partnership with the governments of over 40 low-and middle-income countries to assess how more targeted resource allocation decisions can lead to improved outcomes for major infectious diseases including HIV, TB and malaria (e.g. [30][31][32][33][34][35])). These studies have influenced health resource allocation, shifting funding to the most costeffective mix of programs and assisting with national strategic plan development and operational planning in many countries.

Methods
The Optima approach combines: (i) a core epidemiological model that relates intervention outcomes to epidemiological outcomes; (ii) cost functions that relate coverage and expenditure on interventions to intervention outcomes; (iii) an objective function defined by national strategic targets and constraints defined by logistic, ethical, political and financial considerations; (iv) a formal mathematical optimization algorithm around other components to estimate the most allocatively efficient use of resources. To develop the Optima Nutrition tool, we followed this approach, using a replicated version of the LiST model to serve as the core epidemiological model. Both LiST and the Optima approach have been described in detail elsewhere [24,30], but in this section we provide a short summary of them both, and describe how they were integrated to develop Optima Nutrition. In addition, we provide details on the data that were collated and used in the demonstration example of Optima Nutrition's application to Bangladesh.

Epidemiological model structure
The core epidemiological model used within Optima Nutrition is a dynamic, deterministic, compartmental model which tracks the number of children in a population from birth until 5 years of age across five age bands: < 1 month, 1-5 months, 6-11 months, 12-23 months and 24-59 months (see Fig. 1). Children enter the model cohort via the < 1 month age-band at birth and exit the cohort either when they reach the age of 60 months or by death, which can happen at any age. Children in each age-band are categorized by height-for-age and breastfeeding practice. The number of children in each category is simulated as they age through, and out of, the modeled population, estimating overall stunting and mortality. Risk factors for stunting and death are: birth outcomes including pre-term birth or a child being born small for gestational age, stunting in the previous age-band, suboptimal breastfeeding practices, and incidence of diarrhea ( Fig. 1). In the model, children in the < 1 month age-band can die due to diarrhea, pneumonia, meningitis, asphyxia, sepsis, prematurity and "other" causes, while children in all other age bands can die of diarrhea, pneumonia, measles and "other" causes. The relative risks of dying from each cause are related to the child's breastfeeding and height-for-age status. Risks of stunting and mortality associated with these risk factors are taken from established literature [36][37][38]; "other" causes are used to capture, and match to, population statistics of known overall mortality rates for the given application context. The model equations, along with mathematical expressions that link parameter values from the literature to these model rates, are the same as used in LiST and provided in [23].
The model divides children in each age-band into four height-for-age categories: more than 3 standard deviations below the global norm ('severely stunted'), 2-3 standard deviations below the norm ('moderately stunted'), 1-2 standard deviations below the norm ('mildly stunted' , and less than one standard deviation below the norm ('normal'). Children in all age bands who are more than two standard deviations below the global norm are considered stunted. Breastfeeding practice is categorized as none, partial (liquids and solids in addition to breastmilk), predominant (breastmilk supplemented by other liquids), and exclusive.

Interventions included in optima nutrition
Six nutritional interventions that target child mortality and stunting are included in this application of the model: antenatal micronutrient supplementation (AMS), balanced energy-protein supplementation during pregnancy, promotion of exclusive breastfeeding, promotion of improved infant and young child feeding (IYCF) practices, public provision of complementary foods and vitamin A supplementation.
The six interventions included in this release of the Optima Nutrition tool either directly impact mortality and the risk of stunting in different age bands, or have Fig. 1 Diagram of the Optima Nutrition model. The number of children under 5 years is tracked across five age bands. Children enter the model by birth: term or pre-term, small for gestational age (SGA) or average for gestational age (AGA). Children leave the model either by reaching 5 years of age, or by death. Risk factors which affect stunting and mortality, and interventions included in the model to reduce mortality and stunting, are shown influence by impacting associated risk factors including incidence of diarrhea, breastfeeding practice or birth outcomes. The impact of these interventions on different age bands in the modeled cohort is illustrated in Fig. 1: Antenatal micronutrient supplementationgiven to pregnant women, improves birth outcomes [39] Balanced protein-energy supplementationgiven to pregnant women below poverty line, improves birth outcomes [40]. Breastfeeding promotionbehavioral intervention that targets pregnant women and (mothers of ) children < 6 months, increases prevalence of exclusive breastfeeding of children under 6 months of age, which reduces incidence of diarrhea and all-cause mortality in this age group [37,41]. Infant and young child feeding programbehavioral intervention targeting (mothers of ) children 6-23 months of age, which reduces stunting and all-cause mortality by promoting partial breastfeeding and correct complementary feeding practices. Complementary feeding education can only benefit the population with sufficient income to afford recommended complementary foods [42]. Public provision of complementary foodscomplementary food provided to children 6-23 months below poverty line in addition to complementary feeding education, in order to reduce their risk of stunting. We assume that education around complementary feeding does not impact the population below the poverty line unless supplemented with public provision of complementary foods [43]. Vitamin A supplementationgiven to children 6 months and older to reduce the incidence of and mortality from diarrhea [44,45].
As a form of validation of the epidemiological component of our model, we compared the effects of scaling up the included interventions on stunting and mortality to what was estimated by LiST. For the number of child deaths per year, Optima and LiST are in good agreement, with the same percentage of averted deaths. There are small differences between the predictions of stunting averted in the two models (Optima has a slightly higher estimate), which can be explained by the differences in updating timesteps (yearly in LiST, monthly in Optima).

Cost functions and resource optimization framework
The Optima approach links the core epidemiological model to a resource optimization algorithm through cost functions related to each intervention. Cost functions are a key driver of resource optimization and encapsulate the set of relationships between (a) the cost of service delivery, (b) the resulting coverage levels of these services among targeted populations, and (c) how these coverage levels of services influence behavioral, clinical and epidemiological outcomes. Such relationships dictate how incremental changes in spending directly or indirectly affect outcomes of interest or associated risk factors. The cost functions can take any functional form. Most typically, a 3-parameter logistic (sigmoidal) function is fitted to available historical expenditure data, or a 2-parameter logistic function is parameterized using unit costs and information about logistical or feasibility constraints for program coverage levels. A schematic illustration of a cost function is shown in Fig. 2. The cost functions combined with the epidemiological model can project the expected level and trend in number of stunted children and number of deaths in the context of different intervention programs operating together at various funding and coverage levels.
The next step in resource optimization is to clearly define the objective using an objective function. Different Fig. 2 Generic form of a cost function associated with an intervention program used in an Optima model objective functions will lead to different resource allocations and consequently different outcomes. For example, the objective to solely minimize the number of stunted children at age 5 years and the objective to solely minimize the number of child deaths would lead to different funding allocations and have conflicting outcomes. Minimizing cases of stunting would require preferential funding of the nutrient-intake interventions that have little direct impact on general survival (such as complementary feeding), which may result in increased mortality, particularly in stunted children. On the other hand, minimizing child deaths would preferentially fund interventions with direct impact on mortality (such as breastfeeding promotion), and may increase stunting prevalence because of improved survival of stunted children. We explored numerous objective functions (any of which could easily be used in the model) but have established a default objective function for Optima Nutrition of maximizing the number of children who are alive and not stunted by age 5 years (the 'alive and not stunted' objective). Although any time horizon can be used in analyses, the default time horizon for Optima Nutrition is from the next year of programming (e.g. 2018) until 2030, the end of the sustainable development goals (SDGs).
Many optimization algorithms could be used around the calibrated epidemiological model informed by setting-specific data, and the cost functions, to calculate the resource allocation that will be the best solution against the objective function. Optima Nutrition uses an Adaptive Stochastic Descent (ASD) algorithm [46]. ASD uses simple principles to form probabilistic assumptions about (a) which parameters have the greatest effect on the objective function, and (b) optimal step sizes for each parameter.
A demonstration of optima nutrition: Bangladesh as an illustrative case study We demonstrate the use of Optima Nutrition through an example application. For illustrative purposes, the model is populated with available data from Bangladesh. Bangladesh has a population of about 160 million, increasing at 2 million (1.25%) annually. While the underfive mortality rate has dropped considerably over the last 25 years, it is currently estimated at 46 deaths per 1000 live births, of which half are neonatal. Of the approximately 15 million children aged less than 5 years in Bangladesh, over 5 million (36%) are stunted [47]. This high stunting prevalence is contributing to relatively very high child mortality as well as reduced cognitive development and related consequences [48].
The epidemiological model within Optima Nutrition was populated with available data from seven administrative divisions of Bangladesh (Barisal, Chittagong, Dhaka, Khulna, Rajshahi, Rangpur and Sylhet). Demographic data (population size in each of the five agegroups) was taken from the Bangladesh Bureau of Statistics [49], and the 2016 number and future projections of live births were taken from the UN Population Division database. Child mortality and prevalence of stunting, breastfeeding practices in each age group and poverty data were taken from the Bangladesh Demographic and Health Surveys [50,51] Unit cost data to inform cost functions was taken from a report led by the World Bank and UNICEF for the Government of Bangladesh [52]. These unit costs were calculated using program experience approach, which captures all aspects of service delivery-including the costs of commodities, transportation and storage, personnel, training, supervision, monitoring and evaluation, relevant overhead, wastage, etc. for each intervention. We assume that the relationship between investment and the number of people reached by each intervention is approximately linear at low and moderate coverage levels, but that marginal costs steadily increase at higher coverage levels (see Fig. 2) as the remainder of the target population may be hard to reach [53]. We have not accounted for any start-up costs in this application because of insufficient data and because recurrent annual costs are most important in the optimization of the annual budget. For each of the interventions considered, the estimated national 2014 coverage levels and unit costs are provided in Table 1. Regionspecific Unit costs were the same as national for all interventions except AMS, which was the most expensive in Rangpur ($1.82) and the least expensive in Dhaka ($1.78). The unit costs are assumed to be the marginal costs at low program coverage, which is applicable for current scale of programs in Bangladesh.
For each of the seven administrative divisions, we estimated the optimal distribution of resources across intervention programs under a range of different total available budgets. This provided a relationship between total available budget and number of stunted children and/or child deaths for each geographical region, representing the best available outcome attainable for each budget. This 'budget-outcome' relationship was used with interpolations to generate a curve for a predicted projection of the best available outcome attainable for any budget. By comparing the budget-outcome curves between geographical regions, the optimization algorithm was extended to optimize national outcomes by shifting funding between divisions.

Results
Here we present the results of the case study for Bangladesh that was carried out to demonstrate the use of Optima Nutrition. Currently, no published estimates of expenditure on the nutrition-specific interventions in  [54]. However, those figures do not disaggregate expenditure by specific interventions. Therefore, the current expenditure on the six interventions included in the model was approximated by multiplying the unit cost of each intervention by the current number of beneficiaries covered by that intervention. It needs to be noted that this is a rough estimate. Based on this approach, current annual expenditure on the six interventions included in the analyses was estimated to total~US$22 million (based on 2014 data). This equates to just $0.75 per person in need. Funding was allocated to breastfeeding promotion (64%), promotion of IYCF for children aged 6-23 months (19%) and vitamin A supplementation (17%) (Fig. 3). Our analyses at the national level suggest that the alive and not stunted objective could be maximized by shifting allocations to spend a majority (69%) on promotion of IYCF for children aged 6-23 months and the remainder (31%) on vitamin A supplementation. With the same level of resources this programmatic shift would be expected to increase the number of children in Bangladesh living without stunting after age 5 years by 1.37 million cumulatively by 2030, an extra~5% compared to continuation of the status quo. Excluding external determinants of reduced stunting, this reallocation of the little available resourcing could reduce national stunting prevalence from 36% to 32% in 2030. The Optima model also allows the user to identify the optimal mix of interventions as the available resource envelope expands. This function of the model is particularly useful because it permits to determine at which point of the envelope expansion new interventions could be added or funded to maximize the program impact on stunting-free survival. For illustration purposes, we calculated the optimal allocation of resources across interventions assuming that the current financing for the six nutritional interventions will expand from the current level to 150%, 200%, 300%, and 400% of the current estimated budget (Fig. 4a). If more resources become available, then IYCF should continue to receive funding until it eventually reaches relatively high coverage and where the marginal cost becomes relatively high (occurring at~150% of the current budget). Once these services are financed, public provision of complementary foods should be the next intervention to be included, followed by antenatal micronutrient supplementation (at around 200% extra total budget). If the total national budget for nutrition were quadrupled and optimally allocated, over 2.2 million more children would reach age 5 years without stunting, and stunting prevalence would drop to below 30% by 2030 (Fig. 4b).
If one aims to optimize resources against different objectives then there will be trade-offs in terms of the outcomes related to the objectives. For example, an objective which minimizes stunting may result in greater numbers of deaths than an objective which is solely focused on minimizing deaths. Our analyses reveal these trade-offs ( (Fig. 4c). The effect of increased deaths by maximizing children remaining alive and not stunted is small (77,000) compared to the increased number of averted cases of stunting (1.4 million) if one was to minimize mortality.
If greater budgets were available to provide greater coverage of interventions, then further improvements in the number of children remaining alive and not stunted would be possible. However, the incremental gain per additional dollar spent decreases (Fig. 4a). Sustainable Development Goal 2 calls for a 40% reduction of the number of stunted children by 2025, which if adopted at the country-level for Bangladesh would roughly mean reducing prevalence to below 21.5%. Our analysis suggests that this goal cannot be reached even with a 4-fold increase in spending on these six interventions, even when these resources are spent most cost-effectively. Therefore, to reach this goal at the national level, expansion of other types of interventions such as water, sanitation and hygiene (WASH) [55] and maternal education [56] as well as the crucial underlying economic and development factors would be necessary.
Optima Nutrition can be used to not only identify the broad program areas for prioritization at a national level given a specific resource envelope, but to also identify priority geographical or administrative regions for targeting. In Bangladesh, Dhaka has the highest number of stunted children (around 2.5 million), whereas the southern regions of Barisal and Khulna have the least (Fig. 5a). However, the highest prevalence of stunting is in Sylhet where around 70% of children in the oldest age group are stunted compared with around 35% in Khulna (Fig. 5b). The current coverage levels of each intervention vary by region (Fig. 5c). Sylhet receives the lowest combined coverage of the interventions and has the highest fraction of the population under poverty line, both of which may account for its higher burden of stunting prevalence. In practice, given the already low financing of nutritional interventions overall we would not recommend reducing funding from any region. However, for illustration purposes we found that if the current available funding in Bangladesh were to be optimally reallocated geographically and by intervention area then it would be possible to have additional impact on the number of children who are living and not stunted by age 5 years. Specifically, compared to the status quo allocation, by 2030 improved allocative efficiency could increase the number of nonstunted children alive at age 5 years by: (i) 1.32 million (5.1%) by broad national program funding allocations; (ii) by 1.36 million (5.3%) if geographical targeting was also included (Fig. 5c). Improved outcomes could be achieved by slightly more prioritization to Dhaka and Chittagong at the expense of Rangpur (Fig. 5c). Similar analyses could be conducted for different levels of resources to guide resource allocations to target the right programs in the right places for greatest impact.

Discussion
Over the past decade, a convincing investment case for nutrition has been made. The existing literature has explored the cost and impact of expanding high-impact evidence-based interventions [9,52,53] and produces estimates of economic impact showing that investing in nutrition in general and in stunting reduction in particular produces economic returns that outweigh the cost of interventions several-fold [57,58].
While this investment case has been critical in bringing nutrition back to the forefront of the international development agenda, it has been, at the same time, rather general. While it is clear why expanding the coverage of nutrition interventions is needed, little information is available about how such expansion should be carried out in a way that would maximize the impact of every dollar invested.
Decision tools based on mathematical modeling can be useful in informing allocation decisions. Compared to other health areas, such as HIV/AIDS, however, surprisingly few decision-analytic tools for nutrition exist, and none them allows for identifying an optimal allocation of resources across a wide range of nutritional interventions. One group of existing models, including Optifood and the Cost of the Diet (CoD) tool, uses the linear programming approach to develop dietary recommendations under a set of constraints (e.g. affordability, Fig. 5 Map of the divisions of Bangladesh, color-coded by a total stunting cases in children younger than 5 years; b prevalence of stunting among children younger than 5 years; c additional children who remain not stunted and alive at age 5 years through an optimal allocation of resources along with the estimated optimal redistribution of resources (2014 spending (left bar) compared to the optimized for the division (right bar)). Image produced by the authors availability of specific foods in a given country context). Optifood has been recently used to develop recommendations for complementary feeding in Indonesia [59], Myanmar [60], and Kenya [61]. Linear programming tools can also be combined with expenditure and consumption data to estimate the proportion of the population that can and cannot afford optimal diets. The CoD has been used in that way to advocate of the expansion of public provision of complementary foods in Indonesia (see [62]). While important, dietary recommendations are not easily translated into specific interventions and the linear programming approach provides no information regarding cost-effectiveness or efficiency of interventions.
The Lives Saved Tool (LiST) has been used in estimating comparative cost-effectiveness of nutrition interventions (e.g. [63,64]). However, the tool itself is not designed to assess cost-effectiveness directly and the studies mentioned above combined LiST modelling with additional Excel-based models to develop comparative cost effectiveness estimates. The traditional approach to technical assessments of how to achieve the best value for money is to calculate a cost-effectiveness ratio for each intervention in isolation, then rank them in a league table, nominating funding to each intervention in sequence to scale until all resources are depleted. We and others have shown that the traditional league table approach is not the best use of resources, because it does not account for the interactions between interventions and disease burden dynamics due to other implementations [65].
MINIMOD is the only other decision-analytic model that examines allocative efficiency of nutrition investments and identifies a mix of interventions that maximizes specific nutrition or health outcomes. One particular strength of the model is that is allows for comparison of different delivery platforms for a single intervention (e.g. vitamin a supplementation through campaigns, facility-based supplementation, fortification of different food vehicles) [66]. However, currently, the model is limited to micro-nutrient interventions and requires very specific data on consumption, diet composition, and micro-nutrient deficiency levels which need to be collected through specialized surveys (see [67]). Furthermore, MINMOD does not have a user interface which limits its utility for program managers and decision makers. In contrast, Optima nutrition considers a much broader range of interventions, including micronutrient supplementation, behavioural interventions, and provision of complementary foods. Most of the data the model requires is easily available through surveys such as DHS or MICS, of which several waves exist for most developing countries. Finally, a graphic user interface is under development. The final version of the tool will be cloud-based and available online to researchers, program managers and policy makers. As with any model being applied to a complex realworld problem, applications of the Optima Nutrition tool will have limitations. Mathematical models are simplifications of the processes that they represent. The current application had limitations in model structure. This includes considering the populations in Bangladesh to be uniformly mixed within each age band. Interventions were also considered without flow-on effects; for example, breastfeeding promotion targeted to mothers of children aged < 6 months may have ongoing benefits for age-appropriate breastfeeding up to 23 months, but these effects are likely to be setting-and programspecific and are largely unknown. In particular, details of the content of the programs (e.g. number of visits and help provided during each visit for IYCF) are rarely studied and therefore difficult to quantify. Another type of limitation is around model parameterization and data. Where studies were available which evaluated the impact of nutrition interventions from the literature, they were used in this study, assuming the effectiveness levels were consistent universally, when in reality they are more likely to be country-dependent. This leaves inherent uncertainty in the optimization. In addition, uncertainty in the demographics (e.g. age structure of under-five population), nutritional (e.g. stunting prevalence, feeding practices) and costing (e.g. unit costs of interventions, structure of cost functions) data collected in country surveys contribute to overall uncertainty in the results and conclusions of the study. The results are implications of the data used. We used the best data available, assuming that population surveys (such as DHS) are reasonably precise and representative of the target populations. We used a complementary nutritional interventions costing study in Bangladesh to parameterize the unit costs used in this model. That costing study was performed with an intention to be used in this analytical study. Despite the limitations in the structure and parameterization of the tool, we believe that the approach taken in this study appropriately sheds new insight into prioritization of nutrition interventions in Bangladesh. Modelled outcomes should only serve as guidance in the decisionmaking process, representing the best available evidence at the time. Other factors, including various social, political and implementation demands and constraints need to be considered.
Currently, Optima nutrition is limited to a set of six nutrition-specific interventions affecting stunting. An extension of the model to other aspects of malnutrition (wasting, anaemia), nutrition behaviours (prevalence of breastfeeding) and nutrition-specific interventions targeting those outcomes is ongoing. The expanded version of Optima Nutrition will also include some nutritionsensitive interventions such as water and sanitation (WASH) and family planning.