Evaluation of the effectiveness of care groups in expanding population coverage of Key child survival interventions and reducing under-5 mortality: a comparative analysis using the lives saved tool (LiST)

Background Globally, less than half of Countdown Countries will achieve the Millennium Development Goal of reducing the under-5 mortality rate (U5MR) by two-thirds by 2015. There is growing interest in community-based delivery mechanisms to help accelerate progress. One promising approach is the use of a form of participatory mothers’ groups, called Care Groups, for expanding coverage of key child survival interventions, an essential feature for achieving mortality impact. Methods In this study we evaluate the effectiveness of Care Group projects conducted in 5 countries in Africa and Asia in comparison to other United States Agency for International Development-funded child survival projects in terms of increasing coverage of key child survival interventions and reducing U5MR (estimated using the Lives Saved Tool, or LiST). Ten Care Group and nine non-Care Group projects were matched by country and year of program implementation. Results In Care Group project areas, coverage increases were more than double those in non-Care Group project areas for key child survival interventions (p = 0.0007). The mean annual percent change in U5MR modelled in LiST for the Care Group and non-Care Group projects was −4.80 % and −3.14 %, respectively (p = 0.09). Conclusions Our findings suggest that Care Groups may provide a promising approach to significantly increase key child survival interventions and increase reductions in U5MR. Evaluations of child survival programs should be a top priority in global health to build a greater evidence base for effective approaches for program delivery.


Background
The under-5 mortality rate in the least-developed countries of the world has declined from 171 to 98 deaths per 1000 live births between 1990 and 2011 [1]. Despite this progress and the highly effective and inexpensive community-based interventions for addressing the leading causes of under-5 mortality now available [2,3], less than half of the 75 Countdown Countries will achieve the Millennium Development Goal (MDG) for children of reducing the under-5 mortality rate by two-thirds by 2015 [4]. Furthermore, to achieve the new post-MDG target of ending preventable child deaths by 2015 [5], the annual rate of decline in under-5 mortality will need to double [6].
The child survival literature is replete with studies of intervention effectiveness, analysis of constraints encountered in program implementation, and policy issues. One of the important gaps in the literature, however, is identifying effective delivery strategies for expanding coverage of key child survival interventions. The need for studies on effective delivery strategies is urgent-among the Countdown countries, median national coverage of almost all of the key child survival interventions apart from immunizations and Vitamin A is less than 50 % [4].
The United States Agency for International Development (USAID) has been funding non-governmental organizations (NGOs) to implement child survival programs through the Child Survival and Health Grants Program (CSHGP) since 1987. CSHGP projects implement a standard set of proven high-impact technical interventions and work collaboratively with Ministries of Health and communities to promote behavior change and increased health service utilization through frequent contact with community leaders, groups of mothers, and household visits [7]. One analysis of 12 CSHGP-supported child survival projects provided plausible evidence that these projects doubled the rate of decline in under-5 mortality relative to that in other areas in the same country where the projects were not being implemented (5.8 % versus 2.5 %) [8].
There is a lack of standardized and rigorously evaluated strategies for delivery of community-based child survival interventions. In this study we evaluate Care Groups, a community-based delivery strategy that has emerged through CSHGP, in comparison to other CSHGP-funded child survival programs which do not utilize a common standardized strategy, in terms of increasing coverage of key child survival interventions and reducing under age five mortality, using mortality estimates from the Lives Saved Tool (LiST).

The care group model
The Care Group model was developed by World Relief, an international NGO based in Baltimore, MD, and its child survival staff working in Gaza Province, Mozambique in the late 1990s. The approach involves the formation of mothers' groups of approximately 10 Care Group volunteers who are each responsible for visiting on average 10 households closest to their home. A facilitator (i.e., supervisor) visits a Care Group every 2-4 weeks to teach the volunteers 1-3 new key messages to share with their neighbors. Household visits by Care Group volunteers are conducted every 2 weeks. Over a 2-year period, an array of lessons covering household behaviors to promote maternal and child health and healthcare utilization are disseminated to every household in the project area where there is a pregnant woman or a mother of a child less than 59 months of age. Through the use of volunteers at the household level, rather than paid promoters, the Care Group model presents a low cost approach for program delivery. One paid local facilitator (often called a promoter) can oversee the activities of approximately 400-1000 households [9][10][11].
To date, 23 NGOs have implemented the Care Group Model in 27 countries [12,13]. Two peer-reviewed publications have documented the effectiveness of the Care Group Model in increasing coverage of child survival interventions and reducing under age five mortality [14][15][16]. However, none of these studies have conducted a systematic evaluation of Care Group projects compared to other child survival programs.
Features of the care group and Non-care group projects included in the analysis Below we describe the typical delivery approaches used for Care Group projects compared to other CSHGP child survival projects, non-Care Group projects, through two representative examples. The non-Care Group projects all use collaborative approaches with communities and community groups to expand coverage of key child survival interventions and most utilize home visitation in some form along with women's group meetings (e.g., mothers clubs). However, they vary widely in the number of home visits per month and the form of women's groups utilized. Project interventions included education about the prevention and treatment of malaria, nutrition education, education about maternal and newborn care, and awareness and prevention of sexually transmitted diseases (STIs) and HIV/AIDS. The project included the training and participation of traditional healers, health facility workers, volunteer CHWs and community-based organizations. Behavior change strategies included mass media campaigns, interpersonal and peer communication, and support groups. The project worked with 479 existing volunteer CHW who were members of 12 associations (4 CHW associations, 4 persons living with HIV/AIDS associations, and 4 traditional birth attendant associations).

Study inclusion criteria
A list of Care Group projects funded by the CSHGP was obtained from the Care Group Info website [17] in March 2013. Information on non-Care Group projects, including the data from the baseline and endline household surveys, was obtained from the USAID Evaluations database [18].
Care Group projects were selected for inclusion in the present analysis if the following eligibility criteria were met: (1)There was a Demographic Health Survey (DHS) or Multiple Indicator Cluster Survey (MICS) available for the country where the Care Group project was implemented within 3 years of both the project baseline and endline household surveys. (2)There was at least one non-Care Group child survival project funded by CSHGP that was implemented in the same country within 3 years of the Care Group project that met criterion 1. This allowed us to match Care Groups and non-Care Groups by country. (3)The project had baseline and endline Knowledge, Practices, and Coverage (KPC) data for a clearly defined subnational area in which it was intervening.
Ten Care Group and nine non-Care Group child survival projects met these study eligibility criteria. Six of the projects took place in Cambodia, three in Kenya, three in Malawi, four in Mozambique, and three in Rwanda ( Table 1). All of these projects were implemented between 1998 and 2010, and the majority lasted 5-6 years.

LiST modeling
The Lives Saved Tool (LiST) is a modeling tool that estimates the impact on mortality outcomes of scaling up 30 child health interventions [19]. LiST-specified effect sizes for each intervention are derived according to standards set by from the Child Health Epidemiology Reference Group [20]. In the present study, LiST version 4.68 was used to estimate the impact of each child survival project on the under-5 mortality rate (U5MR). This version of the software was downloaded from the Johns Hopkins Institute for International Programs website in March 2013 [21]. The methods and assumptions of the LiST tool have been previously described [22]. LiST has been validated to produce reasonably accurate estimates of national under-5 and neonatal mortality, when coverage changes and other necessary parameters are accurately measured [23]. LiST has also been validated to produce reasonable sub-national mortality estimates in various settings in South Asia and Africa [24][25][26][27][28].
We modeled baseline mortality rates for child survival project areas using subnational DHS or MICS mortality data ( Table 2). Since DHS and MICS subnational under-5 mortality rates are derived from deaths that occurred over a 10-year period prior to the survey, we used mortality data from the later of the two DHS or MICS reports as the baseline mortality rate modeled in LiST. This allowed us to have a more recent estimate of U5MR in the population. DHS reports were downloaded from the Measure DHS website [2]. MICS reports were obtained from UNICEF's Child Info website [16].

Modeling project coverage changes in LiST
Project coverage data at baseline and endline were obtained from either the project household KPC surveys (obtained from the USAID CSHGP or Care Group Info website) or from project staff [18]. The methodology used for KPC survey sampling is described elsewhere [8]. All USAID CSHGP-funded organizations are required to report the results of their KPC surveys using a standard methodology in their final evaluation reports, and these reports are uploaded to the USAID Evaluations web-based database [29]. If project coverage data were not available for specific indicators at baseline or endline, they were supplemented by DHS or MICS coverage data for the subnational region where the project was implemented. The proportion of the subnational population covered by the projects ranged from 3 to 100 % with a median of 16 %. The International Rescue Committee project in Kibungo Province, Rwanda conducted between 2001 through 2005 targeted the entire subnational population.
In our analysis we focus on the 17 high-impact coverage indicators modeled in LiST that were evaluated in a similar analysis conducted by Ricca et al. (Table 3) [8]. If the exact LiST indicator was not available in the project report or DHS surveys, we used a similar "project indicator" described in Table 3, based on previous methods implemented in Ricca et al. [8].

Statistical analysis
The U5MR estimated by the LiST tool was used to determine the annual percentage change in U5MR over the project period. We compared changes in intervention coverage and U5MR between the Care Group and non-Care Group CSHGP projects using the Wilcoxon signed-rank test. The mean number of households per CHW for Care Group and non-Care Group CSHGP projects were compared using the signed-rank test. There is some debate in the statistical literature relating to the analysis of matched data as we have, where matching is based on intervention status, and whether matching needs to be explicitly accounted. In our present study we do not match by country cluster because our research question is more focused on the ability of the Care Group child survival programs to change intervention coverage, and reduce annual U5MR rather than differences in project delivery between countries [30].

Ethical approvals
We obtained an exemption from the Johns Hopkins institutional review board to conduct our analysis.

Results
Coverage changes for care group and Non-care group child survival projects Changes in the 17 high-impact coverage indicators included in the LiST analysis are presented for the 10 Care Group and 9 non-Care Group CSHGP projects in Table 4. On average, each Care Group child survival project implemented 10 out of the 17 high-impact interventions modeled in LiST compared to 7 for the non-Care Group projects. In Care Group project areas, coverage increases for high impact interventions were more than double those in non-Care group project areas for antenatal care visits, tetanus toxoid vaccination, multiple micronutrient supplementation, complementary feeding, hand washing with soap, oral rehydration therapy, oral antibiotics for pneumonia, and malaria treatment.
Overall, Care Group projects yielded significantly higher increases in coverage than non-Care Group projects for high-impact coverage indicators measured (p = 0.0007). The mean number of households per CHW was lower for Care Group projects (11:1) compared to non-Care Group projects (22:1) (p = 0.009). The mean expenditure per child life saved was not significantly different between the Care and non-Care Group projects at 62.90 USD and 66.61 USD, respectively.

LiST modeling
All available coverage indicators were modeled in LiST to calculate the mean percent annual change in U5MR for the Care Group and non-Care Group CSHGP projects. The mean annual percent change in U5MR for the Care Group and non-Care Group projects was −4.80 and −3.14 %, respectively (a ratio of 1.53, p = 0.09). The mean annual percent change in U5MR was higher for the Care Group projects versus non-Care Group CSHGP projects for 4 out of the 5 countries included in our analysis.

Discussion
This analysis presents the first published evaluation of the effectiveness of Care Groups in comparison to other CSHGP-funded child survival programs in terms of  Our present analysis has several limitations. First, since we wanted to compare Care Group projects with non-Care Group projects implemented in the same country around the same time (to control for contextual influences), the sample size of our current evaluation was small. This likely limited our power to detect significant differences between Care group and non-Care Group projects in terms of mean annual percent change in U5MR. As more Care Group projects are implemented, a meta-analysis of these projects is urgently needed. Second, there were some differences observed between CSHGP project and LiST definitions for coverage indicators. However, these differences should be consistent between Care Group and non-Care Group projects, and therefore minimally affect comparisons of mortality estimates between the two groups. Third, we lacked data on actual mortality impact for the projects; therefore, we relied on estimates from LiST. With the goal of validating the use of the LiST tool, a recent evaluation of a Care Group project implemented in Mozambique found that the LiST modelling of intervention coverage data, when compared to direct measurements of mortality decline, yielded reasonably accurate estimates of mortality reduction (39 % compared to 34 %, respectively) [13]. Fourth, we lacked a control area in the region where each project was conducted. Randomized controlled trials are needed to rigorously evaluate the impact of the Care Group model on changes in coverage and U5MR. Fifth, we did not compare a single community-based delivery mechanism to the Care Group model; instead, we included all other CSHGP projects implemented at the same time and in the same country. Future studies should compare the Care Group model to other commonly used community-based delivery strategies such as Participatory Learning and Action Groups [31].
Our current findings are consistent with previous evidence that the Care Group model is effective in significantly expanding coverage of key child survival interventions [15,32,33] reducing undernutrition [15], reducing childhood diarrhea [32], and lowering under-5 mortality [9,14]. We believe these positive effects are due to the additional reinforcement Care Groups can provide through their significantly lower community worker to household ratio (11:1), a feature that made Care Groups distinct from the other child survival programs included in this analysis. The lower community worker to household ratio allowed more time for lessons, and motivated community workers to switch to new topics and interventions every few months to keep their audience interested.
All countries in the present study are on the Countdown to 2015 list, and therefore among the 74 countries worldwide where 97 % of the world's child deaths are occurring [4]. Innovations such as the Care Group Model are needed to expand coverage of key child survival interventions to achieve the 2035 goal of ending preventable child mortality, which will require doubling the global rate of decline in under-5 mortality.

Conclusions
Our findings suggest that the Care Group model may provide a promising approach to significantly expand key child survival interventions and increase reductions under age 5 mortality which warrants future investigation. In spite of the many limitations of this study, we believe that it is an important contribution to the literature. It assesses evidence from multiple settings and projects regarding the effectiveness of the Care Group model in comparison to other child survival projects in improving delivery of multiple child survival interventions, rather than the efficacy of single interventions, which has been the focus of most research to date. Since evaluations of many Care Group projects in addition to those reviewed here are now publicly available, further analysis of these findings is warranted, including a metaanalysis of the findings from all available evaluations. In addition, new programs using the Care Group model are needed that are carefully documented and combined with rigorous external evaluations. Evaluations of integrated child survival programs of all types in the peerreviewed scientific literature should be a top priority in global health to build a greater evidence base for effective approaches for program delivery.