Advances in childhood immunisation in South Africa: where to now? Programme managers’ views and evidence from systematic reviews

The Expanded Programme on Immunisation (EPI), which is one of the most powerful and cost-effective public health programmes to improve child survival [1–4], was introduced in South Africa in 1974. The programme, however, remained fragmented because of the system of apartheid until 1995; when the national EPI was formed through the unification of all immunisation services in the country [5]. Since then, there have been significant advances in immunisation services delivery in South Africa, including the introduction of the hepatitis B virus vaccine and Haemophilusinfluenzae type b conjugate vaccine into EPI in 1995 and 1999, respectively. In addition, neonatal tetanus elimination was validated in 2002 and interruption of wild poliovirus transmission was declared in 2006 [5]. More recently, in April 2009, South Africa became the first country in Africa to introduce nationwide routine childhood vaccination against rotavirus and Streptococcus pneumoniae; at no cost to recipients.

Despite these advances, there is some evidence that EPI South Africa faces a number of challenges [6–9]. The vaccination coverage is low (with only two-thirds of children estimated to receive the full series of three doses of the diphtheria-tetanus-pertussis vaccine by one year of age) [8, 10], measles outbreaks are frequent [6, 9], and community knowledge of immunisation is low [7]. In order to ensure evidence-informed selection and implementation of effective healthcare interventions [11] that would overcome existing barriers and reach every child in South Africa with life-saving vaccines, we conducted an audit to elicit programme managers’ views of the challenges facing EPI and potential solutions for these challenges. Our objectives were: (1) to determine what EPI programme managers in South Africa consider as key barriers to effective implementation of the programme in the country; (2) to determine what EPI programme managers consider as the best interventions to overcome the identified barriers; and (3) to conduct a comprehensive search of peer-reviewed literature, identify, and synthesise current best evidence on the effectiveness of the interventions proposed by the EPI programme managers.

Eight days after the initial email request, we sent a follow-up reminder to non-responders. Responses were collated two weeks later. The findings of this first audit were presented at a national immunisation conference in October 2009; which was attended by all key immunisation stakeholders in South Africa [12]. The conference agenda included a critical appraisal of vaccines and vaccine administration, globally and with specific reference to South Africa. In June 2010, the two questions were again sent to all the 10 EPI managers in the country.

The exercise was initially undertaken as a programme evaluation for management purposes, and therefore no ethics review board approval was sought. However, only later we realised that the results could be of interest to a wider audience. All EPI managers who responded to the questionnaire provided consent for their responses to be published. In addition, we obtained the permission of the South African National Department of Health to publish the findings. In this paper we present the key challenges identified by the managers, the key solutions they proposed, and the findings from systematic reviews on the effects of the proposed interventions. Each respondent provided five barriers and five corresponding corrective interventions. In summarising the responses, we treated the barriers identified (five per manager) equally without applying any weighting; irrespective of whether the barriers were identified by the national or provincial managers, or whether the barrier was classified by the manager as most important or least important. We did the same for the proposed interventions (one per identified barrier).

On 30 November 2010 we searched the Health Systems Evidence database, the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews of Effectiveness, and PubMed. One author (CSW) conducted the search using the search strategy shown in Table 1. We screened the search outputs in the order in which the databases are listed here, starting with Health Systems Evidence database and ending with PubMed. Figure 1 shows a summary of the search and selection process. When we found more than one systematic review that assessed a particular intervention, we chose the one that was more comprehensive and/or more recent. Two authors (CSW and MSS) independently screened the search results, selected relevant systematic reviews, and assessed the quality of selected reviews using the AMSTAR tool [13]. In particular, we assessed whether the authors of the review reported the study selection criteria; conducted a literature search that was comprehensive enough to avoid publication, language and indexing biases; undertook duplicate study selection and data extraction; used reliable criteria to assess the risk of bias in included studies; reported the characteristics of included studies appropriately; and combined data from included studies using reliable methods. Based on these criteria, we concluded whether the review was well conducted (i.e. reliable) or not. We have only reported data from reviews that we considered to be reliable. At each stage, the two authors compared their results and resolved any disagreements by discussion and consensus.

A. Interventions directed at healthcare workers
Health Systems Evidence:
Health system topic	Provider-targeted strategy
Type of synthesis	Systematic review OR policy brief OR systematic review protocol
Type of question	Effectiveness
Publication date range	2000 to 2010
Cochrane Database of Systematic Reviews (CDSR):
Search all text	(Training OR education OR workshop OR supervision OR (outreach AND visit*) OR (audit AND feedback) OR monitoring) AND (“Immunization”[Mesh] OR “Vaccination”[Mesh] OR “ Immunization Programs”[Mesh])
Limits	None
Database of Abstracts of Reviews of Effectiveness (DARE):
Search all text	(Training OR education OR workshop OR supervision OR (outreach AND visit*) OR (audit AND feedback) OR monitoring) AND (“Immunization”[Mesh] OR “Vaccination”[Mesh] OR “Immunization Programs”[Mesh])
Limits	None
PubMed:
Search terms	(Training OR education OR workshop OR supervision OR (outreach AND visit*) OR (audit AND feedback) OR monitoring) AND (“Immunization”[Mesh] OR “Vaccination”[Mesh] OR “Immunization Programs”[Mesh])
Publication date	01 January 2000 to 31 December 2010
Publication type	Reviews
B. Interventions directed at parents or communities
Health Systems Evidence:
Health system topic	Consumer-directed strategy
Type of synthesis	Systematic review OR policy brief OR systematic review protocol
Type of question	Effectiveness
Publication date range	2000 to 2010
Cochrane Database of Systematic Reviews (CDSR):
Search all text	(mobiliz* OR mobilis* OR communicat* OR advoca*) AND (“Immunization”[Mesh] OR “Vaccination”[Mesh] OR “Immunization Programs”[Mesh])
Limits	None

Finally, we used the GRADE approach [14, 15] to assess the quality of the evidence for the effectiveness of the proposed strategies. This method results in an assessment of the quality of a body of evidence as high, moderate, low, or very low. High quality evidence implies that “further research is very unlikely to change our confidence in the estimate of effect”. Moderate quality evidence means that “further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate”. Evidence is considered of low quality if “further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate”, and very low quality if “we have very little confidence in the effect estimate”. We began the rating of the quality of evidence with the study design; evidence from systematic reviews of randomised controlled trials as high-quality and that from systematic reviews of observational studies as low-quality. In addition, five reasons led us to downgrade the quality of evidence from systematic reviews of randomised controlled trials and three to upgrade the quality of evidence from systematic reviews of observational studies. For pooled data from randomised controlled trials, the factors that led to rating down the quality of evidence were risk of bias, heterogeneity, indirectness, imprecision, and publication bias. Regarding risk of bias, concerns that limited our confidence in the evidence include lack of allocation concealment, lack of blinding of outcome assessment, and a large loss to follow-up. Heterogeneity of effects across studies for which there were no compelling explanations also reduced our confidence in the evidence. Indirectness refers to differences between the population, intervention, comparison group and outcome of interest to us, and those included in the relevant reviews e.g. we used the evidence on strategies for improving patients’ understanding of health information as a proxy for evidence on parents’ understanding of the importance of childhood immunisation [16]. For imprecision, if we found that studies included relatively few participants and few events and thus had estimates of effects with wide confidence intervals, we rated down the quality of the evidence. Finally, we downgraded the quality of evidence if there was a high likelihood of publication bias. Furthermore, we upgraded the quality of the evidence if the pooled estimates revealed a large magnitude of effect, if we had negligible concerns about confounders, or if there was a strong dose-response gradient [14].

Decision makers in South Africa need to use considerable judgement about how best to use limited resources they have for maintaining and improving the quality of health care in order to maximise population benefits. In making such decisions, they need to consider the potential areas for quality improvement activities, the likely benefits and costs required to introduce new quality improvement interventions, and the likely benefits and costs as a result of any changes in the behaviour of healthcare workers. With the values and preferences of programme managers as a starting point, we found that the use of interactive educational meetings and workshops, audit and feedback, supportive supervision, parent reminder and recall systems, conditional cash transfers, community health workers, mass media interventions, and interactive communication tools could be effective in improving EPI performance in South Africa. In selecting which combination of interventions to use for which community and at which time point, decision makers would need to consider population characteristics, available resources, and competing priorities. A major strength of our approach comes from having a framework for interventions (i.e. collating managers’ views of the barriers to effective implementation of the EPI programme and their proposals of effective interventions for addressing these barriers) before looking for systematic reviews of effects. If we had searched the literature for effective interventions for improving childhood immunisation, without such a framework, we would have been driven by what systematic reviews have been done and what is easily researchable; rather than the priorities of key immunisation stakeholders.

The settings and designs of the studies included in the systematic reviews that assessed the effects of interactive educational meetings, audit and feedback, and educational outreach visits for improving healthcare worker performance varied widely; but the studies consistently showed that these strategies can improve healthcare worker performance [17–19].The reviews also found that multifaceted interventions may not be any more effective than educational meetings, outreach visits, or audit and feedback alone [17–19]. The consistency of effects across different study designs and healthcare settings and conditions suggests that these findings would be applicable to the EPI in South Africa. The low quality of the evidence on the effects of passive distribution of printed educational materials [20] implies that we have limited confidence in the effectiveness of this strategy to improve the knowledge of vaccine and immunisation issues among healthcare workers in South Africa [14]. Social mobilisation for immunisation may include active community participation, contextualisation of information in the local customs and culture, and involvement of a broad range of stakeholders and the mass media. The moderate quality of the evidence on parent reminder and recall systems [21], community health workers [22], interactive communication tools [16], conditional cash transfers [24], and mass media interventions [23] is an indication that these strategies could have significant effects in mobilising communities and increasing demand for routine childhood immunisation services in South Africa.

Appropriate financing mechanisms are needed to ensure that there are sufficient funds (at all times in all health districts) to recruit and retain a sufficient number of qualified staff, procure adequate quantities of vaccines, purchase and maintain the appropriate vaccine-related equipment and other logistics; and maintain and improve the quality of care. Evidence of effects, population characteristics, societal values, and attitudes are important when selecting funding schemes which aim to ensure universal coverage of health services. Childhood immunisation services are provided free of charge at public health facilities in South Africa [5]. Therefore, sustainable EPI financing mechanisms relevant to the South African context would include the use of general tax revenues or a social health insurance scheme to cover the costs of immunisation services. Social health insurance refers to compulsory health insurance that aims to provide universal coverage. The compulsory nature of social health insurance should reduce adverse selection and enable redistributive mechanisms between poor and affluent segments of the population. However, few examples exist of social health insurance schemes operating at a large scale in low and middle-income countries; none of which provides strong evidence related to their impact [25]. The National Health Insurance (NHI), whose planned introduction in the health sector in South Africa has become a very topical issue, is a type of social health insurance scheme. The South African National Department of Health indicates that the NHI “is based on the key principles of universality, that every South African would be entitled to benefit from the services it covers, and that it would be funded partly by compulsory contributions by all persons who are earning an income and partly by tax. All these funds would be placed in a single pool. This pool would be available to fund all health care in the public and private health sector under conditions that would apply to all health care service providers.” [26].

Our study has some limitations. One provincial manager did not respond to our questions. However, given the consistency of the key challenges across the other eight provinces, we do not think that the EPI challenges in the province of the manager who did not respond would be substantially different from those of the other provinces. There is a possibility that the presentation of findings of the first round to EPI managers may have influenced the responses in the second round of the audit. However, this does not seem to have been the case because new challenges (e.g. reluctance from parents) emerged from the second round. Regardless of these potential shortcomings, we believe that this study is a good reflection of the challenges encountered in the planning, delivering, and monitoring of childhood immunisation services in South Africa. Most of the evidence of effects described above comes from studies conducted in high-income countries, and applicability to South Africa or other low and middle-income countries may be limited. There is thus a need for high-quality studies from low and middle-income countries, assessing the effects of strategies for improving immunisation services. In the meantime, implementation of such strategies in South Africa should be pilot-tested and their impacts and costs rigorously monitored and evaluated [11].

In line with the Millennium Development Goals, we have to ensure that our children’s right to health, development and survival is respected, protected and promoted. EPI is central to this vision. We found numerous promising strategies for improving EPI performance in South Africa. However, their implementation would need to be tailored to local circumstances and accompanied by rigorous monitoring and evaluation. The suggested interventions (by EPI programme managers) for handling immunisation challenges and the availability (or lack) of sound scientific evidence on their effectiveness, emphasise the need for partnerships between health policy makers, programme managers, and researchers in order to ensure that health decisions are always informed by the best available evidence [27, 28]. Collaboration between the three stakeholders needs to be continuous, as the results of this audit show that challenges to immunisation may vary from time to time. Such policymaker-implementer-researcher partnerships are needed to create a health system that is effective, equitable, and sustainable; one in which EPI will be better able to meet its obligations.