Are we doing enough? Evaluation of the Polio Eradication Initiative in a district of Pakistan's Punjab province: a LQAS study

The success of the Global Polio Eradication Initiative (GPEI), launched after a World Health Assembly resolution of 1988, was remarkable and nearly 5 million children were protected from being paralyzed by 2003 [1, 2]. About 10 billion doses of polio vaccine have been administered since 1988 on hundreds of national and sub-national immunization days (NIDs/SNIDs) at a cost of US$ 4.5 billion [3, 4]. But four countries - Afghanistan, Pakistan, India and Nigeria - never interrupted wild polio virus (WPV) transmission and re-infection of 26 countries in 2006-07 raised questions about the feasibility of polio eradication. By the end of 2007, polio incidence had decreased by 35% and polio transmission had stopped in all but 6 re-infected countries [5], yet polio eradication seems illusory as endemic countries are missing targets and 1648 cases were reported worldwide in 2008 [6].

Pakistan adopted the Polio Eradication Initiative (PEI) within the Expanded Program on Immunization (EPI) in 1994 remarkably decreasing the number of cases to 32 by 2007 including 19 WPV type 1 and 13 WPV type 3 cases with virus found in 18 of 120 districts [2, 6–8]. But polio resurged in 2008 infecting 49 districts with 118 cases including 81 WPV type 1 and 37 WPV type 3 cases - highest amongst the previous six years [2, 6–8]. Pakistan reportedly achieved all targets set in the GPEI strategic plan but failed to interrupt virus transmission [5, 9–11], probably due to sub-district coverage gaps, low routine coverage, operational weaknesses in the quality of services and large numbers of children missed during NIDs/SNIDs. The scenario calls for immediate action.

As health programs mature, large-scale surveys become less useful. Instead, data from small-scale studies are required to evaluate different aspects of a program [12]. The aim of this study, in one district of Pakistan's Punjab province, was to establish valid and reliable estimate for the effectiveness of polio eradication services and to ultimately provide scientific evidence for tailoring future interventions. Specific objectives were the evaluation of routine immunization services including oral polio vaccine (OPV) coverage and the characteristics of vaccinated and unvaccinated children, the evaluation of logistics management and the quality of monitoring systems, and the assessment of national immunization days (NIDs) including NIDs coverage and the quality of NIDs service delivery.

The historically used LQAS design is still an appealing technique for rapid evaluation of preventive programs, especially in rural areas of developing countries [19–23]. It is very efficient in determining the performance of individual subunits in a specific area [22, 23]. It was observed that most health personnel were unaware of this technique and its effectiveness in local evaluations.

Five out of twenty lots were rejected for unacceptably low routine immunization coverage, while the validity of routine overage was questionable to the extent that all lots were rejected. Weakness in routine immunization is the main constraint in polio eradication [21]. A national immunization card program could significantly increase coverage and the validity of coverage [22, 23]. Knowledge gaps underlie low compliance with vaccination schedules, and the quality of interaction between health workers and caretakers is essential to ensure compliance with vaccination schedules [24].

Nearly half of the children sampled could not present a card. Coverage was significantly higher among those having cards, consistent with previous studies [19, 23, 25, 26]. Card availability and immunization status were significantly associated with socioeconomic factors, consistent with previous studies [23, 26–28]. Health workers were the major source of immunization and information. Outreach services have been recommended as the most efficient way to increase coverage [27–29] and there is a need for interventions aimed at fostering the communication skills of health workers [30, 31]. The main reasons for routine immunization failure were educational constraints and problems of accessibility and availability, in agreement with previous literature [19, 21, 23, 32].

All 20 lots were rejected for poor compliance in logistics management and the quality of monitoring systems. Previous compliance studies have also indicated poor performance regarding cold chains and logistics [32–37]. The functioning of refrigerators was poor due to security and electricity problems. Maintenance of refrigerator temperatures within the correct range and vaccine storage were also poor, which can be attributed to the lack of technical skills among health workers and poor attitudes towards maintaining cold chain charts. Maintenance of frozen icepacks and cold box/vaccine carriers was poor. This merits necessary action because in case of electricity failure, it would be the only way to maintain cold chains, as power generators were not available at most of the facilities. Previous studies in India revealed vaccine carriers were non-compliant in about one-third facilities, and power failures were also indicated as major constraint [34, 36]. Poor use of safety boxes indicates a lack of awareness about safety measures. A holistic rather than logistic approach should be used for vaccine safety [38].

The quality of monitoring systems and data was poor in all aspects, in agreement with previous studies [32, 39–41]. This was attributed to the lack of supervision and knowledge among those who were responsible, consistent with previous studies [42]. There is a need for monitoring systems to be viewed with a broader perspective, not focusing only on technicalities but also on the support mechanisms [32, 41, 43]. In Pakistan, EPI/PEI records are not computerized; however, the feasibility of linked immunization database systems has been established elsewhere [44], therefore, computerizing EPI/PEI records and linking them with health management information systems already in place might improve data quality. The implications of a poor quality data system are reflected in the efficiency of health services. Previous experience has shown that significant improvements in data quality and monitoring systems can be made by data quality self-assessment (DQS) and the use of data for action [45–49]. This strategy is recommended in the Reaching Every District (RED) approach and Global Framework for Immunization Monitoring and Surveillance (GFIMS) by the World Health Organization [48–51].

The 15 out of 20 lots were rejected for unacceptably low NIDs coverage by finger-mark. Poor coverage during NIDs which had been designed to deliver supplementary OPV doses to all children has been indicated as an underlying factor for the continued transmission of polio [9, 52].

All 20 lots were rejected for poor NIDs service delivery. Poor technical skills of health workers, logistic problems, poor planning, and deficient communication skills were highlighted as the main problems. Similar findings were previously reported and associated with increased risk of non-vaccination [36, 53–56]. In particular, communication with the clients was very poor. Previous studies have revealed the importance of effective communication to improve the coverage [57, 58]. Decisions have generally not been based on studies of populations' knowledge and attitudes about immunization. Had this datum been strategically used, interventions could have been more effective in reaching zero-dose children [54]. Poor planning and lack of technical skills accounted for service disruptions [59], and supportive supervision could significantly increase the performance of immunization sessions [36, 46, 60].

The study does have limitations. Since the status and validity of routine immunization obtained from cards was likely to be an underestimate, immunization by card plus history was included. It is an established practice to estimate coverage by maternal history [61, 62]. Some characteristics evaluated in the study were of subjective nature that might be a limitation in itself. Central selection rather than the random walk method was followed in coverage estimation because of different settlement designs in villages/wards in Pakistan and that may be a limitation. The per lot error for NIDs coverage was high because of the low sample size as a higher sample was not logistically feasible; however, the error for overall coverage was not affected by that.

Although the study was conducted in a rural district of Pakistan's Punjab province, some of findings may be generalized to other areas with similar health system infrastructure, socio-cultural environments and topography. The findings suggest that LQAS studies could be conducted in other areas to assess performance on a sub-national level.

Short-comings revealed in polio eradication services are potentially important due to lack of similar studies in the region and the failure to achieve polio eradication despite continuous efforts by the global community over the past two decades. Following interventions are recommended:

Further studies in the region to evaluate findings are suggested. Polio continues to be a public health problem and we need to have a better understanding of the factors involved in achieving polio eradication.