Risks of serious complications and death from smallpox vaccination: A systematic review of the United States experience, 1963–1968

Abstract

Background

The United States (US) has re-instituted smallpox vaccinations to prepare for an intentional release of the smallpox virus into the civilian population. In an outbreak, people of all ages will be vaccinated. To prepare for the impact of large-scale ring and mass vaccinations, we conducted a systematic review of the complication and mortality risks of smallpox vaccination. We summarized these risks for post-vaccinial encephalitis, vaccinia necrosum (progressive vaccinia), eczema vaccinatum, generalized vaccinia, and accidental infection (inadvertant autoinoculation).

Methods

Using a MEDLINE search strategy, we identified 348 articles, of which seven studies met our inclusion criteria (the number of primary vaccinations and re-vaccinations were reported, sufficient data were provided to calculate complication or case-fatality risks, and comparable case definitions were used). For each complication, we estimated of the complication, death, and case-fatality risks.

Results

The life-threatening complications of post-vaccinial encephalitis and vaccinia necrosum were at least 3 and 1 per million primary vaccinations, respectively. Twenty-nine percent of vaccinees with post-vaccinial encephalitis died and 15% with vaccinia necrosum died. There were no deaths among vaccinees that developed eczema vaccinatum; however, 2.3% of non-vaccinated contacts with eczema vaccinatum died. Among re-vaccinees, the risk of post-vaccinial encephalitis was reduced 26-fold, the risk of generalized vaccinia was reduced 29-fold, and the risk of eczema vaccinatum was reduced 12-fold. However, the risk reductions of accidental infection and vaccinia necrosum were modest (3.8 and 1.5 fold respectively).

Background

In the United States (US) routine smallpox vaccination of the general population ended in 1972 [1]. The last known case of naturally-acquired smallpox occurred in Somalia in 1977, and in 1980 the World Health Organization (WHO) declared that smallpox had been eradicated from the world [2]. Today, most of the US and the world's population is considered susceptible to smallpox infection. Since the September 2001 airplane attacks on the World Trade Center and the anthrax spore attacks using the US mail, the US has taken several steps to mitigate a possible intentional release of variola major (smallpox) virus [3–6], including re-instituting a national smallpox vaccination program [7]. For Phase 1 of this program, the Advisory Committee on Immunization Practices (ACIP) recommended voluntary vaccination of 500,000 members of public health response and hospital health care teams [8]. In Phase 2, up to 10 million first responders such as police, fire, paramedics, and other health care staff would be offered voluntary smallpox vaccination.

However, the implementation the smallpox vaccination program is not without controversy [9]. Vaccinating individuals against smallpox infection, even voluntarily, means exposing vaccinees and their contacts to the risks of serious adverse reactions without the clear benefit of protection against an infection which does not exist and may never exist again. Volunteers must balance their individual risks and benefits of smallpox vaccination. The individual risks and benefits from being vaccinated before an intentional release depends on the probability of an adverse vaccine reaction, the probability that smallpox virus will be released, and the probability that an individual would be directly exposed to a release or an unrecognized case of smallpox [10]. Once an outbreak occurs, ring and mass vaccination would result in a large number of primary vaccinees of all ages, including some with underlying or unrecognized risks for vaccine adverse reactions. In this scenario, we would expect to see an increased number and risk of vaccine complications. Estimation of the vaccine complication risks by age, including their precision and estimation biases, are necessary to formulate policies and inform individual choice regarding the risks and benefits of pre-outbreak smallpox vaccination, and to estimate the burden of vaccine complications from ring and mass vaccination during an smallpox outbreak in the general population.

Recent publications [11–16] have summarized or cited the complication risks associated with receiving smallpox vaccination; however, to our knowledge, no systematic review of this older literature has been conducted. The purpose of this systematic review is to review and summarize the age-specific complication and case fatality risks of post-vaccinial encephalitis, vaccinia necrosum (progressive vaccinia), eczema vaccinatum, generalized vaccinia, and accidental infection (inadvertant autoinoculation) from smallpox vaccinations conducted in the US. We restricted our review to US studies because the New York City Board of Health vaccinia strain, which was the only strain used in the US, is the strain in current use by the the national vaccination program (and will be in the future) [11].

Methods

Searching

Two reviewers (T.J.A., S.U.) conducted the literature searches, selection of studies, and data extraction. To identify potential studies we searched MEDLINE which covered 1965 through November 2002. We entered the following search string: ((Smallpox Vaccine/*adverse effects [MeSH terms] AND "English" [Language]) OR ((Vaccination/*adverse effects [MeSH terms] AND "vaccinia" [MeSH terms]) AND "English" [Language])). From this search result, potentially relevant studies on smallpox vaccine complications were identified, screened, and retrieved. The reference lists in all retrieved articles, including reviews, were evaluated to identify, screen, and retrieve additional articles not identified in the MEDLINE search. Selected primary authors were contacted to identify additional studies we may have missed [17, 18]. Based on this consultation and our inclusion criteria (described below) we did not pursue articles published before 1965 or from additional US databases.

Selection criteria

All US studies of smallpox vaccine complications were included for further review. Articles were screened further based on the following criteria: (1) the number of persons vaccinated and whether the vaccine complications were reported by vaccination status; that is, whether it was a primary vaccination or a re-vaccination, (2) sufficient data were provided to calculate complication or case-fatality risks, and (3) one or more vaccination complications were consistent with case definitions as defined by Neff and colleagues [19]. Prior to this publication there were no consistent case definitions of smallpox vaccine complications. The following summarizes the Neff criteria:

• Encephalitis: post-vaccinial central nervous system involvement, including separately or in combination, the following symptoms: meningeal signs, ataxia, muscular weakness, paralysis, lethargy, coma or convulsions,

• Vaccinia necrosum (progressive vaccinia): spreading necrosis at the site of the vaccination, with or without metastatic necrotic lesions occurring elsewhere on the body,

• Eczema vaccinatum: generalized spread of vaccinial lesions or local implantation of vaccinia in a person who has eczema or a past history of eczema,

• Generalized vaccinia: generalized vaccinial lesions, with no eczema or other pre-existing lesion,

• Accidental infection: vaccinial lesions resulting from accidental autoinoculation of vaccinia virus in the eye or mouth or on other parts of the body, with no eczema or other pre-existing skin lesions.

Validity assessment

For this review we did not develop a quality score. Instead, we collected relevant information that might bias the complication and case fatality risk estimates from each article. This included information on the study location, whether potential vaccinees were screened (e.g., history of eczema/atopic dermatitis), reporting source of vaccine complications and deaths (numerators), and source of numbers of persons vaccinated (denominators). This information was summarized to help the reader interpret the findings.

Data abstraction

Data from each article were abstracted separately by the two primary reviewers. For quality control, two additional staff independently abstracted the data, and any disagreements were resolved by the primary reviewers.

Study characteristics

The selected studies were published in English. They had the type and number of vaccine complications, and an estimate of the number of persons vaccinated. The type and number of complications came from several sources which are summarized in Table 1. The number of persons vaccinated were either known as in the US cohort studies [20, 21], or were estimated using data from US national or state immunization surveys [19, 22–25].

Table 1 Comparison of smallpox vaccination adverse effects studies meeting selection criteria for this systematic review

Quantitative data synthesis

Data in all the studies, except the Ratner study, were stratified by vaccination status and by the following age groups (age <1 year, ages 1 to 4 years, ages 5 to 19 years, ages 20+ years, and all ages combined). For the Ratner study [21], data in the age groups "Age 5 to 14" and "Age 15+" were counted in the groups "Age 5 to 19" and "Age 20+," respectively.

For primary vaccinations, and for each complication, the study risk estimates were stratified by age group and by study, and combined across age groups and studies. First, the age-specific risk of each complication was calculated as the number of reported complications divided by the number vaccinated for that age group. Second, the age-specific risk of death from post-vaccinial encephalitis, vaccinia necrosum, and eczema vaccinatum was calculated as the number of deaths from each complication divided by the number vaccinated for that age group. Third, the age-specific case fatality risks from post-vaccinial encephalitis, vaccinia necrosum, and eczema vaccinatum were calculated as the number of deaths from a specific complication divided by the number with that specific complication. For all risk estimates, exact 95% confidence intervals were calculated based on the binomial distribution using R, a freely available open source programming language for statistical computing and graphics [26].

Although contact vaccinia was not the focus of this review, for non-vaccinees with contact-acquired eczema vaccinatum that was reported, we calculated the case fatality risk.

For re-vaccinations, the same risk estimates were calculated as for primary vaccinations. However, because the risk of complications was much lower for revaccinations, the risks estimates were stratified by study only and summarized in one table.

Results

Our search yielded a total of 348 studies: 342 from the MEDLINE search, 2 from the reference lists review [27, 28], 2 from author communications [29, 30], and 2 from our files [27, 31]. From this pool, 18 articles of smallpox vaccination complications were identified for further review. Using our selection criteria we excluded 11 studies: 3 did not use the Neff criteria [31–33], 3 did not report complications by vaccination status [34–36], 3 did not report the total number of vaccinations by vaccination status [27, 37, 38], and 2 studies' results [39, 40] were reported in another article [25]. The remaining 7 articles form the basis of this systematic review and represent data from approximately 13,206,095 primary vaccinations and 18,349,359 re-vaccinations [19–25] (Table 1).

The US studies covered the years 1963, 1967, and 1968. Five of the 7 studies were period analyses [19, 22–25]. For a period analysis, the number of vaccine complications for a selected geographic location and period of time were ascertained from a variety of sources (Table 1, columns 6 and 7). The at-risk population was the number of persons vaccinated in that location during the same time period. This number was estimated from a variety of sources including population-based regional and national surveys (Table 1, column 8). One study was a prospective cohort study [21] and the other was a clinic-based retrospective cohort study [20].

Primary vaccination

Post-vaccinial encephalitis

The risk of post-vaccinial encephalitis (complication risk, CR), the risk of a post-vaccinial encephalitis death (mortality risk, MR), and the individual risk of dying of post-vaccinial encephalitis (case fatality risk, CFR) by age group are displayed in Table 2. The highest risk for developing post-vaccinial encephalitis was among infants aged <1 year old (risk ratio, 2.80, compared to vaccinees aged 1 year or older). Although the summary risk in this age group was 6.8 cases per million vaccinations, the 1970 Lane et al. study [25] that prospectively surveyed physicians estimated a much higher risk (CR, 176.5 cases per million; 95% CI, 36.4–515). The risk in vaccinees aged >1 year was much lower and did not change with age.

Table 2 Risk of post-vaccinial encephalitis (P.V.E.) and death from primary smallpox vaccinations, Stratified by age and study, 1963–1968

The summary case fatality risk for post-vaccinial encephalitis was highest in infants aged <1 year (CFR, 44.4 deaths per 100 cases; 95% CI, 13.7–78.8), however, it was not statistically different from the other age groups (P value = 0.21). For all ages groups, combined there were 11 deaths among 38 cases of post-vaccinial encephalitis (CFR, 28.9 deaths per 100 cases; 95% CI, 15.4–45.9).

Vaccinia necrosum (progressive vaccinia)

The risk of vaccinia necrosum, the risk of a vaccinia necrosum death, and the individual risk of dying from vaccinia necrosum by age group are displayed in Table 3. The highest summary risks for developing vaccinia necrosum were among vaccinees aged 20 years or older (CR, 5.3 cases per million; 95% CI, 1.4–13.5). For all ages combined, there were 13 cases of vaccinia necrosum from over 13 million vaccinations (CR, 1.0 cases per million; 95% CI, 0.5–1.7) Almost all cases of vaccinia necrosum occurred in persons with a previously diagnosed hematopoetic malignancy or immunodeficiency condition [24].

Table 3 Risk of vaccinia necrosum (V.N.) (progressive vaccinia) and death from primary smallpox vaccinations, Stratified by age and study, 1963–1968

For all age groups combined, there were 2 deaths among 13 cases of vaccinia necrosum (CFR, 15.4 deaths per 100 cases; 95% CI, 1.9–45.4).

Eczema vaccinatum

The risk of eczema vaccinatum, the risk of a eczema vaccinatum death, and the individual risk of dying from eczema vaccinatum by age group are displayed in Table 4. For all age groups combined, there were 169 cases of eczema vaccinatum from over 13 million vaccinations (CR, 12.8 cases per 1 million vaccinations; 95% CI, 10.9–14.9). The summary risks of eczema vaccinatum did not differ across age groups. However, within age groups, the highest risks were recorded from the US studies that used physician surveys to ascertain complications not captured by other reporting mechanisms [22, 25]. In these two studies, the risks of eczema vaccinatum ranged from 30.3 to 96.8 cases per million vaccinations, and were higher in infants <1 year, although not statistically significant.

Table 4 Risk of eczema vaccinatum (E.V.) and death from primary smallpox vaccinations, Stratified by age and study, 1963–1968

For all age groups combined, there were no deaths among 169 cases of eczema vaccinatum (CFR, 0 deaths per 100 cases; 95% CI, 0.0–2.2). However, among non-vaccinees that developed eczema vaccinatum after contact transmission from a recent vaccinee, 3 of 132 cases died (2.3 deaths per 100 cases; 95% CI, 0.5–6.5) [19, 22, 24, 25].

Generalized vaccinia

The risk of generalized vaccinia is displayed in Table 5. The summary risk of developing generalized vaccinia was highest among infants aged <1 year (103.3 cases per million vaccinations; 95% CI, 86.7–122.2). However, within age groups, the highest risks were recorded from the US studies that used physician surveys to ascertain complications not captured by other reporting mechanisms [22, 25]. The complication risks ranged from 133.3 to 1647 cases per million vaccinations and were highest in infants aged <1 year. The highest risks were reported from the study that prospectively surveyed physicians [25].

For all age groups combined, there were no deaths among 527 cases of generalized vaccinia (CFR, 0 deaths per 100 cases; 95% CI, 0.0–0.7).

Table 5 Risk of generalized vaccinia (G.V.) and accidental infection (A.I.) from primary smallpox vaccinations, Stratified by age and study, 1963–1968

Accidental infection (inadvertent autoinoculation)

The risk of accidental infection is displayed in Table 5. For all age groups combined, there were 857 cases of accidental infection from over 13 million vaccinations (CR, 64.9 cases per 1 million vaccinations; 95% CI, 60.6–69.4). The summary risks of accidental infection did not differ across age groups. However, within age groups, the highest risks were recorded from two US studies: the Lane study [25] that prospectively surveyed physicians and the Neff pediatric clinic study [20]. In the Lane study, the risk ranged from 371.2 to 2118 cases per million vaccinations, and was highest in infants aged <1 year. The Neff study reported 3 accidental infections from 517 vaccinations in the age group 1 to 4 years (CR, 5803 cases per million; 95% CI, 1198–16,863).

For all age groups combined, there were no deaths among 857 cases of accidental infection (CFR, 0 deaths per 100 cases; 95% CI, 0.0–0.4).

Re-vaccination

For smallpox vaccine re-vaccinations, the risks of developing specific complications, the risks of a specific complication death, and the individual risks of dying from specific complications by age group are displayed in Table 6. Because the risks were so low compared to primary vaccination, the risks were not stratified by age group. Compared to primary vaccinations, the summary risk of post-vaccinial encephalitis was 26 times lower in re-vaccinations, vaccinia necrosum was 1.5 times lower, eczema vaccinatum was 12 times lower, generalized vaccinia was 29 times lower, and accidental infection was 3.8 times lower. Only 2 deaths occurred from over 19.5 million re-vaccinations (MR, 0.1 deaths per million; 95% CI, 0.01–0.4). These two deaths occurred among 12 cases of vaccinia necrosum (CFR 16.7 deaths per 100 cases; 95% CI, 2.1–48.4).

Table 6 Risk of serious complications (S.C.) and death from smallpox re-vaccinations for ages greater than 1, 1963–1968

Discussion

To our knowledge, this is the first systematic review of the severe complications of smallpox vaccination in the USA. In addition to the age-specific complication and mortality risks, the case fatality risks were summarized. Using pooled summary measures, for every million primary vaccinations there were 60 cases of accidental infection, 40 cases generalized vaccinia, 13 cases of eczema vaccinatum, 3 cases of post-vaccinial encephalitis, and 1 case of vaccinia necrosum. However, complications for which vaccinia immune globulin (VIG) was not used (post-vaccinial encephalitis) or not generally required (accidental infection, generalized vaccinia) were significantly underreported because they were not captured by the VIG distribution program that was used by investigators to ascertain complications. In contrast, the US studies that used direct physician surveys [22, 25] reported significantly higher complication risks. For example, the number of complications per million primary vaccinations for all ages combined was 13.4 cases of post-vaccinial encephalitis [25], 263.4 cases of generalized vaccinia [25], and 838.9 cases of accidental infection [22].

The complications with the highest summary case fatality risks were post-vaccinial encephalitis (CFR, 28.9%) and vaccinia necrosum (CFR, 15.4%). There were no recorded deaths among vaccinees from other major complications, including eczema vaccinatum. Deaths from eczema vaccinatum were observed only among non-vaccinees that acquired the disease after contact to a recent vaccinee [38]. Neff et al. reported that contact-acquired eczema vaccinatum resulted in 3 (2.3%) deaths among the 132 cases in the US studies [41]. In contrast, in the United Kingdom (UK) studies contact-acquired eczema vaccinatum resulted in 9 (7.9%) deaths among 89 cases; however, the UK vaccinations occurred in the 1950s using a different vaccinia strain [29, 30].

Compared to primary vaccinees, in re-vaccinees the summary risk was 26-fold lower for post-vaccinial encephalitis, 29-fold lower for generalized vaccinia, 12-fold lower for eczema vaccinatum, 3.8-fold lower for accidental infection, and 1.5-fold lower for vaccinia necrosum. Prior vaccination predicted much lower risks for only 3 of the 5 major complications. Therefore, rather than the risks being significantly modified by prior vaccination status, the risk for accidental infection is more likely linked to behavior (autoinoculation) and the risk of vaccinia necrosum is more likely determined by immune status at the time of vaccination. This is consistent with the observation that vaccinia necrosum occurred almost exclusively in vaccinees with an severe cell-mediated immunodeficiency [42].

A strength of this systematic review is that the data were analyzed and summarized by age groups. The complication risks from each study, or combined, can be compared within and across age groups. For example, compared to vaccinees aged 1 year or older, infants aged <1 year had higher summary risks of post-vaccinial encephalitis (risk ratio, 2.80) and generalized vaccinia (risk ratio, 3.14). In contrast, compared to vaccinees aged <20 years, vaccinees aged 20 or older had a higher summary risk of vaccinia necrosum (risk ratio, 7.27).

There are several limitations to this systematic review. First, the studies reviewed were conducted in the 1960s, and routine smallpox vaccination in the United States ended in the early 1970s. Before routine smallpox vaccinations ceased, the characteristics of those persons vaccinated changed as the vaccination risks became better known and potential vaccinees were screened (e.g., infants aged <1 year, history of eczema). To the extent that routine screening for risk factors was occurring, this would have reduced the vaccine complication risks. Only the Ratner prospective cohort study that was conducted in 1967 was able to report vaccination screening criteria [21].

Second, the methods for ascertaining vaccine complications and deaths (i.e., the numerator in the risk estimates) differed between studies. The number of complications ascertained increased as the quantity and quality of ascertainment methods increased. For example, the studies that used direct physician surveys [22, 25] recorded more complications that were less severe (generalized vaccinia, accidental infection) or for which VIG was not used (post-vaccinial encephalitis). Therefore, the Lane et al. study [25] that used a prospective physician survey to enhance reporting yielded the highest (and likely the most accurate) vaccine complication risks for that study period. With respect to complications deaths, only 3 studies [19, 22, 24] reviewed death certificates for this outcome. So it is possible that the other studies underestimated deaths due to vaccine complications.

Third, in the Lane and Neff studies reviewed, the investigators were unable to verify the clinical diagnosis from the reports of adverse events. For example, generalized non-viremic rashes were sometimes misclassified as generalized vaccinia, possibly resulting in an overestimation of the risk of generalized vaccinia [17, 18].

Fourth, the methods for estimating the number of primary and repeat vaccinations (i.e., the denominator in risk estimates) differed between studies. Two studies had data on the actual number of vaccinations given [20, 21], and in the remaining studies the number of vaccinations were estimated from national or state immunization surveys [19, 22, 24, 25]. Because the complications evaluated for this review were infrequent, the complication risk estimates are more sensitive to variability in the numerator than in the denominator. Therefore, inaccuracies in the estimations of the populations vaccinated would not likely account for the observed heterogeneity in risk estimates.

Fifth, geographical and temporal factors may have influenced the heterogeneity of risk estimates. Pre-vaccination screening practices and the diagnosis and treatment of complications differed across regions and changed over time. Because these sources of bias likely contribute more to the heterogeneity of risk estimates than random error, the pooled summary measures presented may not be appropriate. For example, the Centers for Disease Control and Prevention (CDC) cite, but do not combine, risks estimates from the two Lane studies [24, 25] in their report on smallpox vaccination and adverse reactions [13] in order to provide a range of risk estimates. Based on this review, the CDC approach seems appropriate.

Finally, over the past 40 years many factors have changed that might have affected these studies and their interpretation, including the design and conduct of epidemiologic surveillance studies, the diagnosis and treatment of vaccine complications, and a change in the distributions of risk factors (e.g., atopic dermatitis, immunocompromised) among populations that might undergo vaccinations. In spite of all the study limitations, these are the best available US studies of sufficient size to estimate the complication and case fatality risks for each major complication among vaccinees.

This systematic review summarizes the major smallpox vaccine complication and case fatality risks for predicting the age-specific burden of vaccine complications in the event of widespread ring and mass vaccinations. Because atopic dermatitis and immune compromised states are more common today than when these studies were conducted [12, 43–45], higher vaccine complication rates may be observed in an outbreak. In comparison, in the current pre-event smallpox vaccination program, rigorous screening of almost 500,000 vaccinees has resulted in no cases of eczema vaccinatum and progressive vaccinia [46, 47]; however, very high rates of myopericarditis (about 1:1,700 in civilian vaccinees and 1 per 12,800 in military vaccinees) and possibly myocardial ischemia have been observed [48, 47, 49]. Both potential primary and re-vaccinees are at risk for myopericarditis. Similar to post-vaccinial encephalitis, there is no screening criteria to reduce complications rates. For these reasons, and because the long term effects of myopericarditis are not known, the ACIP has recommended a "pause" in the USA smallpox vaccination program until the risks and benefits can be studied more carefully [50]. Unless there is a smallpox outbreak, at the current time, the individual risks of pre-event smallpox vaccination outweigh the potential benefits.