In the United States (US) routine smallpox vaccination of the general population ended in 1972 [1]. The last known case of naturally-acquired smallpox occurred in Somalia in 1977, and in 1980 the World Health Organization (WHO) declared that smallpox had been eradicated from the world [2]. Today, most of the US and the world's population is considered susceptible to smallpox infection. Since the September 2001 airplane attacks on the World Trade Center and the anthrax spore attacks using the US mail, the US has taken several steps to mitigate a possible intentional release of variola major (smallpox) virus [3–6], including re-instituting a national smallpox vaccination program [7]. For Phase 1 of this program, the Advisory Committee on Immunization Practices (ACIP) recommended voluntary vaccination of 500,000 members of public health response and hospital health care teams [8]. In Phase 2, up to 10 million first responders such as police, fire, paramedics, and other health care staff would be offered voluntary smallpox vaccination.

However, the implementation the smallpox vaccination program is not without controversy [9]. Vaccinating individuals against smallpox infection, even voluntarily, means exposing vaccinees and their contacts to the risks of serious adverse reactions without the clear benefit of protection against an infection which does not exist and may never exist again. Volunteers must balance their individual risks and benefits of smallpox vaccination. The individual risks and benefits from being vaccinated before an intentional release depends on the probability of an adverse vaccine reaction, the probability that smallpox virus will be released, and the probability that an individual would be directly exposed to a release or an unrecognized case of smallpox [10]. Once an outbreak occurs, ring and mass vaccination would result in a large number of primary vaccinees of all ages, including some with underlying or unrecognized risks for vaccine adverse reactions. In this scenario, we would expect to see an increased number and risk of vaccine complications. Estimation of the vaccine complication risks by age, including their precision and estimation biases, are necessary to formulate policies and inform individual choice regarding the risks and benefits of pre-outbreak smallpox vaccination, and to estimate the burden of vaccine complications from ring and mass vaccination during an smallpox outbreak in the general population.

Recent publications [11–16] have summarized or cited the complication risks associated with receiving smallpox vaccination; however, to our knowledge, no systematic review of this older literature has been conducted. The purpose of this systematic review is to review and summarize the age-specific complication and case fatality risks of post-vaccinial encephalitis, vaccinia necrosum (progressive vaccinia), eczema vaccinatum, generalized vaccinia, and accidental infection (inadvertant autoinoculation) from smallpox vaccinations conducted in the US. We restricted our review to US studies because the New York City Board of Health vaccinia strain, which was the only strain used in the US, is the strain in current use by the the national vaccination program (and will be in the future) [11].

Our search yielded a total of 348 studies: 342 from the MEDLINE search, 2 from the reference lists review [27, 28], 2 from author communications [29, 30], and 2 from our files [27, 31]. From this pool, 18 articles of smallpox vaccination complications were identified for further review. Using our selection criteria we excluded 11 studies: 3 did not use the Neff criteria [31–33], 3 did not report complications by vaccination status [34–36], 3 did not report the total number of vaccinations by vaccination status [27, 37, 38], and 2 studies' results [39, 40] were reported in another article [25]. The remaining 7 articles form the basis of this systematic review and represent data from approximately 13,206,095 primary vaccinations and 18,349,359 re-vaccinations [19–25] (Table 1).

The US studies covered the years 1963, 1967, and 1968. Five of the 7 studies were period analyses [19, 22–25]. For a period analysis, the number of vaccine complications for a selected geographic location and period of time were ascertained from a variety of sources (Table 1, columns 6 and 7). The at-risk population was the number of persons vaccinated in that location during the same time period. This number was estimated from a variety of sources including population-based regional and national surveys (Table 1, column 8). One study was a prospective cohort study [21] and the other was a clinic-based retrospective cohort study [20].

To our knowledge, this is the first systematic review of the severe complications of smallpox vaccination in the USA. In addition to the age-specific complication and mortality risks, the case fatality risks were summarized. Using pooled summary measures, for every million primary vaccinations there were 60 cases of accidental infection, 40 cases generalized vaccinia, 13 cases of eczema vaccinatum, 3 cases of post-vaccinial encephalitis, and 1 case of vaccinia necrosum. However, complications for which vaccinia immune globulin (VIG) was not used (post-vaccinial encephalitis) or not generally required (accidental infection, generalized vaccinia) were significantly underreported because they were not captured by the VIG distribution program that was used by investigators to ascertain complications. In contrast, the US studies that used direct physician surveys [22, 25] reported significantly higher complication risks. For example, the number of complications per million primary vaccinations for all ages combined was 13.4 cases of post-vaccinial encephalitis [25], 263.4 cases of generalized vaccinia [25], and 838.9 cases of accidental infection [22].

The complications with the highest summary case fatality risks were post-vaccinial encephalitis (CFR, 28.9%) and vaccinia necrosum (CFR, 15.4%). There were no recorded deaths among vaccinees from other major complications, including eczema vaccinatum. Deaths from eczema vaccinatum were observed only among non-vaccinees that acquired the disease after contact to a recent vaccinee [38]. Neff et al. reported that contact-acquired eczema vaccinatum resulted in 3 (2.3%) deaths among the 132 cases in the US studies [41]. In contrast, in the United Kingdom (UK) studies contact-acquired eczema vaccinatum resulted in 9 (7.9%) deaths among 89 cases; however, the UK vaccinations occurred in the 1950s using a different vaccinia strain [29, 30].

Compared to primary vaccinees, in re-vaccinees the summary risk was 26-fold lower for post-vaccinial encephalitis, 29-fold lower for generalized vaccinia, 12-fold lower for eczema vaccinatum, 3.8-fold lower for accidental infection, and 1.5-fold lower for vaccinia necrosum. Prior vaccination predicted much lower risks for only 3 of the 5 major complications. Therefore, rather than the risks being significantly modified by prior vaccination status, the risk for accidental infection is more likely linked to behavior (autoinoculation) and the risk of vaccinia necrosum is more likely determined by immune status at the time of vaccination. This is consistent with the observation that vaccinia necrosum occurred almost exclusively in vaccinees with an severe cell-mediated immunodeficiency [42].

A strength of this systematic review is that the data were analyzed and summarized by age groups. The complication risks from each study, or combined, can be compared within and across age groups. For example, compared to vaccinees aged 1 year or older, infants aged <1 year had higher summary risks of post-vaccinial encephalitis (risk ratio, 2.80) and generalized vaccinia (risk ratio, 3.14). In contrast, compared to vaccinees aged <20 years, vaccinees aged 20 or older had a higher summary risk of vaccinia necrosum (risk ratio, 7.27).

There are several limitations to this systematic review. First, the studies reviewed were conducted in the 1960s, and routine smallpox vaccination in the United States ended in the early 1970s. Before routine smallpox vaccinations ceased, the characteristics of those persons vaccinated changed as the vaccination risks became better known and potential vaccinees were screened (e.g., infants aged <1 year, history of eczema). To the extent that routine screening for risk factors was occurring, this would have reduced the vaccine complication risks. Only the Ratner prospective cohort study that was conducted in 1967 was able to report vaccination screening criteria [21].

Second, the methods for ascertaining vaccine complications and deaths (i.e., the numerator in the risk estimates) differed between studies. The number of complications ascertained increased as the quantity and quality of ascertainment methods increased. For example, the studies that used direct physician surveys [22, 25] recorded more complications that were less severe (generalized vaccinia, accidental infection) or for which VIG was not used (post-vaccinial encephalitis). Therefore, the Lane et al. study [25] that used a prospective physician survey to enhance reporting yielded the highest (and likely the most accurate) vaccine complication risks for that study period. With respect to complications deaths, only 3 studies [19, 22, 24] reviewed death certificates for this outcome. So it is possible that the other studies underestimated deaths due to vaccine complications.

Third, in the Lane and Neff studies reviewed, the investigators were unable to verify the clinical diagnosis from the reports of adverse events. For example, generalized non-viremic rashes were sometimes misclassified as generalized vaccinia, possibly resulting in an overestimation of the risk of generalized vaccinia [17, 18].

Fourth, the methods for estimating the number of primary and repeat vaccinations (i.e., the denominator in risk estimates) differed between studies. Two studies had data on the actual number of vaccinations given [20, 21], and in the remaining studies the number of vaccinations were estimated from national or state immunization surveys [19, 22, 24, 25]. Because the complications evaluated for this review were infrequent, the complication risk estimates are more sensitive to variability in the numerator than in the denominator. Therefore, inaccuracies in the estimations of the populations vaccinated would not likely account for the observed heterogeneity in risk estimates.

Fifth, geographical and temporal factors may have influenced the heterogeneity of risk estimates. Pre-vaccination screening practices and the diagnosis and treatment of complications differed across regions and changed over time. Because these sources of bias likely contribute more to the heterogeneity of risk estimates than random error, the pooled summary measures presented may not be appropriate. For example, the Centers for Disease Control and Prevention (CDC) cite, but do not combine, risks estimates from the two Lane studies [24, 25] in their report on smallpox vaccination and adverse reactions [13] in order to provide a range of risk estimates. Based on this review, the CDC approach seems appropriate.

Finally, over the past 40 years many factors have changed that might have affected these studies and their interpretation, including the design and conduct of epidemiologic surveillance studies, the diagnosis and treatment of vaccine complications, and a change in the distributions of risk factors (e.g., atopic dermatitis, immunocompromised) among populations that might undergo vaccinations. In spite of all the study limitations, these are the best available US studies of sufficient size to estimate the complication and case fatality risks for each major complication among vaccinees.

This systematic review summarizes the major smallpox vaccine complication and case fatality risks for predicting the age-specific burden of vaccine complications in the event of widespread ring and mass vaccinations. Because atopic dermatitis and immune compromised states are more common today than when these studies were conducted [12, 43–45], higher vaccine complication rates may be observed in an outbreak. In comparison, in the current pre-event smallpox vaccination program, rigorous screening of almost 500,000 vaccinees has resulted in no cases of eczema vaccinatum and progressive vaccinia [46, 47]; however, very high rates of myopericarditis (about 1:1,700 in civilian vaccinees and 1 per 12,800 in military vaccinees) and possibly myocardial ischemia have been observed [48, 47, 49]. Both potential primary and re-vaccinees are at risk for myopericarditis. Similar to post-vaccinial encephalitis, there is no screening criteria to reduce complications rates. For these reasons, and because the long term effects of myopericarditis are not known, the ACIP has recommended a "pause" in the USA smallpox vaccination program until the risks and benefits can be studied more carefully [50]. Unless there is a smallpox outbreak, at the current time, the individual risks of pre-event smallpox vaccination outweigh the potential benefits.