To our knowledge, this is the first systematic review of the severe complications of smallpox vaccination in the USA. In addition to the age-specific complication and mortality risks, the case fatality risks were summarized. Using pooled summary measures, for every million primary vaccinations there were 60 cases of accidental infection, 40 cases generalized vaccinia, 13 cases of eczema vaccinatum, 3 cases of post-vaccinial encephalitis, and 1 case of vaccinia necrosum. However, complications for which vaccinia immune globulin (VIG) was not used (post-vaccinial encephalitis) or not generally required (accidental infection, generalized vaccinia) were significantly underreported because they were not captured by the VIG distribution program that was used by investigators to ascertain complications. In contrast, the US studies that used direct physician surveys [22, 25] reported significantly higher complication risks. For example, the number of complications per million primary vaccinations for all ages combined was 13.4 cases of post-vaccinial encephalitis , 263.4 cases of generalized vaccinia , and 838.9 cases of accidental infection .
The complications with the highest summary case fatality risks were post-vaccinial encephalitis (CFR, 28.9%) and vaccinia necrosum (CFR, 15.4%). There were no recorded deaths among vaccinees from other major complications, including eczema vaccinatum. Deaths from eczema vaccinatum were observed only among non-vaccinees that acquired the disease after contact to a recent vaccinee . Neff et al. reported that contact-acquired eczema vaccinatum resulted in 3 (2.3%) deaths among the 132 cases in the US studies . In contrast, in the United Kingdom (UK) studies contact-acquired eczema vaccinatum resulted in 9 (7.9%) deaths among 89 cases; however, the UK vaccinations occurred in the 1950s using a different vaccinia strain [29, 30].
Compared to primary vaccinees, in re-vaccinees the summary risk was 26-fold lower for post-vaccinial encephalitis, 29-fold lower for generalized vaccinia, 12-fold lower for eczema vaccinatum, 3.8-fold lower for accidental infection, and 1.5-fold lower for vaccinia necrosum. Prior vaccination predicted much lower risks for only 3 of the 5 major complications. Therefore, rather than the risks being significantly modified by prior vaccination status, the risk for accidental infection is more likely linked to behavior (autoinoculation) and the risk of vaccinia necrosum is more likely determined by immune status at the time of vaccination. This is consistent with the observation that vaccinia necrosum occurred almost exclusively in vaccinees with an severe cell-mediated immunodeficiency .
A strength of this systematic review is that the data were analyzed and summarized by age groups. The complication risks from each study, or combined, can be compared within and across age groups. For example, compared to vaccinees aged 1 year or older, infants aged <1 year had higher summary risks of post-vaccinial encephalitis (risk ratio, 2.80) and generalized vaccinia (risk ratio, 3.14). In contrast, compared to vaccinees aged <20 years, vaccinees aged 20 or older had a higher summary risk of vaccinia necrosum (risk ratio, 7.27).
There are several limitations to this systematic review. First, the studies reviewed were conducted in the 1960s, and routine smallpox vaccination in the United States ended in the early 1970s. Before routine smallpox vaccinations ceased, the characteristics of those persons vaccinated changed as the vaccination risks became better known and potential vaccinees were screened (e.g., infants aged <1 year, history of eczema). To the extent that routine screening for risk factors was occurring, this would have reduced the vaccine complication risks. Only the Ratner prospective cohort study that was conducted in 1967 was able to report vaccination screening criteria .
Second, the methods for ascertaining vaccine complications and deaths (i.e., the numerator in the risk estimates) differed between studies. The number of complications ascertained increased as the quantity and quality of ascertainment methods increased. For example, the studies that used direct physician surveys [22, 25] recorded more complications that were less severe (generalized vaccinia, accidental infection) or for which VIG was not used (post-vaccinial encephalitis). Therefore, the Lane et al. study  that used a prospective physician survey to enhance reporting yielded the highest (and likely the most accurate) vaccine complication risks for that study period. With respect to complications deaths, only 3 studies [19, 22, 24] reviewed death certificates for this outcome. So it is possible that the other studies underestimated deaths due to vaccine complications.
Third, in the Lane and Neff studies reviewed, the investigators were unable to verify the clinical diagnosis from the reports of adverse events. For example, generalized non-viremic rashes were sometimes misclassified as generalized vaccinia, possibly resulting in an overestimation of the risk of generalized vaccinia [17, 18].
Fourth, the methods for estimating the number of primary and repeat vaccinations (i.e., the denominator in risk estimates) differed between studies. Two studies had data on the actual number of vaccinations given [20, 21], and in the remaining studies the number of vaccinations were estimated from national or state immunization surveys [19, 22, 24, 25]. Because the complications evaluated for this review were infrequent, the complication risk estimates are more sensitive to variability in the numerator than in the denominator. Therefore, inaccuracies in the estimations of the populations vaccinated would not likely account for the observed heterogeneity in risk estimates.
Fifth, geographical and temporal factors may have influenced the heterogeneity of risk estimates. Pre-vaccination screening practices and the diagnosis and treatment of complications differed across regions and changed over time. Because these sources of bias likely contribute more to the heterogeneity of risk estimates than random error, the pooled summary measures presented may not be appropriate. For example, the Centers for Disease Control and Prevention (CDC) cite, but do not combine, risks estimates from the two Lane studies [24, 25] in their report on smallpox vaccination and adverse reactions  in order to provide a range of risk estimates. Based on this review, the CDC approach seems appropriate.
Finally, over the past 40 years many factors have changed that might have affected these studies and their interpretation, including the design and conduct of epidemiologic surveillance studies, the diagnosis and treatment of vaccine complications, and a change in the distributions of risk factors (e.g., atopic dermatitis, immunocompromised) among populations that might undergo vaccinations. In spite of all the study limitations, these are the best available US studies of sufficient size to estimate the complication and case fatality risks for each major complication among vaccinees.
This systematic review summarizes the major smallpox vaccine complication and case fatality risks for predicting the age-specific burden of vaccine complications in the event of widespread ring and mass vaccinations. Because atopic dermatitis and immune compromised states are more common today than when these studies were conducted [12, 43–45], higher vaccine complication rates may be observed in an outbreak. In comparison, in the current pre-event smallpox vaccination program, rigorous screening of almost 500,000 vaccinees has resulted in no cases of eczema vaccinatum and progressive vaccinia [46, 47]; however, very high rates of myopericarditis (about 1:1,700 in civilian vaccinees and 1 per 12,800 in military vaccinees) and possibly myocardial ischemia have been observed [48, 47, 49]. Both potential primary and re-vaccinees are at risk for myopericarditis. Similar to post-vaccinial encephalitis, there is no screening criteria to reduce complications rates. For these reasons, and because the long term effects of myopericarditis are not known, the ACIP has recommended a "pause" in the USA smallpox vaccination program until the risks and benefits can be studied more carefully . Unless there is a smallpox outbreak, at the current time, the individual risks of pre-event smallpox vaccination outweigh the potential benefits.