Skip to main content

Advertisement

You are viewing the new article page. Let us know what you think. Return to old version

Invasive meningococcal disease epidemiology and control measures: a framework for evaluation

BMC Public Healthvolume 7, Article number: 130 (2007) | Download Citation

Abstract

Background

Meningococcal disease can have devastating consequences. As new vaccines emerge, it is necessary to assess their impact on public health. In the absence of long-term real world data, modeling the effects of different vaccination strategies is required. Discrete event simulation provides a flexible platform with which to conduct such evaluations.

Methods

A discrete event simulation of the epidemiology of invasive meningococcal disease was developed to quantify the potential impact of implementing routine vaccination of adolescents in the United States with a quadrivalent conjugate vaccine protecting against serogroups A, C, Y, and W-135. The impact of vaccination is assessed including both the direct effects on individuals vaccinated and the indirect effects resulting from herd immunity. The simulation integrates a variety of epidemiologic and demographic data, with core information on the incidence of invasive meningococcal disease and outbreak frequency derived from data available through the Centers for Disease Control and Prevention. Simulation of the potential indirect benefits of vaccination resulting from herd immunity draw on data from the United Kingdom, where routine vaccination with a conjugate vaccine has been in place for a number of years. Cases of disease are modeled along with their health consequences, as are the occurrence of disease outbreaks.

Results

When run without a strategy of routine immunization, the simulation accurately predicts the age-specific incidence of invasive meningococcal disease and the site-specific frequency of outbreaks in the Unite States. 2,807 cases are predicted annually, resulting in over 14,000 potential life years lost due to invasive disease. In base case analyses of routine vaccination, life years lost due to infection are reduced by over 45% (to 7,600) when routinely vaccinating adolescents 12 years of age at 70% coverage. Sensitivity analyses indicate that herd immunity plays an important role when this population is targeted for vaccination. While 1,100 cases are avoided annually when herd immunity effects are included, in the absence of any herd immunity, the number of cases avoided with routine vaccination falls to 380 annually. The duration of vaccine protection also strongly influences results.

Conclusion

In the absence of appropriate real world data on outcomes associated with large-scale vaccination programs, decisions on optimal immunization strategies can be aided by discrete events simulations such as the one described here. Given the importance of herd immunity on outcomes associated with routine vaccination, published estimates of the economic efficiency of routine vaccination with a quadrivalent conjugate vaccine in the United States may have considerably underestimated the benefits associated with a policy of routine immunization of adolescents.

Open Peer Review reports

Background

Invasive meningococcal disease, although relatively rare, can have devastating consequences, with case-fatality rates over 10% [1–3] and rates of permanent sequelae amongst survivors, including neurological complications, limb loss, mental retardation, hearing loss, renal failure and paralysis in up to 20% of reported cases [2–5].

Most cases of meningitis in industrialized countries are due to endemic disease. In the US, for example, endemic cases make up approximately 97% of cases [1]. Nevertheless, sporadic outbreaks occur [1, 6], often leading to considerable anxiety in the affected population and to costly outbreak control measures. Between 1994 and 2002, there were approximately 75 outbreaks across the US [6]. The overall incidence of meningococcal disease varied from year to year, but was roughly 1 to 2 cases per 100 000 individuals per year, with the great majority of cases due to serogroups B, C, Y and W-135 [1]. Serogroup A disease, though more common in the past, is now relatively rare in industrialized countries [1].

Until recently, routine vaccination for meningococcal disease had not been recommended in the US, except for individuals entering college [7], but vaccination during outbreaks was common [6]. A quadrivalent polysaccharide vaccine, protective against serogroups A, C, Y and W-135 had been the only vaccine used. That vaccine, however, has relatively poor efficacy in younger children and does not provide long-lasting protection [8]. There have also been suggestions that the effectiveness of polysaccharide meningococcal vaccines against colonization may be weak and short-lived [9]. New conjugate vaccines are expected to offer better protection [8, 9] and may also reduce carriage of meningococci [10], allowing for the possibility of indirect benefits of vaccination brought about by herd immunity [11] In the UK, routine vaccination of children with a conjugate C vaccine has been implemented and has been successful [10–12] 11. Menactra®, a quadrivalent conjugate vaccine was approved for use in the US in early 2005, and the Centre for Disease Control's Advisory Committee on Immunization Practices has now recommended routine vaccination of young adolescents and college students [13].

Given the relatively low incidence of disease, a means of assessing outcomes under various vaccination strategies is necessary for stakeholders to make informed decisions on vaccinating with the newer, more effective vaccines. While a number of epidemiological and health economic models of varying complexity have been developed [14–23], we propose a new approach to the modelling of meningococcal disease: an individual person model using discrete event simulation techniques [24, 25].

The simulation presented here was designed to provide decision makers with insight into the direct effects of immunization in vaccinated individuals and the potential indirect effects, often referred to as herd immunity, resulting from reduced transmission, both in terms of incidence and of outbreaks. In the United States, economic evaluations of vaccination for meningococcal vaccine have focused exclusively on the direct effects of immunization on vaccinated individuals [19, 26, 27]. This narrow focus could mean that these evaluations, regardless of the quality of their economic data, significantly underestimate the benefits and cost offsets from reduced disease resulting from routine vaccination. Although there remain significant data gaps, one of the goals in developing this simulation was to assess the impact of herd immunity on the incidence of disease and to guide future research by allowing for evaluation of the importance of key epidemiologic inputs and assumptions. To our knowledge, previous models have not attempted to capture the population-wide direct and indirect effects of vaccination on both incidence of disease and on outbreaks.

This paper provides a detailed description of the discrete event simulation developed to evaluate the epidemiology of invasive meningococcal disease in the United States, and the resulting predictions of the potential impact of routinely vaccinating adolescents with a quadrivalent conjugate vaccine. As the simulation presented here uses a novel approach to modelling meningococcal disease, and is the first to assess the impact of the direct and indirect effects of vaccination on disease incidence as well as the occurrence and extent of meningitis outbreaks, testing the validity and robustness of results is vital. As such, after describing the model's structure and the vaccination strategies considered, we detail the method used for testing its robustness and ensuring its validity.

Methods

The model simulates the experience of dynamic communities of individuals over a specified time horizon and under various epidemiological conditions. In this paper, we simulate US communities with starting populations of 200,000 individuals. Reported results are scaled up to reflect outcomes for the entire US population. Epidemiological, demographic, and health utility data used to populate the simulation are detailed in Appendix 1.

Each simulated community consists of a number of primary schools, high schools and one college. Schools are included in the analysis as epidemiological evidence [7, 28, 29] indicates that they are a common location for clusters of disease. Individuals are assigned specific characteristics such as age, gender and educational status and the evolution of their status regarding meningococcal disease and vaccination is tracked on a daily basis (Figure 1).

Figure 1
figure1

Schematic representation of the simulation. Individuals are created and follow the flow-chart on a daily basis.

Full size image

When a case occurs, the health effects, and impact on quality of life are calculated over the lifetime of the infected individual. At the same time, school-contacts and household contacts for that case are identified according to the distribution of household and school sizes in the community. These contacts are then exposed for a limited period of time to an increased risk of developing meningococcal disease.

The model determines whether an outbreak, either in the community or in a specific school, is declared. Outbreaks are defined on the basis of the number of cases of the same serogroup occurring within a three-month period: two or more in the same school for school outbreaks or ten or more per 100,000 individuals for community outbreaks [6, 30]. In order to avoid double counting, school outbreaks occurring during a community outbreak, were considered to be part of the community outbreak. If an outbreak is triggered in the simulation each individual may be vaccinated as a result of outbreak control interventions by public health authorities.

Given the complex nature of outbreaks [31, 32], we introduce high-risk periods at a rate calibrated to fit observed outbreak data in terms of frequency, serogroup distribution and scale (i.e., community or school outbreak). During these periods, all individuals are exposed to a risk of infection that is higher than the baseline endemic risk. Details of the method used to integrate these high-risk periods into the simulation are provided in Appendix 2.

In addition to vaccination during outbreaks, the model allows for the introduction of routine vaccination for specified subgroups of the population. When individuals are vaccinated, either through a routine vaccination program or as a result of outbreak control measures, they attain a degree of protection against invasive meningococcal disease for a defined period of time. The number of individuals vaccinated and directly protected against carriage of Neisseria meningitidis can also affect the overall risk for the entire population.

To estimate the impact of vaccination on Neisseria meningitidis transmission, we assumed that the vaccine used would be the one currently licensed for use in the US: a conjugate meningococcal vaccine protecting against serotypes A, C, Y and W-135. In the absence of long-term efficacy data for this vaccine, we extrapolated long-term protection from UK effectiveness data [12] for a C conjugate meningococcal vaccine and assumed that the quadrivalent vaccine was equally effective against all serogroups. The mean duration of direct protection against meningococcal infection under these assumptions is 30.1 years (Figure 2).

Figure 2
figure2

Estimated effectiveness of vaccination with conjugate meningococcal disease vaccine over time.

Full size image

The meningococcal C vaccination campaign in the UK [11] demonstrated that benefits accrue not only to vaccinated individuals, but also to the entire population. In order to capture these indirect effects, we assumed that vaccinating individuals with a conjugate vaccine decreases the probability that they are carriers and can transmit the disease. Consistent with a recent epidemiological model of the impact of meningococcal vaccination in the UK [20], individuals in the same age group were assumed to mix more frequently than individuals across age groups. Inputs were calibrated to reproduce the outcome observed in the UK that with vaccination of 81% of individuals under the age of 20 years, disease incidence fell by 67% in unvaccinated individuals in the same age group and 35% in adults [11]. The method used for calculating indirect effects is described in greater detail in Appendix 2.

Simulation settings

Analyses are run for routine vaccination with a quadrivalent conjugate vaccine of 12-year olds and compared to simulations with no routine vaccination. The analyses assume 71% coverage amongst 12 years olds [19] attained in the first year of the program and maintained throughout its duration.

In both strategies, outbreaks of disease due to vaccine-preventable serogroups are assumed to lead to outbreak immunization campaigns. For outbreaks in primary and secondary schools, 100% coverage iss assumed, while for those in colleges, 90% is used. For community-based outbreaks, the coverage is set at 90% for children and adolescents and 70% for those of college age [33, 34].

Simulations are run over a 100 year time horizon, but results are presented for years when all people between the ages of 12 and 100 years have either been vaccinated or were eligible for vaccination as part of a routine vaccination program – that is, when maximum vaccine penetration has been reached (referred to as steady state).

Sensitivity analyses

The novelty of the model presented creates a need for testing its validity and robustness. In terms of validity, the best approach would be to run the model with data observed in a given setting or community and then testing whether the model is able to reproduce data observed in another setting or community The nature of data available, however, makes such an approach unfeasible. In order to ensure that the model reproduce correctly what is known about transmission of meningococcal disease and outbreaks, we developed calibration procedures aimed at reproducing at best available data for the three key vaccination-related parameters in the model: duration of vaccine protection, indirect effects of vaccination, and outbreak occurrence and scale. These calibration procedures are detailed in Appendix 2.

Testing the robustness of simulated findings is driven by the identification of key parameters of influence. This was accomplished by performing sensitivity analyses on different features and input data of the model. These analyses were performed on population size, relative risk of contracting the disease when coming into contact with an active case, herd immunity, vaccine efficacy, vaccine coverage rates, and baseline incidence.

We assessed the contribution of the relative risk in contact contacts by running analyses with no increased risk to contacts, and doubling the baseline relative risk in contacts. The impact of herd immunity was varied from 0% (no herd immunity) to 150% (50% greater indirect effect compared to base case). The rate of waning for vaccine efficacy was doubled and halved leading to an estimated average duration of direct protection from vaccination of between 15 and 60 years. Coverage rates for routine vaccination were varied from 50% to 90% for routine vaccination, and baseline incidence was varied by 25% in either direction. Finally, the impact of the size of the community on model predictions was tested by considering a larger community size than used in the base case (500,000 individuals instead of 200,000). For this parameter, however, we proceeded differently than for the other sensitivity analyses, as a specific calibration procedure on the outbreak rate was performed. Given that the community size parameter is somewhat arbitrary, it is important that the impact of the variable be controlled by recalibrated parameter values.

Results

In the absence of routine vaccination, the simulation closely reproduces the incidence of disease and outbreaks observed in the US, with an estimated 2,807 cases per year, 375 of which are fatal and 471 lead to permanent disability (Table 1). This leads to a loss of more than 14,000 life years and of almost 17,000 quality adjusted life years (QALYs). Predictions on incidence, deaths related to meningococcal disease, and outbreaks accurately reproduce the known epidemiology of meningococcal disease in the US in the years prior to the recommendation to routinely vaccinate adolescents.

Table 1 Steady state summary results from simulations comparing routine vaccination of 12-year-olds to no routine vaccination, under base case assumptions and for sensitivity analyses on coverage rates, low vaccine efficacy (doubling waning of vaccine effectiveness) and halving herd immunity
Full size table

Routine vaccination of 12 year olds quickly leads to a substantial reduction in the number of cases of meningococcal disease. Assuming immediate coverage of 71%, the incidence of vaccine-preventable disease falls by 35% after 10 years, from 0.62 to 0.41 per 100,000 person years (Figure 3). With routine vaccination simulations indicate a marked drop in cases amongst targeted age groups (Figure 4), but benefits decrease as vaccinated individuals age and protection conferred through immunization wanes. Figure 4 also illustrates the indirect benefits of vaccination, as evidenced by the gap that opens up with routine vaccination across all age groups, including in those younger than 12 years. Direct benefits of vaccination obviously result in no change in incidence below the age of 12, and a much smaller decline in adult cases.

Figure 3
figure3

Incidence of meningococcal disease (total and vaccine preventable) over 100 years following initiation of routine vaccination of 12 year olds.

Full size image
Figure 4
figure4

Estimated number of cases with and without routine vaccination of adolescents at steady state†. The dashed line ('Only direct effects') represents the estimated number of cases with vaccination, but assuming no herd immunity. † Steady state refers to years when all individuals in the simulation between the ages of 12 and 100 years have been eligible for vaccination under the routine program.

Full size image

By steady state, routine vaccination of 12 year olds reduces the average annual number of cases by over 1,000, a decrease of 39% in all cases and 62% in vaccine-preventable cases (Table 1). There are 189 fewer deaths and 190 fewer cases resulting in permanent disability, leading to a 46% reduction in life years lost (from 14,151 to 7,616), and 43% reduction in QALYs lost (from 16,852 to 9,626). Under base case assumptions, just over 2,500 individuals would need to be vaccinated to avoid a case, and almost 15,000 to avoid one death due to meningococcal disease. In addition to a reduction in the number of cases, the simulation predicts a 74% lower occurrence of outbreaks with routine vaccination (from 9.1 to 2.4), and 97% fewer outbreaks caused by vaccine-preventable serogroups (from 6.9 to 0.2).

Sensitivity analyses

Results presented in Figures 5 and 6 allow for identification of those parameters that have a strong influence on the predicted impact of vaccination using the simulation. Not surprisingly, both the direct effects of vaccination (vaccine efficacy) and the indirect effects (herd immunity) have a strong impact on both the number of cases and on the number of outbreaks prevented by vaccination. Without herd immunity, the number of cases avoided with routine immunization drops avoided from 1,100 per year to 380 per year. The number of outbreaks prevented falls from 6.7 to about 5. Reducing the duration of the protection conferred by the vaccine by 50% (low vaccine efficacy) leads to a reduction in cases compared to no routine vaccination of 24%. Using the base case estimate of duration of protection, a 39% reduction is predicted. The impact of the high (90%) and low (50%) vaccination coverage rates also influence predictions, but not nearly as great as when direct and indirect effectiveness of the vaccine is considered. Table 1 provides more detailed results on outcomes when varying coverage rates, and reducing the direct and indirect effectiveness of the vaccine.

Figure 5
figure5

Cases avoided per year with routine vaccination of adolescents under different scenarios at steady state. Larger community size – 500,000 individuals. No/high contact relative risk – 0%/200% of base case relative risk. Low/high incidence – 75%/125% of base case incidence. Low/high vaccine efficacy – doubling/cutting in half the rate of decay for vaccine effectiveness. No/Low/High herd immunity – 0%/50%/150% of base case herd immunity. 50%/90% coverage – coverage rate for routine vaccination.

Full size image
Figure 6
figure6

Outbreaks avoided per year with routine vaccination under different scenarios at steady state. Larger community size – 500,000 individuals. No/high contact relative risk – 0%/200% of base case relative risk. Low/high incidence – 75%/125% of base case incidence. Low/high vaccine efficacy – doubling/cutting in half the rate of decay for vaccine effectiveness. No/Low/High herd immunity – 0%/50%/150% of base case herd immunity. 50%/90% coverage – coverage rate for routine vaccination.

Full size image

Other parameters have a much more limited influence on predictions. Varying incidence changes absolute number of cases with (2,124 versus 1,717 in the base case) and without (3,517 versus 2,807 in the base case) routine vaccination but has a relatively small effect on net outcomes. Similarly, although modification of the relative risk of exposed contacts has a large effect on the frequency of outbreaks in the absence of vaccination (+62% for the high relative risk scenario and -90% for the no increased risk scenario), the percent reduction in outbreaks resulting from vaccination changes very little.

After calibration for outbreak related parameters, analyses using a larger community size indicate that this parameter has only a very limited influence on the reduction of cases and outbreaks. It should be noted, however, that calibration of outbreak related parameters limit the potential influence of changes in community size. Analyses indicate that increasing the size of communities to 500,000 individuals results in small differences not only in the reduction of cases and outbreaks resulting from vaccination, but also on the absolute number of cases across the US (1,724 cases with routine vaccination versus 1,717 cases with routine vaccination in the base case).

Discussion

Results from the simulations suggest that routine immunization of an adequately large subset of the population with an effective quadrivalent vaccine can lead to substantial reductions in cases of disease. Furthermore, the indirect benefits associated with herd immunity seem to be just as important as the direct benefits conferred upon vaccinees.

Under base case assumptions, routine immunization of 12 year olds in the US is predicted to reduce the incidence of invasive disease by almost 40%, and vaccine-preventable disease by over 60%. Outbreaks of invasive meningococcal disease from serogroups A, C, Y, and W-135 are all but eliminated. Sensitivity analyses clearly demonstrate the importance of herd immunity and duration of immunity after vaccination in determining the long-term impact of routine vaccination policies.

By providing a framework for the evaluation of strategies to control invasive meningococcal disease which takes into consideration the direct and indirect benefits of immunization, the short and long-term consequences of the disease, and the impact of outbreaks at both the institutional and community level, the simulation developed for these analyses provides a more complete description of the potential impact of meningococcal vaccination than the simpler cohort models used in the past to evaluate the economic efficiency of routine vaccination in the US for meningococcal disease [19, 26, 27]. An important next step using the results of these simulations will be to incorporate cost and resource use data on managing infections, vaccination activities and outbreak control, in order to re-evaluate the cost-effectiveness of instituting routine vaccination of 12 year olds. Given that benefits resulting from herd immunity and outbreak frequency reduction were not considered in the most recent estimate put forth by the CDC [19], one would expect that the cost-effectiveness of routine vaccination of US adolescents with a quadrivalent conjugate vaccine would be considerably lower than their ratio of $121,000 per life year gained.

Modelling the epidemiology of infectious diseases can be complex, but is necessary for reasonable evaluation of infection control strategies [25]. Although still relatively uncommon in epidemiology, discrete event simulations are increasingly used; a recent example being an assessment of smallpox management strategies in the US [35]. Discrete event simulation is a useful technique when modelling interaction between individuals in a population. The simulation is specified and run at the level of the individual, with each person assigned specific characteristics which are carried in the model and updated periodically. This simulation technique also allows the events occurring in individuals to influence outcomes in other individuals, by specifying interaction between them and the consequences of this interaction [24].

As with any model, one of the main challenges rests with limited data availability. Often, available data alone are not sufficient, requiring calibration procedures to be performed in order to ensure that the model best represents what is known about the disease and its prevention. In the simulation described here, calibration was necessary for three key parameters: duration of protection, herd immunity and outbreak risks.

Recently published models in this area have varied greatly in terms of sophistication and scope, ranging from simple deterministic models that ignore the indirect effects of vaccination or incorporate them by brute force [15–17, 19, 22] 16, to more sophisticated dynamic transmission and stochastic mathematical models [18, 20, 21, 23]. The latter have demonstrated the importance of taking the indirect effects of vaccination into consideration. For example, an economic evaluation of use of a conjugate C meningococcal vaccine in the UK using a dynamic age-structured transmission model estimated that cases avoided due to the indirect effects of vaccination were more than three times greater than the direct effect [23], a finding in agreement with our results for the US. Recent models have also highlighted the importance of outbreaks [19, 21], but none have evaluated the effect of vaccination on outbreaks. Although uncommon, outbreaks are a significant public health concern given the serious consequences and the fear generated in affected communities. The simulation described in this paper incorporates real-world data into a framework that allows for evaluation of both the direct and indirect benefits of vaccination, including the effects on outbreaks.

The model does have limitations that should be taken into consideration. Although not a limitation of the model per se, this simulation deals only indirectly with the relationship between carriage and invasive disease because the required data are lacking. We opted to include it anyway given its importance and calibrated the inputs to the observed herd immunity effects of the serogroup C vaccination campaign in the UK [11]. This simplified approach allows for a clear and relatively straightforward testing of assumptions but caution should be exercised in interpreting results as a linear relation is assumed between direct vaccine protection in the community and the extent of herd immunity for the same age group. For example, it is almost surely the case that changing coverage from 10% to 20% versus 70% to 80% will not have the same effect on herd immunity. Furthermore, the strength of the relationship between vaccine protection and coverage was estimated based on the experience of a single large-scale vaccination program [11]. While herd immunity with vaccination seems to have taken place with other vaccination programs, other factors not considered by the simulation may come into play in determining the indirect benefits of vaccination. The effect of vaccination on carriage, and as a consequence on herd immunity, was dealt with conservatively by assuming a shorter duration of protection against carriage than against the disease, and sensitivity analyses evaluated the impact of different assumptions.

By the same token, the incidence of outbreaks is forced, with calibration to existing data. The impact of vaccination on the occurrence of these outbreaks can then be taken into consideration, and sensitivity analyses can be run under alternative assumptions on the frequency and severity of high risk periods. The model does not take into consideration what factors lead to outbreaks, such as the appearance of more invasive strains of meningococcal disease.

Another limitation of the analyses presented here is the assumption of uniform communities of equal size. In reality, the epidemiology of disease may well vary by the size of the community as well as by other community characteristics. The sensitivity analyses run on community size, however, are reassuring in that when calibration procedure account for changes in the size of the community, only a limited impact on model predictions is observed.

Furthermore, uptake rates for both outbreak and routine vaccination may vary from community to community. With adequate data on inter-community migration patterns, as well as the distribution and make-up of communities in a given region or country, the model could handle differing communities and the effects of migration. Acquiring the data required for such analyses, however, would be a considerable undertaking. Furthermore, the computing time required to model large numbers of heterogeneous communities may be a deterrent. Uptake of routine vaccination was also assumed to be immediate. More gradual uptake would delay the time required to reach steady state and achieve maximum benefits.

Finally, the model currently assumes that the overall endemic risk of infection, outbreak frequency, and other epidemiological characteristics of the disease are constant over time. Again, while the model could incorporate longitudinal changes in these, we are not aware of any data that would allow forecasting of these changes over time.

Despite these limitations, the model accurately replicates the current incidence of disease and outbreaks in populations, and provides plausible ranges of estimates of outcomes with routine vaccination of adolescents. The simulation results also highlight the key role herd immunity can play in determining outcomes, and the sensitivity of predicted results to assumptions on the strength of herd immunity, as well as the duration of vaccine effectiveness.

Conclusion

The development of new vaccines, such as Menactra, a quadrivalent conjugate vaccine, and Menjugate, a conjugate C vaccine currently used for vaccination in countries such as the UK and Canada, provide important opportunities for reducing the burden of disease. Longer-term data on the effectiveness of these vaccines, both in terms of protection against disease and against carriage, will allow for more precise estimates. In the absence of appropriate real world data on outcomes of large-scale vaccination programs, however, decisions on optimal immunization strategies should be aided by tools such as this discrete event simulation. Analytic frameworks that ignore the indirect effects of vaccination resulting from herd immunity may produce misleading results.

Appendix 1 – simulation input data

General population characteristics

The age and gender distributions of individuals in each community at the onset of the simulation were based on overall US census data for the year 2000, as was the fertility rate used to calculate the number of births per year. All-cause death hazards (Hazard = b•m Age) were derived from 2000 US life table data for all ages, except under the age of one where the risk of death is much higher than predicted using a Gompertz function. Thus, infant mortality was taken directly from the life table.

The size of households (mean 2.6) was based on 2002 statistics from the US Census Bureau. For individuals under the age of 18, it was assumed that at least one adult would be residing in the same household.

The model assumes that all individuals of the appropriate age attend primary school, and that all individuals begin high school. Only 89.1% of individuals, however, complete high school [36], and, for the sake of simplicity it is assumed that all high school dropouts quit at the age of 15. Of those graduating from high school, 45% enroll in college [36], and 63.4% of them are assumed to complete their undergraduate degree [37]. Those not completing their degree are assumed to leave college at the age of 20 years. Dormitories are assumed to be the place of residence for 26% of college freshmen and 12% of remaining college students [7]. Data on the size of US schools were taken from statistics provided by the US National Center for Education Statistics [36].

Epidemiological data

Sero-specific risks of infection were calculated based on data in the CDC's Active Bacterial Core Surveillance (ABCs) reports for 1997 through 2001 [38] (Table 2). As these reports did not identify rates for serogroups A and W-135, data from the US for 1992 through 1996 [39] were used to estimate that 54.8% of "Other" cases were group W-135 disease, while only 2.1% were group A disease. Weibull functions ( P 1 · A g e P 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGqbaudaWgaaWcbaGaeGymaedabeaakiabl+y6NjabdgeabjabdEgaNjabdwgaLnaaCaaaleqabaGaemiuaa1aaSbaaWqaaiabikdaYaqabaaaaaaa@3795@ ) were fit to these data in order to calculate the rate of invasive disease at any given age (Table 3). A 3% reduction was applied to reflect that the model uses them for endemic disease only [1, 39]. College students living in dormitories were subject to an increased risk of infection (freshmen RR = 4.8, others 1.3) [7].

Table 2 Average annual rates (per 100,000 individuals) of meningococcal disease in the US (1997–2001)
Full size table
Table 3 Coefficients for Weibull functions† used to estimate rate of invasive disease
Full size table

The extent and duration (30 days) of increased risks of household and school contacts (Table 4) were derived from a UK study that examined clusters of meningococcal disease in household and school settings between 1993 and 1995 [28]. Given no apparent increased risk in nursery school settings, these types of schools were not included in the simulation. It was assumed that these rates applied to a population with high rates of chemoprophylaxis in household contacts [28] and that contacts were only at higher risk for disease caused by the same serogroup as the primary case.

Table 4 Relative risk of infection over time given presence of an index case
Full size table

A case fatality rate of 12.3% was used in the simulation based on data from the 1997 to 2001 ABCs reports. Data from the literature supplemented this result in order to develop serogroup [40] and age-specific [41] case fatality rates. Rates of long-term sequelae for patients who do not die from invasive disease were based on the literature (20% in adults [5], 13.2% under the age of 18 [42]), without sero-specificity (data were not available).

Health utilities

For the assessment of consequences related to meningitis cases in the absence of long term sequelae, we use age-specific health utilities to calculate quality adjusted life years (QALYs) which were based on reported EuroQOL scores [43]. A regression equation was fit, such that

Health_Utility = 0.985337 - 0.000922 Age - 0.000031 Age 2.

For adults with long-term sequelae, a loss of 26.25% in health utility was reported [5]. For individuals under the age of 18, types of complications were taken from a large study on outcomes of invasive disease in children in Ireland from 1995 to 2000 [4], and combined with complication-matched utility losses for each complication [5, 44–46] for a resulting average utility loss of 28%. For cases resulting in death or disability, outcomes are calculated over the individual's expected lifetime had they not been infected.

Appendix 2 – calibration procedures for key parameters

Outbreaks

In order to incorporate outbreaks in the model, high incidence periods were introduced in the simulation. The calibration procedure consisted of determining the increase in risk and the frequencies of high risk periods for each serogroup that best reproduced the available data on outbreak frequency and size in the US [6].

Over an eight year period, there were 76 outbreaks of meningococcal disease in the US, 48 of serogroup C disease, 19 of serogroup B disease, and 9 of serogroup Y disease. Of these, 26 were community-wide outbreaks, while 50 were limited to single institutions. Calibration to these outcomes was conducted by first determining, for each serogroup, what increase in risk would be required to lead to the correct distribution of community versus school-based outbreaks, assuming that 34% of outbreaks for all serogroups are community outbreaks. The resulting risks of infection during these high incidence periods, which were assumed to last for three months. For serogroup B, age-specific risks of infection were increased by a factor of 87.5; for serogroup C, by a factor of 118.5; and for serogroup Y, by a factor of 97.9. The second step of the calibration procedure relates to the frequency of occurrence of these high incidence periods, and resulted in annual probabilities of 0.36% per community for serogroup B disease, 0.88% for serogroup C disease, and 0.34% for serogroup Y disease.

Direct effects of vaccination

In the absence of long-term efficacy data for a conjugate meningococcal vaccine protecting against serotypes A, C, Y and W-135, we used UK effectiveness data [12] for a C conjugate meningococcal vaccine and assumed that the quadrivalent vaccine was equally effective against all serogroups. To extrapolate long-term protection from the 4 years of UK effectiveness data, and consistent with prevailing assumptions in infectious disease transmission models [20], we assumed that waning of vaccine protection is exponential (VE(t) = se -wt). The coefficients for the equation were estimated using the least squares method from data corresponding to adolescents vaccinated between 11 and 16 years of age. The mean duration of direct protection against meningococcal infection under these assumptions is 30.1 years (Figure 2).

For estimates of protection against carriage, necessary for calculation of the indirect effects of vaccination, we conservatively assumed that protection waned at twice the rate of protection against disease (w = 0.0648 for protection against carriage in the base case, and 0.1296 in sensitivity analyses).

Indirect effects of vaccination

The meningococcal C vaccination campaign in the UK [11], demonstrated that benefits accrue not only to vaccinated individuals, but also to the entire population. These are referred to as the "indirect" effects of vaccination (i.e., the reduction in the risk of contracting the disease due to fewer individuals transmitting it, or "herd" immunity). In order to capture these effects, an assumption was made that vaccinating individuals with a conjugate vaccine decreases the probability that they are carriers and can transmit the disease. Consistent with a recent epidemiological model of the impact of meningococcal vaccination in the UK [20], individuals in the same age group were assumed to mix more frequently than individuals across age groups. Inputs were calibrated to reproduce the outcome observed in the UK that with vaccination of 81% of individuals under the age of 20 years, disease incidence fell by 67% in unvaccinated individuals in the same age group and 35% in adults [11]. Conservatively, the indirect effects are assumed to decrease at rates twice that of direct ones. Specifically, indirect effects are included in the model through a factor reducing age-specific rates of invasive disease. These factors are calculated based on the reduction of carriers in the population resulting from vaccination and their contribution to the transmission of the disease within age group i. Formally:

[ h i ] = h 0 ∗ [ a ij c i p i ∑ j a ij c j p j ] ∗ [ v i ] MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqGGBbWwcqqGObaAdaWgaaWcbaGaeeyAaKgabeaakiabc2faDjabg2da9iabbIgaOnaaBaaaleaacqaIWaamaeqaaOGaey4fIOYaamWaaeaadaWcaaqaaiabbggaHnaaBaaaleaacqqGPbqAcqqGQbGAaeqaaOGaee4yam2aaSbaaSqaaiabbMgaPbqabaGccqqGWbaCdaWgaaWcbaGaeeyAaKgabeaaaOqaamaaqafabaGaeeyyae2aaSbaaSqaaiabbMgaPjabbQgaQbqabaGccqqGJbWydaWgaaWcbaGaeeOAaOgabeaakiabbchaWnaaBaaaleaacqqGQbGAaeqaaaqaaiabbQgaQbqab0GaeyyeIuoaaaaakiaawUfacaGLDbaacqGHxiIkcqGGBbWwcqqG2bGDdaWgaaWcbaGaeeyAaKgabeaakiabc2faDbaa@5676@

where h0 is the parameter measuring the overall impact of vaccination on the reduction of attack rates, a ij the elements of the matrix representing the relative impact of age groups j on the transmission of disease to individuals belonging to age group i, v i the proportion of carriers protected by vaccination in age group i, c i , the rate of carriers in age group i and p i the number of individuals in age group i.

The v i values depend on the vaccination strategy analyzed and p i on the population considered. If one assumes that the relative propensity to be a carrier according to age remains stable when vaccination is implemented, measuring indirect effects simply requires the identification of the values associated to h0 and a ij . This was accomplished by using data from the UK [11] on the age-specific reduction in incidence among non-vaccinated individuals and 2001 statistics for the size and age distribution of the UK population. Six age groups were considered: under three years, four to six years, seven to 12 years, 13 to 16 years, 17 to 19 years, and, 20 or greater years of age. The carriage rates considered were based on published data [47].

Development of the matrix required assuming no age-dependence of a ij to avoid too many unknowns. These values were fixed at 1 (specifically ∀ i ≠ j a ij = 1, a66 = 1). The remaining values of a ij and h0 were therefore calculated to match the UK data assuming a vaccine efficacy of 90%: a11 = 80.4, a22 = 40.2, a33 = 2.4, a44 = 6.8, a55 = 6.3, h0 = 2.55.

References

  1. 1.

    Rosenstein NE, Perkins BA, Stephens DS, Popovic T, Hughes JM: Meningococcal disease. N Engl J Med. 2001, 344: 1378-1388. 10.1056/NEJM200105033441807.

  2. 2.

    Jaccard Ruedin H, Ninet B, Pagano E, Rohner P: Epidemiology of meningococcal disease in Switzerland, 1999–2002. Eur J Clin Microbiol Infect Dis. 2004, 23: 517-522. 10.1007/s10096-004-1159-8.

  3. 3.

    Erickson L, De Wals P: Complications and sequelae of meningococcal disease in Quebec, Canada, 1990–1994. Clin Infect Dis. 1998, 26: 1164-10.1086/520303.

  4. 4.

    Healy CM, Butler KM, Smith EO, Hensey OP, Bate T, Moloney AC, MacMahon P, Cosgrove J, Cafferkey MT: Influence of serogroup on the presentation, course, and outcome of invasive meningococcal disease in children in the Republic of Ireland, 1995–2000. Clin Infect Dis. 2002, 34: 1323-1330. 10.1086/340050.

  5. 5.

    Erickson LJ, De Wals P, McMahon J, Heim S: Complications of meningococcal disease in college students. Clin Infect Dis. 2001, 33: 737-739. 10.1086/322587.

  6. 6.

    Brooks RB, Woods CW, Rosenstein NE: Neisseria meningitidis outbreaks in the United States, 1994–2002. [Abstract] 41st Annual Meeting of the Infectious Diseases Society of America, San Diego, California, October 9–12, 2003

  7. 7.

    Centers for Disease Control and Prevention: Prevention and control of meningococcal disease and meningococcal disease and college students. Recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR. 2000, 49: 13-20.

  8. 8.

    Zimmer SZ, Stephens DS: Meningococcal conjugate vaccines. Expert Opin Pharmacother. 2004, 5: 855-863. 10.1517/14656566.5.4.855.

  9. 9.

    Yazdankhah SP, Caugant DA: eisseria meningitidis: an overview of the carriage state. Journal of Medical Microbiology. 2004, 53: N821-832. 10.1099/jmm.0.45529-0.

  10. 10.

    Maiden MCJ, Stuart JM, for the UK Meningococcal Carriage Group: Carriage of serogroup C meningococci 1 year after meningococcal C conjugate polysaccharide vaccination. Lancet. 2002, 359: 1829-1830. 10.1016/S0140-6736(02)08679-8.

  11. 11.

    Ramsay ME, Andrews NJ, Trotter CL, Kaczmarski EB, Miller E: Herd immunity from meningococcal serogroup C conjugate vaccination in England: database analysis. BMJ. 2003, 326: 365-366. 10.1136/bmj.326.7385.365.

  12. 12.

    Trotter CL, Andrews NJ, Kaczmarski EB, Miller E, Ramsay ME: Effectiveness of meningococcal serogroup C conjugate vaccine 4 years after introduction. Lancet. 2004, 364: 365-367. 10.1016/S0140-6736(04)16725-1.

  13. 13.

    Bilukha OO, Rosenstein N: Prevention and control of meningococcal disease. Recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Recomm Rep. 2005, 54: 1-21.

  14. 14.

    Getsios D, Caro I, El-Hadi W, Caro JJ: Assessing the economics of vaccination for neisseria meningitidis in industrialized nations: a review and recommendations for further research. Int J Technol Assess Health Care. 2004, 20: 280-288.

  15. 15.

    Welte R, van den Dobbelsteen G, Bos JM, de Melker H, van Alphen L, Spanjaard L, Rumke HC, Postma MJ: Economic evaluation of meningococcal serogroup C conjugate vaccination programmes in The Netherlands and its impact on decision-making. Vaccine. 2004, 23: 470-479. 10.1016/j.vaccine.2004.06.019.

  16. 16.

    Jaccard Ruedin H, Ess S, Zimmermann HP, Szucs T: Invasive meningococcal and pneumococcal disease in Switzerland: cost-utility analysis of different vaccination strategies. Vaccine. 2003, 21: 4145-4152. 10.1016/S0264-410X(03)00562-0.

  17. 17.

    Tuckwell HC, Hanslik T, Valleron A-J, Flahault A: A mathematical model for evaluating the impact of vaccination schedules: application to Neisseria meningitidis. Epidemiol Infect. 2003, 130: 419-429.

  18. 18.

    Ranta J, Mäkelä PH, Arjas E: Predicting meningococcal disease outbreaks in structured populations. Stat Med. 2004, 23: 927-945. 10.1002/sim.1666.

  19. 19.

    Shepard CW, Ismael R, Ortega-Sanchez R, Scott RD, Rosenstein NE, the ABCs Team: Cost-effectiveness of conjugate meningococcal vaccination strategies in the United States. Pediatrics. 2005, 115: 1220-1232. 10.1542/peds.2004-2514.

  20. 20.

    Trotter CL, Gay NJ, Edmunds WJ: Dynamic models of meningococcal carriage, disease, and the impact of serogroup C conjugate vaccination. Am J Epidemiol. 2005, 162: 89-100. 10.1093/aje/kwi160.

  21. 21.

    Stollenwerk N, Maiden MCJ, Jansen VAA: Diversity in pathogencity can cause outbreaks of meningococcal disease. PNAS. 2004, 101: 10229-10234. 10.1073/pnas.0400695101.

  22. 22.

    De Wals P, Trottier P, Pépin J: Relative efficacy of different immunization schedules for the prevention of serogroup C meningococcal disease: a model-based evaluation. Vaccine. 2006, 24: 3500-3504. 10.1016/j.vaccine.2006.02.010.

  23. 23.

    Trotter CL, Edmunds WJ: Reassessing the cost-effectiveness of meningococcal serogroup C conjugate (MCC) vaccines using a transmission dynamic model. Med Decis Making. 2006, 26: 38-47. 10.1177/0272989X05284109.

  24. 24.

    Caro JJ: Pharmacoeconomic analyses using discrete event simulation. Pharmacoeconomics. 2005, 23: 323-332. 10.2165/00019053-200523040-00003.

  25. 25.

    Koopman J: Modeling infection transmission. Annu Rev Public Health. 2004, 25: 303-326. 10.1146/annurev.publhealth.25.102802.124353.

  26. 26.

    Jackson LA, Schuchat A, Gorsky RD, Wenger JD: Should college students be vaccinated against meningococcal disease? A cost-benefit analysis. Am J of Public Health. 1995, 85 (6): 843-845.

  27. 27.

    Scott RD, Meltzer MI, Erickson LJ, De Wals P, Rosenstein NE: Vaccinating first-year college students living in dormitories for meningococcal disease. An economic analysis. Am J Prev Med. 2002, 23 (2): 98-105. 10.1016/S0749-3797(02)00462-2.

  28. 28.

    Hastings L, Stuart J, Andrews N, Begg N: A retrospective survey of clusters of meningococcal disease in England and Wales, 1993 to 1995: estimated risks of further cases in household and educational settings. Commun Dis Rep. 1997, 7: R195-R200.

  29. 29.

    Zangwill KM, Schuchat A, Riedo FX, Pinner RW, Koo DT, Reeves MW, Wenger JD: School-based clusters of meningococcal disease in the United States. Descriptive epidemiology and a case-control analysis. JAMA. 1997, 277: 389-395. 10.1001/jama.277.5.389.

  30. 30.

    Centers for Disease Control and Prevention: Control and prevention of serogroup C meningococcal: evaluation and management of suspected outbreaks: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR. 1997, 46: 1-21.

  31. 31.

    Patrick DM, Champagne S, Goh SH, Arsenault G, Thomas E, Shaw C, Rahim T, Taha F, Bigham M, Dubenko V, Skowronski D, Brunham RC: Neisseria meningitidis carriage during an outbreak of serogroup C disease. Clin Infect Dis. 2003, 37: 1183-1188. 10.1086/378743.

  32. 32.

    Kellerman SE, McCombs K, Ray M, Baughman W, Reeves MW, Popovic T, Rosenstein NE, Farley MM, Blake P, Stephens DS, Georgia Emerging Infections Program: Genotype-specific carriage of neisseria meningitidis in Georgia counties with hyper-and hyposporadic rates of meningococcal disease. J Infect Dis. 2002, 186: 40-48. 10.1086/341067.

  33. 33.

    De Wals P, Dionne M, Douville-Fradet M, Boulianne N, Drapeau J, De Serres G: Impact of mass immunization campaign against serogroup C meningococcus in the province of Quebec, Canada. WHO Bulletin OMS. 1996, 74: 407-411.

  34. 34.

    Trotter CL, Ramsay ME, Kaczmarski EB: Meningococcal serougroup C conjugate vaccination in England and Wales: coverage and initial impact of the campaign. Commun Dis Public Health. 2002, 5: 220-225.

  35. 35.

    Bozzette SA, Boer R, Bhatnagar V, Brower JL, Keeler EB, Morton Stoto MA: A model for a smallpox-vaccination policy. N Engl J Med. 2003, 3: 416-425. 10.1056/NEJMsa025075.

  36. 36.

    National Center for Education Statistics: Digest of education statistics tables and figures 2001. Accessed June 16, 2003, [http://nces.ed.gov/programs/digest/d01/index.asp]

  37. 37.

    National Center for Education Statistics: Student effort and educational progress 2003. [http://nces.ed.gov/pubs2003/2003067_3.pdf]

  38. 38.

    Centers for Disease Control and Prevention: Active Bacterial Core Surveillance Reports, 1997–2001. [http://www.cdc.gov/abcs]

  39. 39.

    Rosenstein NE, Perkins BA, Stephens DS, Lefkowitz L, Cartter ML, Danila R, Cieslak P, Shutt KA, Popovic T, Schuchat A, Harrison LH, Reingold AL: The changing epidemiology of meningococcal disease in the United States, 1992–1996. J Infect Dis. 1999, 180: 1894-1901. 10.1086/315158.

  40. 40.

    Squires SG, Pelletier L, Mungai M, Tsang R, Collins F, Stoltz J: Invasive meningococcal disease in Canada, 1 January 1997 to 31 December 1998. Can Commun Dis Rep. 2000, 26: 177-182.

  41. 41.

    Sampaio Moura A, Pablos-Mendez A, Layton M, Weiss D: Epidemiology of meningococcal disease, New York City, 1989–2000. Emerg Infect Dis. 2003, 9: 355-361.

  42. 42.

    Stovall SH, Schutz GE: Meningococcal infections in children in Arkansas. Pediatr Infect Dis J. 2002, 21: 366-370. 10.1097/00006454-200205000-00003.

  43. 43.

    Sturgis P, Thomas R, Purdon S, Bridgewood A, Dodd T: Comparative review and assessment of key health state measures of the general population. 2001, Department of Health, London

  44. 44.

    Bos JM, Rumke HC, Welte R, Postma MJ, Jager JC: Health economics of a hexavalent meningococcal outer-membrane vesicle vaccine in children: potential impact of introduction in the Dutch vaccination program. Vaccine. 2002, 20: 202-207. 10.1016/S0264-410X(01)00254-7.

  45. 45.

    Oostenbrink R, Moll HA, Essink-Bot M: The EQ-5D and the health utilities index for permanent sequelae after meningitis. A head-to-head comparison. J Clin Epidemiol. 2002, 55: 791-799. 10.1016/S0895-4356(02)00448-1.

  46. 46.

    The Diabetes Control and Complications Trial Research Group: Lifetime benefits and costs of intensive therapy as practiced in the diabetes control and complications trial. The Diabetes Control and Complications Trial Research Group. JAMA. 1996, 276: 1409-1415. 10.1001/jama.276.17.1409.

  47. 47.

    Coen PG, Cartwright K, Stuart J: Mathematical modelling of infection and disease due to neisseria meningitidis and neisseria lactamia. Int J Epidemiol. 2000, 29: 180-188. 10.1093/ije/29.1.180.

Pre-publication history

  1. The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/7/130/prepub

Download references

Acknowledgements

This work was supported in part by an unrestricted grant from sanofi pasteur.

Author information

Author notes

    Affiliations

    1. Caro Research Institute, 336 Baker, Concord, MA, USA

      • J Jaime Caro

    2. Division of General Internal Medicine and Department of Epidemiology, Biostatistics and Occupational Health, Faculty of Medicine, McGill University, Montreal, Quebec, Canada

      • J Jaime Caro

    3. Caro Research Institute, Vaggarpsvagen 11, SE24193, Eslov, Sweden

      • Jörgen Möller

    4. Caro Research Institute, 6415 Seaforth Street, Halifax, NS B3L 1R4, Canada

      • Denis Getsios

    5. sanofi pasteur, 2 Ave du Pont Pasteur, 69367, Lyon cedex 07, Lyon, France

      • L Coudeville
      • , Catherine Chevat
      •  & Van Hung Nguyen

    6. sanofi-aventis Canada Inc., Laval, Quebec, Canada

      • Wissam El-Hadi

    7. Caro Research Institute, 185 Dorval Ave., Montreal, Quebec, H9S 5J9, Canada

      • Ingrid Caro

    Authors

    1. Search for J Jaime Caro in:

    2. Search for Jörgen Möller in:

    3. Search for Denis Getsios in:

    4. Search for L Coudeville in:

    5. Search for Wissam El-Hadi in:

    6. Search for Catherine Chevat in:

    7. Search for Van Hung Nguyen in:

    8. Search for Ingrid Caro in:

    Corresponding author

    Correspondence to J Jaime Caro.

    Additional information

    Competing interests

    Part of this work was supported by an unrestricted grant from sanofi pasteur, the maker of Menactra, a quadrivalent conjugate vaccine for meningococcal disease. At the time that this work was conducted and the manuscript written, W. El-Hadi worked for Caro Research. W. El-Hadi now works for sanofi-aventis in Montreal, Quebec, Canada. JJ Caro, J Möller, D Getsios, and I Caro are employees of Caro Research, which has received grants from sanofi pasteur for other, unrelated research projects. L. Coudeville, C. Chevat and VH Nguyen are employees of sanofi pasteur.

    Authors' contributions

    J. Jaime Caro, Jörgen Möller, Denis Getsios and Laurent Coudeville collaborated in the development and programming of the simulation. All of the authors contributed to the design of the model, development and collection of the input data, analysis of the trial results and writing of the manuscript. All authors have read and approved the final manuscript.

    J Jaime Caro, Jörgen Möller, Denis Getsios, L Coudeville, Wissam El-Hadi, Catherine Chevat, Van Hung Nguyen and Ingrid Caro contributed equally to this work.

    Authors’ original submitted files for images

    Authors’ original file for figure 1

    Authors’ original file for figure 2

    Authors’ original file for figure 3

    Authors’ original file for figure 4

    Authors’ original file for figure 5

    Authors’ original file for figure 6

    Rights and permissions

    Reprints and Permissions

    About this article

    Keywords

    • Discrete Event Simulation
    • Meningococcal Disease
    • Herd Immunity
    • Routine Immunization
    • Routine Vaccination

    BMC Public Health

    ISSN: 1471-2458

    Contact us