- Research article
- Open Access
- Open Peer Review
Measuring underreporting and under-ascertainment in infectious disease datasets: a comparison of methods
BMC Public Health volume 14, Article number: 147 (2014)
Efficient and reliable surveillance and notification systems are vital for monitoring public health and disease outbreaks. However, most surveillance and notification systems are affected by a degree of underestimation (UE) and therefore uncertainty surrounds the 'true’ incidence of disease affecting morbidity and mortality rates. Surveillance systems fail to capture cases at two distinct levels of the surveillance pyramid: from the community since not all cases seek healthcare (under-ascertainment), and at the healthcare-level, representing a failure to adequately report symptomatic cases that have sought medical advice (underreporting). There are several methods to estimate the extent of under-ascertainment and underreporting.
Within the context of the ECDC-funded Burden of Communicable Diseases in Europe (BCoDE)-project, an extensive literature review was conducted to identify studies that estimate ascertainment or reporting rates for salmonellosis and campylobacteriosis in European Union Member States (MS) plus European Free Trade Area (EFTA) countries Iceland, Norway and Switzerland and four other OECD countries (USA, Canada, Australia and Japan). Multiplication factors (MFs), a measure of the magnitude of underestimation, were taken directly from the literature or derived (where the proportion of underestimated, under-ascertained, or underreported cases was known) and compared for the two pathogens.
MFs varied between and within diseases and countries, representing a need to carefully select the most appropriate MFs and methods for calculating them. The most appropriate MFs are often disease-, country-, age-, and sex-specific.
When routine data are used to make decisions on resource allocation or to estimate epidemiological parameters in populations, it becomes important to understand when, where and to what extent these data represent the true picture of disease, and in some instances (such as priority setting) it is necessary to adjust for underestimation. MFs can be used to adjust notification and surveillance data to provide more realistic estimates of incidence.
Efficient and reliable surveillance and notification systems are vital for monitoring public health trends and disease outbreaks. They also often form the backbone of evidence-based decision-making processes, as well as infectious disease (ID) public health policies that deal with prioritisation, and the planning of intervention measures and healthcare services . However, there are limitations associated with the use of data from surveillance and notification systems since most systems are affected by a degree of underestimation and therefore uncertainty surrounds the 'true’ incidence of disease . IDs are considered particularly prone to underestimation due to their specific characteristics (e.g. asymptomatic or self-limiting disease courses) and are therefore represented inadequately by raw surveillance data. Thus, when routine data are used to inform decisions relating to resource allocation or to estimate epidemiological parameters in a population, it becomes important to understand when, where and to what extent these data do or do not comprehensively represent the true picture of disease. Furthermore, in certain circumstances, such as priority setting, it is appropriate to adjust infectious disease datasets in order to account for the portion not captured by the surveillance system. There are several metrics that can be employed in priority setting with Disability-Adjusted Life Years (DALYs) being just one composite health metric that combines and measures adverse health effects and premature mortality in a single unit. DALYs were chosen by the European Centre for Disease Prevention and Control (ECDC) and used within the Burden of Communicable Diseases in Europe (BCoDE)-project to generate evidence-based and comparable burden of disease (BoD) estimates for 32 IDs across European Member States (MS) [3–6]. A major prerequisite of DALY calculations is 'true’ incidence data but since data are often obtained from (inter)national-level routine surveillance datasets that are frequently incomplete, data must be adjusted before serving as input for computing disease burden.
Here we present an overview of why, where and in what form underestimation occurs within the morbidity surveillance pyramid (Figure 1A) and we give several disease-specific examples from the literature of the methods that can be used to estimate the extent of underestimation. Furthermore, we compare the extent of underestimation and multiplication factors to adjust for it using key examples from the literature for two diseases, salmonellosis and campylobacteriosis. This body of work was a core aspect of the BCoDE-project.
Underestimation (UE), as defined here, can be understood as the many ways in which surveillance systems fail or are unable to reflect all infections in a given population. Mathematically, UE is the number of infections estimated to have occurred in a population that have not been captured by the surveillance system for every reported case over a given time period. UE can be split into two distinct levels as represented by the surveillance pyramid for IDs (Figure 1A); under-ascertainment (UA) of infections occurring at the community-level and underreporting (UR) of infections occurring at the healthcare-level. Under-ascertained infections occur in individuals that do not seek healthcare and hence cannot be captured by surveillance systems which are typically designed to capture cases that do seek healthcare. UA can be estimated as the number of infections occurring in individuals that do not attend healthcare services for every case that attends. There is a symptomatic fraction of all under-ascertained cases that do not attend healthcare due to mild symptoms and/or the knowledge that the illness is self-limiting or for some other reason, and an asymptomatic fraction that do not seek healthcare as they are not aware of their infection status due to lack of symptoms . Underreported infections are infections in individuals that do seek healthcare, but whose health event is not captured by the surveillance system and not notified through the notification system [7, 8, 11, 12]. UR can be estimated as the number of infected individuals attending healthcare services whose health event is not reported to the appropriate public health body for every attending case whose health event is reported. UR can be due to under-diagnosis which accounts for the cases attending healthcare but whose infection or pathogen is not diagnosed or misdiagnosed [7, 8], and under-notification which accounts for the failure to report (using correct International Classification of Diseases (ICD) codes [13, 14]) all positive diagnoses through the notification system [15, 16]. Reporting completeness refers to the proportion of cases attending healthcare whose health event was correctly diagnosed and appropriately reported . These technical terms are used frequently in the literature, however often with varying definitions. The definitions of UA, UR and UE as stated here were developed during the BCoDE-project [3, 18] and will be used as such for the remainder of the article.
Factors influencing UA in morbidity datasets
Not all people who are infected with a pathogen seek healthcare . One important reason for this is that when symptoms are absent, mild or self-limiting, there is a lack of urgency to seek healthcare . Therefore, surveillance systems can only capture cases with symptoms that are severe enough to motivate infected individuals to attend healthcare services. Health literacy also influences the decision to attend healthcare services or not. If a community has a poor understanding of when to seek healthcare and lacks knowledge of the severity or duration of an illness; then the uptake of healthcare services and levels of case ascertainment could be lower than expected. Such awareness and recognition of disease as well as the urgency or perceived need to seek healthcare can vary through time and space, particularly during outbreak years and especially if there is enhanced surveillance or widespread campaigns and intensive media coverage . Compared with non-outbreak years, the proportion of cases in a population that is ascertained is expected to be greater. Health literacy and perceived need for healthcare may also explain often observed differences between age- and sex-specific ascertainment rates with, for example, children aged less than 15 years being statistically more likely to seek healthcare for gastroenteritis compared to adults (30–64 years) .
In addition, cultural and religious factors could prevent individuals from seeking healthcare if, for example, there is stigma or negative beliefs associated with healthcare services, illness and treatment [22, 23]. There may also be legal, administrative and financial barriers to attending healthcare if individuals are not registered or are unable to register and if healthcare is dependent on the legal status of an individual or ability to pay. Migrants or marginalised groups may be particularly affected by this  and in addition to not having their disease episode captured by the surveillance system, they may not be enumerated at all and hence do not contribute to the country population count or denominator. Individuals from remote communities and their illness may also be uncounted due to healthcare services being physically unreachable. Overall, ascertainment rates are thought to vary significantly within and between disease groups, population groups and countries.
Factors influencing UR in morbidity datasets
Not all cases that attend healthcare will have their health status correctly diagnosed and reported to the appropriate health authorities. This break in the surveillance chain can occur within clinics, hospitals or laboratories due to healthcare workers lacking in ability, capacity or knowledge of how and when to act. Under-diagnosis may arise when biological samples are not requested from or provided by patients, where there are budget restrictions forcing healthcare professionals to limit their requests for testing samples, lack of knowledge of which tests to perform, inadequate diagnostic tools, or due to restrictions of laboratory testing regimes (regulations on which tests to apply routinely, and lack of availability of more specialised tests). Under-notification may result from an inadequate reporting system or lack of knowledge of when, for which diseases and how to report correctly including knowledge of ICD codes [25–29]. The proportion of cases reported is often higher where there is a legal requirement to report (some diseases or pathogens have mandatory reporting statuses)  or where there are incentives for healthcare workers to request or test biological samples from patients or to report results . Furthermore, where there is a perceived urgency to request biological samples or report cases, for example during outbreak years where there is higher awareness and chance of recognition and motivation for testing [31, 32], reporting rates are likely to increase. In contrast, UR may be greater for rarer diseases, those with only occasional outbreaks or those without mandatory reporting statuses. This perceived urgency or necessity to test or report tends to increase for more serious conditions (severity and duration of illness [8, 33]) and can also be age- or sex-dependent . Incomplete reporting of additional information, such as age and sex of the patient, concurrent infections or sequelae following an initial infection  becomes particularly important when UR for a particular disease is age- or sex-specific.
Identifying areas and extent of UE
Various study designs can be used to determine the extent of UR and UA in the surveillance system.
Community-based studies (CBS) aim to generate new estimates of pathogen carriage or infection in a (representative) sample of the population. This alone is useful and interesting but in addition, if this new estimate is considered the 'true’ incidence in the community, it can be compared to notification data and the magnitude of UE deduced. This order of magnitude can then be used as a multiplication factor (MF) (Figure 1B) to adjust disease datasets assuming that the base value to which the MF is being applied was created using the same data type (i.e. MF calculated by comparing CBS data with notification data is used to adjust other notification datasets, and not laboratory data). These observational studies can also be used to produce incidence rates of symptomatic and asymptomatic cases as well as estimate the specific proportion of symptomatic cases presenting to healthcare facilities (by asking about attendance in the questionnaire). Furthermore, the proportion of cases underreported can be estimated by a variation of CBS in which healthcare professionals are surveyed to gain information on propensity to request biological samples and reporting habits.
CBS can take many forms but generally involve active searching within the community for disease episodes, pathogen carriage or infection, with questionnaire-based data acquisition often accompanied by biological sampling. Active searching can be conducted face-to-face, by telephone, internet or post, with several possible study designs e.g. based on probability samples, prospective or retrospective cohorts, population cross-sections, involving representative samples of the whole population or certain interest or high-risk groups only. CBS are especially useful for diseases commonly under-ascertained (i.e. those with many mild and asymptomatic cases of mostly self-limiting illness) and where an unknown burden exists within the community, e.g. sexually transmitted infections with Chlamydia trachomatis [36–40] or Neisseria gonorrhoea [36–39, 41–44]; influenza and influenza-like illnesses [45, 46]; and food and water-borne diseases [2, 47–54]). The first and second Infectious Intestinal Disease studies (IID1 and IID2) were prospective community cohort studies that estimated overall incidence of infectious intestinal disease (IID) in the UK community, the proportion seeking healthcare (ascertained), and the proportion reported to the national public health agency [2, 11, 47, 55]. Weekly surveys (by email or prepaid postcard) recorded if participants had experienced diarrhoea and/or vomiting and if they had, they were asked additional questions and requested to provide a stool sample. Similarly, several studies have employed statistical and mathematical methods to estimate incidence and UE using data collected during a Dutch prospective community-based cohort study of gastroenteritis (SENSOR) and a Dutch GP-based cohort of gastroenteritis (NIVEL) (e.g. [56–63]).
CBS are not without limitations as bias can arise at numerous points. Sampling bias, owing to non-random sampling of a population, can result in a study that is not representative of the entire population with certain groups (such as ethnic, migrant, age or occupational) inadvertently excluded from the study because they are unregistered, not easily locatable, do not have access to a telephone (in the case of telephone surveys), have language barriers or are marginalised for other reasons. Responder bias can also lead to unrepresentative samples since only certain groups of people will agree to participate, and measurement bias can result from case definitions that are undefined, too general, too strict or simply not used consistently. Additionally, interviewers may ask questions or interpret responses in a leading manner or respondents may induce bias during disease occurrence recall. To minimise the effects of bias, Wilking et al.  took several steps in a recent population-based telephone survey of acute gastrointestinal illness in Germany including; contacting listed and unlisted telephone numbers and using the 'last-birthday method’ to reduce responder bias, using the computer-assisted telephone interview (CATI) method to minimise interviewer bias and applying study weights to improve representation of the target population. Telephone surveys, however, have a further limitation associated with them since disease occurrence tends to be based on self-reported symptoms and so without clinically-determined or laboratory-confirmed diagnoses, there is uncertainty surrounding the causative agent and incidence of infection. To address this, Kubota et al.  used data from population-based telephone surveys to adjust the number of cases for each pathogen from active laboratory-based surveillance (whilst modelling for uncertainty). Other telephone surveys focus on general conditions, such as gastroenteritis, rather than specific pathogens and these have been successfully applied in many countries (e.g. [54, 64–66]) with the results often the basis for pyramid reconstruction activities (see below).
Serological surveys are a specific type of CBS that measure sero-incidence (the rate of new infections) or sero-prevalence (the total number of infections in the community or cohort) as quantified by antigen or antibody positivity. This CBS can capture asymptomatic and symptomatic, historical and acute infections but if biological sampling is combined with a questionnaire asking about disease episodes, or if the antibody or antigen threshold at which symptoms manifest is known; then the symptomatic fraction can be obtained (e.g. hepatitis B , hepatitis B and C , pertussis , measles , HIV ). This is crucial for BoD studies since, for the majority of diseases, it is only clinical manifestations and not asymptomatic infections that contribute to burden in terms of DALYs. The exceptions to this are the few infectious diseases with a possible asymptomatic acute stage that can result in sequelae (and death) at a later time and hence contribute to disease burden (including hepatitis B, hepatitis C, chlamydia, Invasive Meningococcal Disease, and Q-fever). When calculating DALYs for these diseases, symptomatic cases serve as input to estimate the numbers of asymptomatic infections and thus the calculated burden attributable to asymptomatic infections is included in the final burden estimate .
It is often difficult to differentiate between historic and current infections and therefore it is important to test for recognised serological markers of recent infection and to have full knowledge of antibody decay rates in different populations [72, 73]. Furthermore, antibodies resulting from natural exposure versus vaccination cannot be distinguished and thus serological surveys of diseases with universal vaccine coverage, including tuberculosis (BCG vaccine), measles, rubella, and other childhood vaccine-preventable diseases, may have limited use.
Returning traveller studies (RTS)
Returning traveller studies (RTS) are further examples of CBS where individuals returning from abroad represent sentinel populations for the reported national incidence of infection in a traveller’s destination of travel . In RTS, the risk of infection for travellers from country A visiting country B is calculated by taking the number of infected travellers returning home from country B from surveillance records as a numerator, and the total number of travellers from country A visiting country B from travel pattern databases as the denominator. This measure of risk can then be used to generate a new estimate of incidence in country B (risk multiplied by the population size), which when compared to the national notification records of country B, can generate a MF of underestimation. Using this method, the incidence and proportion of underestimated cases of salmonellosis in several European countries was calculated by de Jong and Ekdahl  by comparing the incidence of infection in Swedish returning travellers to the national incidence in the countries the travellers had returned from, using Norway as a reference country. This not only produced national-level comparable estimates of incidence, but also a MF of UE (or in this case 'under-detection index’). More recently, Havelaar et al.  calculated incidence rates of salmonellosis and campylobacteriosis and MFs based on disease risks of returning Swedish travellers, anchored to data from SENSOR, the Dutch population-based study on gastroenteritis (see Tables 1 and 2).
While RTS generate comparable estimates of incidence and multipliers across different countries, there are several assumptions and limitations. Even if the travel patterns database is representative of the whole population and that the origin of infection, as reported in surveillance data, is correct ; in calculating risk of infection, the numerator (number of infections caught abroad) will still be affected by UR and UA. This in part is due to general reasons affecting UE, but in addition travellers have different health-seeking behaviour than that of the general population and there may be bias in requesting samples and reporting by health professionals following travel to certain destinations . Furthermore, in the case of Swedish travellers the duration and reason for travel differs by country, with many short business trips to neighbouring Scandinavian countries but longer holidays to Mediterranean countries  and therefore there is a bias in the risk estimates towards countries with the most tourists . The risk of infection differs given the destination of travel but in addition the probability of the infection being ascertained also differs since on a longer trip, the case may have recovered before returning home and attending healthcare . Lastly, a traveller’s risk is not the same as the native population’s risk due to differences in behaviours and activities, as well as immunity to local pathogen populations .
Capture-Recapture Studies (CRS)
Capture-recapture studies (CRS) utilise the ecological principle for studying populations of wildlife by marking subjects on initial release or first capture and recovering information from them on subsequent captures [87–89]. In terms of human disease surveillance, a personal identifier number or code usually represents the 'marker’ and the 'captures’ are records of disease episodes, colonisations or infections found in data sources including national notifications of morbidity and death, hospital and GP records, laboratory reports, as well as other public health registries [87, 90]. Two or more data sources are compared (Hest et al. stated that at least three are preferred to avoid correlations ) or cross-linked (through personal identifiers) and duplicates removed to approximate reporting completeness of each data source, identify the cases that would have been missed if using only a single data source and calculate a new estimate of incidence (Figure 2). There are several examples of CRS studies in the literature spanning a wide range of both communicable and non-communicable diseases (e.g. CRS of tuberculosis in several countries; Greece [79, 91], Italy [92, 93], the Netherlands [94, 95], Romania , UK [97–99] and USA [17, 100, 101]). Despite the usefulness of this method, some cases attending healthcare are not captured in any single data source or are not correctly recorded and hence the new incidence of cases will still be affected by UR . In addition, CRS only correct for UR of infections and do not account for UA occurring in the community.
There are numerous mathematical and statistical methods that can generate new estimates of incidence in the population as well as calculating the predicted proportion of UE occurring at several steps of the reporting chain, which can then be used to generate country- and disease-specific MFs. As well as using simulated data, these methods often utilise data from national surveillance records, CBS, CRS and several other study designs and therefore it is difficult to consider modelling as an independent method for identifying UE (many studies use a combination of methods, e.g. statistical modelling is often used to analyse results of CBS).
Attack rates, that estimate the proportion infected (from which cumulative incidences can be estimated and MFs generated), are calculated using data from CBS or national surveillance data (although these are still subject to UE), (e.g. influenza [102, 103]). Vaccine coverage, when below a certain threshold and when the basic reproduction number is known for a pathogen, can be used to estimate the number of susceptible individuals in the community and therefore expected incidence (and hence MFs), (e.g. measles ). Scallan et al.  and Thomas et al.  used statistical methods to 'scale-up’ counts of laboratory-confirmed cases to an estimated number of illnesses in the United States and Canada respectively, therefore adjusting for UE. Serological data can be modelled statistically to estimate incidence (e.g. HIV ) and past incidence of infection calculated from the current prevalence of antibody in a population using a catalytic model (e.g. hepatitis A ). Decision tree models, (e.g. for Sexually Transmitted Infections (STIs) , hepatitis A ), and similarly probability models and pyramid reconstruction models (e.g. food and water borne disease [9, 33, 58, 85, 111, 112], influenza ) estimate the country- and pathogen-specific probabilities of action at each incremental stage of the surveillance pyramid (e.g. attending healthcare versus not, submitting a sample versus not, reporting versus non-reporting). Further modelling techniques include Bayesian synthesis of multiple evidence sources that estimate the 'true’ incidence of an infection at several steps of the surveillance pyramid, as well as changes in contact patterns and health-seeking behaviour (e.g. H1N1 influenza pandemic [113, 114]). Simulation models, based on outcome trees of disease progression are also tools that can estimate expected incidence, (e.g. hepatitis B ).
An extensive literature review was conducted to identify studies (of designs presented above) that estimate ascertainment or reporting rates for salmonellosis and campylobacteriosis in European Union Member States (MS), plus European Free Trade Area (EFTA) countries Iceland, Norway and Switzerland and all other OECD countries. Articles were considered relevant if they: measured the sensitivity of reporting or reported the rate of UA, UR or UE; reported MFs, measured a new incidence or prevalence of infection from which a MF could be derived; or used any alternative methodology to correct surveillance or notification data. To identify appropriate studies, a literature review for each disease (salmonellosis and campylobacteriosis) and each pathogen (Salmonella spp. and Campylobacter spp.) was conducted in PubMed using the search terms: burden, cost-of-illness, cost of disease, cost-effectiv*, cost-analys*, cost-benefit, cost-utility, disability-adjusted, mathematical model*, multiplication factor*, multiplier*, outbreak*, prospective stud*, quality of life, quality-adjusted, serological stud*, serological survey*, serosurveillance, sero-surveillance, seroprevalence, statistical model*, telephone (*denotes any ending to the search term); linked by 'OR’. The search was restricted to articles written in English and to the years 1990–2011 since surveillance systems, reporting protocols and epidemiological patterns may have been different in the years preceding 1990, hence MFs would be less appropriate for adjusting current surveillance and notification data.
Following identification of these studies, MFs were either taken directly from the literature or derived where the proportion of underestimated, under-ascertained, or underreported cases was known (MF = 100/(percentage reported or ascertained or estimated), Figure 1B). MFs for salmonellosis and campylobacteriosis were compared to gain an understanding of variation between and within countries when using different methods to estimate UR and UA.
MFs were found or derived for all European Union Member States (MS) plus European Free Trade Area (EFTA) countries Iceland, Norway and Switzerland and four other OECD countries (USA, Canada, Australia and Japan) for salmonellosis from 22 references, and similarly for campylobacteriosis (excluding Croatia, Greece, Iceland, Latvia and Portugal) from 18 references. Table 1 (salmonellosis) and Table 2 (campylobacteriosis) present MFs for adjusting surveillance data for UE in one step, and MFs for adjusting for UA and UR in these countries. By multiplying together one MF of UR and one MF of UA, this results in one single MF of UE. MFs were found to vary widely between study types, countries and diseases with MFs for UE ranging from 0.4 (suggesting over-reporting)  in Iceland to 2082.9  in Portugal for salmonellosis and from 0.4  in Sweden and Finland to 39,000  in Bulgaria for campylobacteriosis. In countries with mandatory notification of infection (for salmonellosis this includes all EU countries other than Belgium, France, Luxembourg and Spain which are voluntary, and the UK which requires reporting of the pathogen rather than disease), reporting rates were expected to be higher (and hence MFs lower). Unfortunately, there are too few MFs of UR to verify this from these studies. The most common study type for generating MFs of UE for both diseases identified by the literature review was RTS which can provide comparable multipliers for several countries. Furthermore, most studies provide a single MF to adjust for UE in one step, suggesting this is the most straight-forward approach. Few studies (12 for salmonellosis and 10 for campylobacteriosis) provide any measure of uncertainty surrounding the MF. While age-stratified MFs are preferred since ascertainment and reporting are affected by age; stratification into age bands was only found for UA of salmonellosis (S.braenderup) in Japan in one study . For individuals less than 10 years of age, the MF was lower than for individuals over 10 years and hence the older age group was ascertained less often. No study described sex-specific stratification of MFs for either disease. Two studies stratified MFs by severity of clinical symptoms based on duration of symptoms  and if there was blood in stool samples [33, 49]. Where duration of illness was short, cases were less likely to seek healthcare leading to higher UA and higher MFs. With bloody diarrhoea due to Salmonella in USA, Voetsch et al.  estimated higher ascertainment (lower MF) and lower overall underestimation compared to non-bloody diarrhoea. Few studies make a distinction between strain types. Where MFs of UR were estimated from CRS (two studies of salmonellosis), a MF for each data source is listed showing the degree of UR per data source which is consistent for both studies. The United Kingdom and the Netherlands had the most number of MFs of UE for both salmonellosis and campylobacteriosis. For each country, all studies were reasonably consistent in terms of order of magnitude. This may be due to the overlap of data sources used in the studies. However, the highest MF estimates for both countries and each disease were estimated by the pyramid reconstruction model. These higher estimates of MFs may represent the thorough nature of accounting for each incremental step of the surveillance pyramid which could lead to double-counting of cases.
Here, we discuss the advantages and disadvantages of different methods for identifying UE in the surveillance pyramid and compare MFs resulting from those methods. MFs show considerable between-country and -disease variation which may reflect true differences in reporting and ascertainment rates. However, study design (which often depends on the disease, data type, data quality and availability of resources) and hence the method used to estimate UE likely accounts for the presented within-country variability of MFs. It remains difficult to select the most appropriate study (and corresponding MFs) as there are limitations associated with each. CBS that are representative of the whole population are often favoured for approximating UE, UA, and UR but the quality of MFs derived depends highly on the study design; CRS are very good at estimating UR in the surveillance pyramid but some reported cases may remain undetected and UA is not considered; and RTS and serological surveys also estimate UE effectively but the many associated limitations must be realised. In addition, an important limitation is the uncertainty that surrounds estimates which is relevant to all study types. Many of the studies from the literature review do not report uncertainty for MFs which gives a false impression that we are sure that these point estimates are correct. In fact, a great level of uncertainty is expected . Since the factors contributing to UE are multiplicative (Figure 1B), they explode rapidly and the resulting MF can be very large (this is clearly observed in pyramid reconstruction studies which aim to account and correct for UE at each incremental step of the surveillance chain). Therefore, even small degrees of uncertainty in measuring individual components of UE can lead to wide ranges in incidence estimates and MFs. To estimate predictive intervals, uncertainty can be modelled by incorporating, for example, either uniform or pert probability distributions (rather than fixed point estimates), and using techniques such as Monte Carlo simulations [4, 50, 80, 84].
One systematic way to decide on the best method for estimating UE or choosing MFs is to use the Delphi method or expert consensus. In the next phase of the BCoDE-project, internal ECDC experts with final input from the consortium and other external experts will create lists of the most appropriate country- and disease-specific MFs for 32 IDs. In general we assume that the most appropriate MFs should be disease-, country-, age-, and sex-specific because underestimation rates are disproportionately distributed between diseases, countries with differing surveillance systems and reporting procedures, and between demographic groups. While our literature review returned only one result with age- or sex-stratification of MFs, there are other studies that provide age- (at least for given age bands) [12, 21, 34, 116–119] and sex-specific [117, 118] MFs for general gastroenteritis and diarrhoea (i.e. unspecified pathogen). However, there remains a paucity of age- and sex-specific MFs in the literature.
Where no MF exists, it is (under certain conditions) possible to 'borrow’ or extrapolate from a disease of similar epidemiology or from the same disease in a country with a similar surveillance system, or likewise apply the same MF to a group of diseases (as demonstrated by Mead et al. ). However, it must be acknowledged that the base value of “cases reported” (Figure 1A and B) that we seek to adjust for UE by applying an appropriate MF, may not always capture the same proportion of infections that have occurred or provide comparable information of disease incidence estimates for different diseases or countries. Therefore, borrowing MFs (particularly from different countries and especially from different disease groups (e.g. taking MFs for STIs and applying to gastroenteric disease data)) is not a favoured method owing to the inherent heterogeneity of national surveillance systems in terms of population covered, test sensitivity and specificity, the source of data (physician, laboratory, hospital or other) and surveillance type (whether compulsory versus voluntary reporting of positive results or cases, comprehensive versus sentinel, active versus passive surveillance, case-based versus aggregated reporting ).
Here we did not address UE in the mortality reporting chain. Similar to the surveillance pyramid for morbidity data, the tip of mortality pyramid represents the cases correctly reported. The wide base of the pyramid contains data of all deaths including those that are ascertained and those that are not. Whilst it is expected that in a European setting under-ascertainment of deaths is rare (if not irrelevant), underreporting or over-reporting of mortality events due to certain diseases or conditions is not. The number of deaths may be well reported, but there is considerable misclassification of the cause of death. This misclassification may be deliberate in countries without nationalised healthcare such as the United States, where reimbursement by private insurance may be related to the ICD code used for the primary cause of death. Elsewhere, there may be other reasons to misclassify the cause of death, such as government targets and pressure to reduce the number of deaths due to a certain cause. In addition, often lacking are additional details relating to underlying conditions and sequelae that an individual died with (e.g. secondary and tertiary causes) but not necessarily of (i.e. the primary cause of death) [13, 121, 122]. For example, chronic conditions with infectious causes (e.g. liver cirrhosis) are often not counted as sequelae deaths and therefore the surveillance system may underestimate the long-term burden due to the infection that led to the sequelae.
UE masks the true magnitude of disease incidence and reduces the efficiency of the notification system and surveillance potential . In some instances, such as BoD estimates for the BCoDE-study and for comparing the impact of diseases between countries, it is necessary to quantify and adjust for UE. After correction for UE, preferably by age and sex, surveillance and notification data become a better estimate for evidence-based and comparable disease burden estimations. However, since adjusting for UE results in higher disease burden estimates and can result in diseases with differing ranks of public health importance compared with unadjusted surveillance data; care should be taken to clearly communicate both the need for such adjustment and the methodologies applied to adjust the raw data. The results presented here confirm that UR and UA have a significant impact resulting in UE of surveillance and notification data in our examples for salmonellosis and campylobacteriosis. To a varying extent, this is also true for all other pathogens in the BCoDE-study. The BCoDE-project is currently compiling and verifying estimates of UE and MFs derived from extensive literature reviews for 32 IDs. Here, we have presented several viable approaches for estimating UE and MFs of salmonellosis and campylobacteriosis although the best option will undoubtedly vary between countries.
Keramarou M, Evans MR: Completeness of infectious disease notification in the United Kingdom: a systematic review. J Infect. 2012, 64 (6): 555-564. 10.1016/j.jinf.2012.03.005.
Wheeler JG, Sethi D, Cowden JM, Wall PG, Rodrigues LC, Tompkins DS, Hudson MJ, Roderick PJ: Study of infectious intestinal disease in England: rates in the community, presenting to general practice, and reported to national surveillance. BMJ. 1999, 318 (7190): 1046-1050. 10.1136/bmj.318.7190.1046.
Kretzschmar M, Mangen M-JJ, Pinheiro P, Jahn B, Fèvre EM, Longhi S, Lai T, Havelaar AH, Stein C, Cassini A, et al: New methodology for estimating the burden of infectious diseases in Europe. PLoS Med. 2012, 9 (4): e1001205-10.1371/journal.pmed.1001205.
Mangen M-JJ, Plass D, Havelaar AH, Gibbons CL, Cassini A, Mühlberger N, van Lier A, Haagsma JA, Brooke RJ, Lai T, et al: The Pathogen- and Incidence-Based DALY Approach: An Appropriated Methodology for Estimating the Burden of Infectious Diseases. PLoS ONE. 2013, 8 (11): e79740-10.1371/journal.pone.0079740.
Plass D, Mangen M-JJ, Havelaar AH, Gibbons C, Haagsma J, Jahn B, Lai T, van Lier A, Longhi S, McDonald SA, et al: The incidence-based and pathogen-based disability-adjusted life-years approach for measuring infectious disease burden in Europe: the burden of communicable diseases in Europe (BCoDE) project. Lancet. 2013, 381: S114-
McDonald SA, van Lier A, Plass D, Kretzschmar ME: The impact of demographic change on the estimated future burden of infectious diseases: examples from hepatitis B and seasonal influenza in the Netherlands. BMC Public Health. 2012, 12 (1046): 1471-2458.
Hardnett FP, Hoekstra RM, Kennedy M, Charles L, Angulo FJ, for the Emerging Infections Program FoodNet Working Group: Epidemiologic issues in study design and data analysis related to FoodNet activities. Clin Infect Dis. 2004, 38 (Supplement 3): S121-S126.
MacDougall L, Majowicz S, Dore K, Flint J, Thomas K, Kovacs S, Sockett P: Under-reporting of infectious gastrointestinal illness in British Columbia, Canada: who is counted in provincial communicable disease statistics?. Epidemiol Infect. 2008, 136 (02): 248-256.
Haagsma JA, Geenen PL, Ethelberg S, Fetsch A, Hansdotter F, Jansen A, Korsgaard H, O’Brien SJ, Scavia G, Spitznagel H, et al: Community incidence of pathogen-specific gastroenteritis: reconstructing the surveillance pyramid for seven pathogens in seven European Union member states. Epidemiol Infect. 2012, 27: 1-15.
European Centre for Disease Prevention and Control (ECDC): Report: Surveillance and Prevention of Hepatitis B and C in Europe. 2010, Stockholm, Sweden: ECDC
O’Brien S, Rait G, Hunter P, Gray J, Bolton F, Tompkins D, McLauchlin J, Letley L, Adak G, Cowden J, et al: Methods for determining disease burden and calibrating national surveillance data in the United Kingdom: the second study of infectious intestinal disease in the community (IID2 study). BMC Med Res Methodol. 2010, 10 (1): 39-10.1186/1471-2288-10-39.
Sethi D, Wheeler J, Rodrigues LC, Fox S, Roderick P: Investigation of under-ascertainment in epidemiological studies based in general practice. Int J Epidemiol. 1999, 28 (1): 106-112. 10.1093/ije/28.1.106.
Khosravi A, Rao C, Naghavi M, Taylor R, Jafari N, Lopez AD: Impact of misclassification on measures of cardiovascular disease mortality in the Islamic Republic of Iran: a cross-sectional study. Bull World Health Organ. 2008, 86 (9): 688-696. 10.2471/BLT.07.046532.
Crowcroft NS, Andrews N, Rooney C, Brisson M, Miller E: Deaths from pertussis are underestimated in England. Arch Dis Child. 2002, 86 (5): 336-338. 10.1136/adc.86.5.336.
Martin-Ampudia M, Mariscal A, Lopez-Gigosos RM, Mora L, Fernandez-Crehuet J: Under-notification of cryptosporidiosis by routine clinical and laboratory practices among non-hospitalised children with acute diarrhoea in Southern Spain. Infection. 2012, 40 (2): 113-119. 10.1007/s15010-011-0188-3.
Yuguero O, Serna MC, Real J, Galvan L, Riu P, Godoy P: [Using treatment compliance to determine the under-notification of tuberculosis in a health region for the years 2007–2009]. Aten Primaria. 2012, 44 (12): 703-708. 10.1016/j.aprim.2012.06.001.
Doyle TJ, Glynn MK, Groseclose SL: Completeness of notifiable infectious disease reporting in the united states: an analytical literature review. Am J Epidemiol. 2002, 155 (9): 866-874. 10.1093/aje/155.9.866.
European Centre for Disease Prevention and Control (ECDC): Report: Methodology Protocol for Estimating Burden of Communicable Diseases. 2010, Stockholm, Sweden: ECDC, 20-25.
Reed C, Angulo FJ, Swerdlow DL, Lipsitch M, Meltzer MI, Jernigan D, Finelli L: Estimates of the prevalence of pandemic (H1N1) 2009, United States, April-July 2009. Emerging infectious diseases. 2009, 15 (12): 2004-2007. 10.3201/eid1512.091413.
del Beccaro MA, Brownstein DR, Cummings P, Goldoft MJ, Quan L: Outbreak of Escherichia coli O157:H7 hemorrhagic colitis and hemolytic uremic syndrome: effect on use of a pediatric emergency department. Ann Emerg Med. 1995, 26 (5): 598-603. 10.1016/S0196-0644(95)70011-0.
van Cauteren D, de Valk H, Vaux S, le Strat Y, Vaillant V: Burden of acute gastroenteritis and healthcare-seeking behaviour in France: a population-based study. Epidemiol Infect. 2012, 140 (4): 697-705. 10.1017/S0950268811000999.
Szczepura A: Access to health care for ethnic minority populations. Postgrad Med J. 2005, 81 (953): 141-147. 10.1136/pgmj.2004.026237.
Thomas VN, Saleem T, Abraham R: Barriers to effective uptake of cancer screening among Black and minority ethnic groups. Int J Palliat Nurs. 2005, 11 (11): 564-571.
European Centre for Disease Prevention and Control: Technical Report - Migrant Health: Background Note to the 'ECDC Report on Migration and Infectious Diseases in the EU’. 2009, Stockholm, Sweden: ECDC
Abdool Karim SS, Dilraj A: Reasons for under-reporting of notifiable conditions. S Afr Med J. 1996, 86 (7): 834-836.
Durrheim DN, Thomas J: General practice awareness of notifiable infectious diseases. Public Health. 1994, 108 (4): 273-278. 10.1016/S0033-3506(94)80006-5.
Spedding RL, Jenkins MG, O’Reilly SA: Notification of infectious diseases by junior doctors in accident and emergency departments. J Accid Emerg Med. 1998, 15 (2): 102-104. 10.1136/emj.15.2.102.
Voss S: How much do doctors know about the notification of infectious diseases?. BMJ. 1992, 304 (6829): 755-755. 10.1136/bmj.304.6829.755.
Hsieh Y-H, Kuo M-J, Hsieh T-C, Lee H-C: Underreporting and underestimation of gonorrhea cases in the Taiwan National Gonorrhea Notifiable Disease System in the Tainan region: evaluation by a pilot physician-based sentinel surveillance on Neisseria gonorrhoeae infection. Int J Infect Dis. 2009, 13 (6): e413-e419. 10.1016/j.ijid.2009.02.006.
Riley LW, Finch MJ: Results of the first year of national surveillance of campylobacter infections in the United States. J Infect Dis. 1985, 151 (5): 956-959. 10.1093/infdis/151.5.956.
Yoder JS, Blackburn BG, Craun GC, Hill V, Levy DA, Chen N, Lee SH, Calderon RL, Beach MJ: Surveillance for waterborne-disease outbreaks associated with recreational water –- United States, 2001–2002. MMWR Surveill Summ. 2004, 53: 1-22.
Hoxie NJ, Davis JP, Vergeront JM, Nashold RD, Blair KA: Cryptosporidiosis-associated mortality following a massive waterborne outbreak in Milwaukee, Wisconsin. Am J Public Health. 1997, 87 (12): 2032-2035. 10.2105/AJPH.87.12.2032.
Hall G, Yohannes K, Raupach J, Becker N, Kirk M: Estimating community incidence of Salmonella, Campylobacter, and Shiga toxin-producing Escherichia coli infections, Australia. Emerg Infect Dis. 2008, 14 (10): 1601-1609. 10.3201/eid1410.071042.
van den Brandhof WE, Bartelds AI, Koopmans MP, van Duynhoven YT: General practitioner practices in requesting laboratory tests for patients with gastroenteritis in the Netherlands, 2001–2002. BMC Fam Pract. 2006, 7: 56-10.1186/1471-2296-7-56.
Scholten JN, de Vlas SJ, Zaleskis R: Under-reporting of HIV infection among cohorts of TB patients in the WHO European Region, 2003–2004. Int J Tuberc Lung Dis. 2008, 12: S85-S91.
Borges-Costa J, Matos C, Pereira F: Sexually transmitted infections in pregnant adolescents: prevalence and association with maternal and foetal morbidity. J Eur Acad Dermatol Venereol. 2012, 26 (8): 972-975. 10.1111/j.1468-3083.2011.04194.x.
Adams OP, Carter AO, Prussia P, McIntyre G, Branch SL: Risk behaviour, healthcare access and prevalence of infection with Chlamydia trachomatis and Neisseria gonorrhoeae in a population-based sample of adults in Barbados. Sex Transm Infect. 2008, 84 (3): 192-194. 10.1136/sti.2007.028126.
Lewis DA, Pillay C, Mohlamonyane O, Vezi A, Mbabela S, Mzaidume Y, Radebe F: The burden of asymptomatic sexually transmitted infections among men in Carletonville, South Africa: implications for syndromic management. Sex Transm Infect. 2008, 84 (5): 371-376. 10.1136/sti.2008.029751.
Farley TA, Cohen DA, Elkins W: Asymptomatic sexually transmitted diseases: the case for screening. Prev Med. 2003, 36 (4): 502-509. 10.1016/S0091-7435(02)00058-0.
Schachter J, Stoner E, Moncada J: Screening for chlamydial infections in women attending family planning clinics. West J Med. 1983, 138 (3): 375-379.
Klouman E, Masenga EJ, Sam NE, Klepp KI: Asymptomatic gonorrhoea and chlamydial infection in a population-based and work-site based sample of men in Kilimanjaro, Tanzania. Int J STD AIDS. 2000, 11 (10): 666-674. 10.1258/0956462001915039.
Benzaken AS, Galban EG, Antunes W, Dutra JC, Peeling RW, Mabey D, Salama A: Diagnosis of gonococcal infection in high risk women using a rapid test. Sex Transm Infect. 2006, 82: V26-V28. 10.1136/sti.2006.022566.
Forni J, Miles K, Hamill M: Microscopy detection of rectal gonorrhoea in asymptomatic men. Int J STD AIDS. 2009, 20 (11): 797-798. 10.1258/ijsa.2009.009186.
Hong Y, Fang X, Zhou Y, Zhao R, Li X: Factors associated with sexually transmitted infection underreporting among female sex workers in China. J Womens Health (Larchmt). 2011, 20 (1): 129-136. 10.1089/jwh.2010.2139.
Sypsa V, Bonovas S, Tsiodras S, Baka A, Efstathiou P, Malliori M, Panagiotopoulos T, Nikolakopoulos I, Hatzakis A: Estimating the disease burden of 2009 pandemic influenza A(H1N1) from surveillance and household surveys in Greece. PLoS One. 2011, 6 (6): e20593-10.1371/journal.pone.0020593.
Centers for Disease Control and Prevention (CDC): Self-reported influenza-like illness during the 2009 H1N1 influenza pandemic--United States, September 2009 - March 2010. MMWR Morb Mortal Wkly Rep. 2011, 60 (2): 37-41.
Tam CC, Rodrigues LC, Viviani L, Dodds JP, Evans MR, Hunter PR, Gray JJ, Letley LH, Rait G, Tompkins DS, et al: Longitudinal study of infectious intestinal disease in the UK (IID2 study): incidence in the community and presenting to general practice. Gut. 2012, 61 (1): 69-77. 10.1136/gut.2011.238386.
Mizoguchi Y, Suzuki E, Tsuchida H, Tsuda T, Yamamoto E, Nakase K, Doi H: Outbreak of Salmonella Braenderup infection originating in boxed lunches in Japan in 2008. Acta Med Okayama. 2011, 65 (2): 63-69.
Voetsch AC, van Gilder TJ, Angulo FJ, Farley MM, Shallow S, Marcus R, Cieslak PR, Deneen VC, Tauxe RV: FoodNet estimate of the burden of illness caused by nontyphoidal Salmonella infections in the United States. Clin Infect Dis. 2004, 38 (Suppl 3): S127-134.
Kubota K, Kasuga F, Iwasaki E, Inagaki S, Sakurai Y, Komatsu M, Toyofuku H, Angulo FJ, Scallan E, Morikawa K: Estimating the burden of acute gastroenteritis and foodborne illness caused by campylobacter, salmonella, and vibrio parahaemolyticus by using population-based telephone survey data, Miyagi prefecture, Japan, 2005 to 2006. J Food Prot. 2011, 74 (10): 1592-1598. 10.4315/0362-028X.JFP-10-387.
Kuusi M, Aavitsland P, Gondrosen B, Kapperud G: Incidence of gastroenteritis in Norway–a population-based survey. Epidemiol Infect. 2003, 131 (1): 591-597. 10.1017/S0950268803008744.
Arias C, Sala MR, Dominguez A, Bartolome R, Benavente A, Veciana P, Pedrol A, Hoyo G: Waterborne epidemic outbreak of Shigella sonnei gastroenteritis in Santa Maria de Palautordera, Catalonia, Spain. Epidemiol Infect. 2006, 134 (3): 598-604. 10.1017/S0950268805005121.
Leder K, Sinclair M, Forbes A, Wain D: Household clustering of gastroenteritis. Epidemiol Infect. 2009, 137 (12): 1705-1712. 10.1017/S0950268809990124.
Wilking H, Spitznagel H, Werber D, Lange C, Jansen A, Stark K: Acute gastrointestinal illness in adults in Germany: a population-based telephone survey. Epidemiol Infect. 2013, 1: 1-11.
Adak GK, Long SM, O’Brien SJ: Trends in indigenous foodborne disease and deaths, England and Wales: 1992 to 2000. Gut. 2002, 51 (6): 832-841. 10.1136/gut.51.6.832.
Kemmeren JM, Mangen M-JJ, van Duynhoven YTHP, Havelaar AH: Report: Priorization of foodborne pathogens: disease burden and costs of selected enteric pathogens. 2006, Bilthoven: National Institute for Public Health and the Environment
Haagsma JA, Siersema PD, de Wit NJ, Havelaar AH: Disease burden of post-infectious irritable bowel syndrome in The Netherlands. Epidemiol Infect. 2010, 138 (11): 1650-1656. 10.1017/S0950268810000531.
Havelaar AH, van Pelt W, Ang CW, Wagenaar JA, van Putten JP, Gross U, Newell DG: Immunity to Campylobacter: its role in risk assessment and epidemiology. Crit Rev Microbiol. 2009, 35 (1): 1-22. 10.1080/10408410802636017.
Vijgen SMC, Mangen M-JJ, Koortbeek LM, van Duynhoven YTHP, Havelaar AH: Disease Burden and Related Costs of two Protozoan Pathogens. 2007, Bilthoven: National Institute for Public Health and the Environment
Havelaar AH, van Duynhoven YT, Nauta MJ, Bouwknegt M, Heuvelink AE, de Wit GA, Nieuwenhuizen MG, van de Kar NC: Disease burden in The Netherlands due to infections with Shiga toxin-producing Escherichia coli O157. Epidemiol Infect. 2004, 132 (3): 467-484. 10.1017/S0950268804001979.
Mangen M-JJ, Havelaar AH, A.J.A.M. Bernsen R, Van Koningsveld R, De Wit GA: The costs of human Campylobacter infections and sequelae in the Netherlands: A DALY and cost-of-illness approach. Food Economics - Acta Agriculturae Scandinavica, Section C. 2005, 2 (1): 35-51. 10.1080/16507540510033451.
van Pelt W, de Wit MAS, Wannet WJB, Ligtvoet EJJ, Widdowson MA, van Duynhoven Y: Laboratory surveillance of bacterial gastroenteric pathogens in The Netherlands, 1991–2001. Epidemiol Infect. 2003, 130 (3): 431-441.
Havelaar AH, Haagsma JA, Mangen MJ, Kemmeren JM, Verhoef LP, Vijgen SM, Wilson M, Friesema IH, Kortbeek LM, van Duynhoven YT, et al: Disease burden of foodborne pathogens in the Netherlands, 2009. Int J Food Microbiol. 2012, 156 (3): 231-238. 10.1016/j.ijfoodmicro.2012.03.029.
Muller L, Korsgaard H, Ethelberg S: Burden of acute gastrointestinal illness in Denmark 2009: a population-based telephone survey. Epidemiol Infect. 2012, 140 (2): 290-298. 10.1017/S0950268811000471.
Baumann-Popczyk A, Sadkowska-Todys M, Rogalska J, Stefanoff P: Incidence of self-reported acute gastrointestinal infections in the community in Poland: a population-based study. Epidemiol Infect. 2012, 140 (7): 1173-1184. 10.1017/S0950268811001853.
Fitzgerald M, Scallan E, Collins C, Crowley D, Daly L, Devine M, Igoe D, Quigley T, Smyth B: Results of the first population based telephone survey of acute gastroenteritis in Northern Ireland and the Republic of Ireland. Euro Surveill. 2004, 8 (18): 2456-
McMahon BJ, Holck P, Bulkow L, Snowball M: Serologic and clinical outcomes of 1536 Alaska Natives chronically infected with hepatitis B virus. Ann Intern Med. 2001, 135 (9): 759-768. 10.7326/0003-4819-135-9-200111060-00006.
Hagan H, Snyder N, Hough E, Yu T, McKeirnan S, Boase J, Duchin J: Case-reporting of acute hepatitis B and C among injection drug users. J Urban Health. 2002, 79 (4): 579-585. 10.1093/jurban/79.4.579.
de Melker HE, Versteegh FGA, Schellekens JFP, Teunis PFM, Kretzschmar M: The incidence of Bordetella pertussis infections estimated in the population from a combination of serological surveys. J Infect. 2006, 53 (2): 106-113. 10.1016/j.jinf.2005.10.020.
de Melker H, Pebody RG, Edmunds WJ, Lévy-Bruhl D, Valle M, Rota MC, Salmaso S, van den Hof S, Berbers G, Saliou P, et al: The seroepidemiology of measles in Western Europe. Epidemiol Infect. 2001, 126 (02): 249-259.
Simms I, Rogers P, Catchpole M, McGarrigle CA, Nicoll A: Trends in undiagnosed HIV-1 infection among attenders at genitourinary medicine clinics, England, Wales, and Northern Ireland: 1990–6. Sex Transm Infect. 1999, 75 (5): 332-336. 10.1136/sti.75.5.332.
Teunis PF, van Eijkeren JC, Ang CW, van Duynhoven YT, Simonsen JB, Strid MA, van Pelt W: Biomarker dynamics: estimating infection rates from serological data. Stat Med. 2012, 31 (20): 2240-2248. 10.1002/sim.5322.
Mangen M-JJ, Batz MB, Kasbohrer A, Hald T, Morris JG, Taylor M, Havelaar AH: Integrated approaches for the public health prioritization of foodborne and zoonotic pathogens. Risk Anal. 2010, 30 (5): 782-797. 10.1111/j.1539-6924.2009.01291.x.
Ekdahl K, Giesecke J: Travellers returning to Sweden as sentinels for comparative disease incidence in other European countries, campylobacter and giardia infection as examples. Euro Surveill. 2004, 9 (9): 6-9.
de Jong B, Ekdahl K: The comparative burden of salmonellosis in the European Union member states, associated and candidate countries. BMC Public Health. 2006, 6: 4-10.1186/1471-2458-6-4.
Havelaar AH, Ivarsson S, Lofdahl M, Nauta MJ: Estimating the true incidence of campylobacteriosis and salmonellosis in the European Union, 2009. Epidemiol Infect. 2013, 13: 1-10.
Simonsen J, Molbak K, Falkenhorst G, Krogfelt KA, Linneberg A, Teunis PF: Estimation of incidences of infectious diseases based on antibody measurements. Stat Med. 2009, 28 (14): 1882-1895. 10.1002/sim.3592.
Gallay A, Vaillant V, Bouvet P, Grimont P, Desenclos JC: How many foodborne outbreaks of Salmonella infection occurred in France in 1995? Application of the capture-recapture method to three surveillance systems. Am J Epidemiol. 2000, 152 (2): 171-177. 10.1093/aje/152.2.171.
Jelastopulu E, Merekoulias G, Alexopoulos EC: Underreporting of communicable diseases in the prefecture of Achaia, western Greece, 1999–2004 - missed opportunities for early intervention. Euro Surveill. 2010, 15 (21): 19579-
Gkogka E, Reij MW, Havelaar AH, Zwietering MH, Gorris LGM: Risk-based estimate of effect of foodborne diseases on public health. Greece Emerg Infect Dis. 2011, 17 (9): 1581-1590. 10.3201/eid1709.101766.
Perez-Ciordia I, Ferrero M, Sanchez E, Abadias M, Martinez-Navarro F, Herrera D: Salmonella enteritis in Huesca. 1996–1999. Enferm Infecc Microbiol Clin. 2002, 20 (1): 16-21. 10.1016/S0213-005X(02)72725-0.
Jansson A, Arneborn M, Ekdahl K: Sensitivity of the Swedish statutory surveillance system for communicable diseases 1998. Epidemiol Infect. 2005, 133 (03): 401-407. 10.1017/S0950268804003632.
Mead PS, Slutsker L, Dietz V, McCaig LF, Bresee JS, Shapiro C, Griffin PM, Tauxe RV: Food-related illness and death in the United States. Emerg Infect Dis. 1999, 5 (5): 607-10.3201/eid0505.990502.
Scallan E, Hoekstra RM, Angulo FJ, Tauxe RV, Widdowson MA, Roy SL, Jones JL, Griffin PM: Foodborne illness acquired in the United States–major pathogens. Emerg Infect Dis. 2011, 17 (1): 7-15. 10.3201/eid1701.P11101.
Thomas MK, Majowicz SE, Sockett PN, Aamir F, Pollari F, Doré K, Flint JA, Edge VL: Estimated numbers of community cases of illness Due to salmonella, campylobacter and verotoxigenic escherichia coli: pathogen-specific community rates. Can J Infect Dis Med Microbiol. 2006, 17 (4): 229-234.
Kuusi M, Nuorti JP, Hanninen ML, Koskela M, Jussila V, Kela E, Miettinen I, Ruutu P: A large outbreak of campylobacteriosis associated with a municipal water supply in Finland. Epidemiol Infect. 2005, 133 (4): 593-601. 10.1017/S0950268805003808.
McCarty DJ, Tull ES, Moy CS, Kwoh CK, LaPorte RE: Ascertainment corrected rates: applications of capture-recapture methods. Int J Epidemiol. 1993, 22 (3): 559-565. 10.1093/ije/22.3.559.
Trotter C, Samuelsson S, Perrocheau A, de Greeff S, de Melker H, Heuberger S, Ramsay M: Ascertainment of meningococcal disease in Europe. Euro Surveill. 2005, 10 (12): 247-250.
Hook EB, Regal RR: Capture-recapture methods in epidemiology: methods and limitations. Epidemiol Rev. 1995, 17 (2): 243-264.
van Hest NA, Grant AD, Smit F, Story A, Richardus JH: Estimating infectious diseases incidence: validity of capture-recapture analysis and truncated models for incomplete count data. Epidemiol Infect. 2008, 136 (1): 14-22.
Jelastopulu E, Alexopoulos EC, Venieri D, Tsiros G, Komninou G, Constantinidis TC, Chrysanthopoulos K: Substantial underreporting of tuberculosis in West Greece: implications for local and national surveillance. Euro Surveill. 2009, 14 (11): 19152-
Farchi S, Mantovani J, Borgia P, Giorgi Rossi P: Tuberculosis incidence, hospitalisation prevalence and mortality in Lazio, Italy, 1997–2003. Int J Tuberc Lung Dis. 2008, 12 (2): 193-198.
Baussano I, Bugiani M, Gregori D, van Hest R, Borraccino A, Raso R, Merletti F: Undetected burden of tuberculosis in a low-prevalence area. Int J Tuberc Lung Dis. 2006, 10 (4): 415-421.
van Hest NA, Smit F, Baars HW, de Vries G, de Haas PE, Westenend PJ, Nagelkerke NJ, Richardus JH: Completeness of notification of tuberculosis in The Netherlands: how reliable is record-linkage and capture-recapture analysis?. Epidemiol Infect. 2007, 135 (6): 1021-1029. 10.1017/S0950268806007540.
van Loenhout-Rooyackers JH, Leufkens HGM, Hekster YA, Kalisvaart NA: Pyrazinamide use as a method of estimating under-reporting of tuberculosis. Int J Tuberc Lung Dis. 2001, 5 (12): 1156-1160.
Cojocaru C, van Hest NA, Mihaescu T, Davies PD: Completeness of notification of adult tuberculosis in Iasi County, Romania: a capture-recapture analysis. Int J Tuberc Lung Dis. 2009, 13 (9): 1094-1099.
Teo SSS, Alfaham M, Evans MR, Watson JM, Riordan A, Sonnenberg P, Clark J, Hayward A, Sharland M, Moore-Gillon J, et al: An evaluation of the completeness of reporting of childhood tuberculosis. Eur Respir J. 2009, 34 (1): 176-179. 10.1183/09031936.00031808.
van Hest NA, Story A, Grant AD, Antoine D, Crofts JP, Watson JM: Record-linkage and capture-recapture analysis to estimate the incidence and completeness of reporting of tuberculosis in England 1999–2002. Epidemiol Infect. 2008, 136 (12): 1606-1616. 10.1017/S0950268808000496.
Pillaye J, Clarke A: An evaluation of completeness of tuberculosis notification in the United Kingdom. BMC Public Health. 2003, 3 (1): 31-10.1186/1471-2458-3-31.
Mancuso JD, Tobler SK, Eick AA, Olsen CH: An evaluation of the completeness and accuracy of active tuberculosis reporting in the United States military. Int J Tuberc Lung Dis. 2010, 14 (10): 1310-1315.
Curtis AB, McCray E, McKenna M, Onorato IM: Completeness and timeliness of tuberculosis case reporting: a multistate study. Am J Prev Med. 2001, 20 (2): 108-112. 10.1016/S0749-3797(00)00284-1.
Dijkstra F, Donker GA, Wilbrink B, van Gageldonk-Lafeber AB, van der Sande MA: Long time trends in influenza-like illness and associated determinants in The Netherlands. Epidemiol Infect. 2009, 137 (4): 473-479. 10.1017/S095026880800126X.
McDonald SA, Presanis AM, de Angelis D, van der Hoek W, Hooiveld M, Donker G, Kretzschmar ME: An evidence synthesis approach to estimating the incidence of seasonal influenza in the Netherlands. Influenza Other Respir Viruses. 2014, 8 (1): 33-41. 10.1111/irv.12201.
Stein CE, Birmingham M, Kurian M, Duclos P, Strebel P: The global burden of measles in the year 2000 - a model that uses country-specific indicators. J Infect Dis. 2003, 187 (SUPPL. 1): S8-S14.
Scallan E, Mahon BE, Hoekstra RM, Griffin PM: Estimates of illnesses, hospitalizations and deaths caused by major bacterial enteric pathogens in young children in the United States. Pediatr Infect Dis J. 2013, 32 (3): 217-221.
Thomas MK, Murray R, Flockhart L, Pintar K, Pollari F, Fazil A, Nesbitt A, Marshall B: Estimates of the burden of foodborne illness in Canada for 30 specified pathogens and unspecified agents, circa 2006. Foodborne Pathog Dis. 2013, 10 (7): 639-648. 10.1089/fpd.2012.1389.
le Vu S, le Strat Y, Barin F, Pillonel J, Cazein F, Bousquet V, Brunet S, Thierry D, Semaille C, Meyer L, et al: Population-based HIV-1 incidence in France, 2003–08: a modelling analysis. Lancet Infect Dis. 2010, 10 (10): 682-687. 10.1016/S1473-3099(10)70167-5.
Armstrong GL, Bell BP: Hepatitis A virus infections in the United States: model-based estimates and implications for childhood immunization. Pediatrics. 2002, 109 (5): 839-845. 10.1542/peds.109.5.839.
Aledort JE, Ronald A, Rafael ME, Girosi F, Vickerman P, le Blancq SM, Landay A, Holmes K, Ridzon R, Hellmann N, et al: Reducing the burden of sexually transmitted infections in resource-limited settings: the role of improved diagnostics. Nature. 2006, 1: 59-72.
O’Connor JB, Imperiale TF, Singer ME: Cost-effectiveness analysis of hepatitis A vaccination strategies for adults. Hepatology. 1999, 30 (4): 1077-1081. 10.1002/hep.510300422.
Albert I, Espié E, de Valk H, Denis J-B: A Bayesian evidence synthesis for estimating campylobacteriosis prevalence. Risk Anal. 2011, 31 (7): 1141-1155. 10.1111/j.1539-6924.2010.01572.x.
Bender JB, Smith KE, McNees AA, Rabatsky-Ehr TR, Segler SD, Hawkins MA, Spina NL, Keene WE, Kennedy MH, van Gilder TJ, et al: Factors affecting surveillance data on Escherichia coli O157 infections collected from FoodNet sites, 1996–1999. Clin Infect Dis. 2004, 15 (38): S157-164.
Birrell PJ, Ketsetzis G, Gay NJ, Cooper BS, Presanis AM, Harris RJ, Charlett A, Zhang X-S, White PJ, Pebody RG, et al: Bayesian modeling to unmask and predict influenza A/H1N1pdm dynamics in London. Proc Natl Acad Sci. 2011, 108 (45): 18238-18243. 10.1073/pnas.1103002108.
Presanis AM, Pebody RG, Paterson BJ, Tom BD, Birrell PJ, Charlett A, Lipsitch M, de Angelis D: Changes in severity of 2009 pandemic A/H1N1 influenza in England: a Bayesian evidence synthesis. BMJ. 2011, 8 (343): d5408-
Beutels P, Musabaev EI, van Damme P, Yasin T: The disease burden of hepatitis B in Uzbekistan. J Infect. 2000, 40 (3): 234-241. 10.1053/jinf.1998.0666.
Ziv T, Heymann AD, Azuri J, Leshno M, Cohen D: Assessment of the underestimation of childhood diarrhoeal disease burden in Israel. Epidemiol Infect. 2011, 139 (9): 1379-1387. 10.1017/S0950268810002554.
Imhoff B, Morse D, Shiferaw B, Hawkins M, Vugia D, Lance-Parker S, Hadler J, Medus C, Kennedy M, Moore MR, et al: Burden of self-reported acute diarrheal illness in FoodNet surveillance areas, 1998–1999. Clin Infect Dis. 2004, 15 (38): S219-226.
Herikstad H, Yang S, van Gilder TJ, Vugia D, Hadler J, Blake P, Deneen V, Shiferaw B, Angulo FJ: A population-based estimate of the burden of diarrhoeal illness in the United States: FoodNet, 1996–7. Epidemiol Infect. 2002, 129 (1): 9-17.
de Wit MA, Kortbeek LM, Koopmans MP, de Jager CJ, Wannet WJ, Bartelds AI, van Duynhoven YT: A comparison of gastroenteritis in a general practice-based study and a community-based study. Epidemiol Infect. 2001, 127 (3): 389-397.
Pebody RG, Hellenbrand W, D’Ancona F, Ruutu P: Pneumococcal disease surveillance in Europe. Euro Surveill. 2006, 11 (9): 171-178.
Lilienfeld DE, Stolley PD: Foundations of Epidemiology. 1994, New York: Oxford University Press, 3
Lozano R, Murray CJL, Lopez AD, Satoh T: Miscoding and misclassification of ischaemic heart disease mortality. Global Programme on Evidence for Health Policy Working Paper No 12. 2001, Geneva: World Health Organization
Brabazon ED, O’Farrell A, Murray CA, Carton MW, Finnegan P: Under-reporting of notifiable infectious disease hospitalizations in a health board region in Ireland: room for improvement?. Epidemiol Infect. 2008, 136 (02): 241-247.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/14/147/prepub
The BCoDE-project was funded by the European Centre for Disease Prevention and Control (Specific agreement No 1 to Framework Partnership Agreement GRANT/2008/003).
Alessandro Cassini and Piotr Kramarz (co-authors) are employed by the European Centre for Disease Prevention and Control, which has funded this research. The authors declare that they have no competing interest.
All authors contributed to the development of the methodology. CLG, M-JJM and ALS performed literature reviews and MF calculations. All authors read and approved the final version of the manuscript.