This article has Open Peer Review reports available.
Internet-based monitoring of influenza-like illness (ILI) in the general population of the Netherlands during the 2003–2004 influenza season
© Marquet et al; licensee BioMed Central Ltd. 2006
Received: 26 April 2006
Accepted: 04 October 2006
Published: 04 October 2006
An internet-based survey of influenza-like illness (ILI) – the Great Influenza Survey or GIS – was launched in the Netherlands in the 2003–2004 influenza season. The aim of the present study was to validate the representativeness of the GIS population and to compare the GIS data with the official ILI data obtained by Dutch GPs participating in the Dutch Sentinel Practice Network.
Direct mailings to schools and universities, and repeated interviews on television and radio, and in newspapers were used to kindle the enthusiasm of a broad section of the public for GIS. Strict symptomatic criteria for ILI were formulated with the assistance of expert institutes and only participants who responded at least five times to weekly e-mails asking them about possible ILI symptoms were included in the survey. Validation of GIS was done at different levels: 1) some key demographic (age distribution) and public health statistics (prevalence of asthma and diabetes, and influenza vaccination rates) for the Dutch population were compared with corresponding figures calculated from GIS; 2) the ILI rates in GIS were compared with the ILI consultation rates reported by GPs participating in the Dutch Sentinel Practice Network.
13,300 persons (53% of total responders), replied at least five times to weekly e-mails and were included in the survey. As expected, there was a marked under-representation of the age groups 0–10 years and 81->90 years in the GIS population, although the similarities were remarkable for most other age groups, albeit that the age groups between 21 and 70 years were slightly overrepresented. There were striking similarities between GIS and the Dutch population with regard to the prevalence of asthma (6.4% vs. 6.9%) and the influenza vaccination rates, and to a lesser degree for diabetes (2.4% vs. 3.5%). The vaccination rates in patients with asthma or diabetes, and persons older than 65 years were 68%, 85%, and 85% respectively in GIS, while the corresponding percentages in the Dutch population were 73%, 85% and 87%. There was also a marked similarity between the seasonal course of ILI measured by GIS and the GPs. Although the ILI rate in GIS was about 10 times higher, the curves followed an almost similar pattern, with peak incidences occurring in the same week.
The current study demonstrates that recruitment of a high number of persons willing to participate in on-line health surveillance is feasible. The information gathered proved to be reliable, as it paralleled the information obtained via an undisputed route. We believe that the interactive nature of GIS and the appealing subject were keys to its success.
The internet has changed the way we live and work, and may also become an important tool for epidemiological studies and health surveillance. To achieve this goal, it is mandatory to first assess the accuracy and reliability of communicating health information via internet. This may be especially relevant when there is public anxiety due to bio-terrorist threats or anticipated outbreaks of pandemic infectious diseases [1, 2].
Internet-based surveys may be conducted by means of interactive interviews or by questionnaires designed for self-completion. A major concern of web-based surveys is the non-representative nature of the internet population and the self-selection of participants and it is generally assumed that the results are likely to be biased; firstly because internet has not penetrated equally into all age-groups, and secondly because people who do respond almost certainly have different characteristics from those who do not. It has been argued, however, that internet samples may actually be more representative than traditional samples . When traditional paper-and-pencil methods were compared with internet samples, it was observed that internet samples were more diverse with respect to gender, socioeconomic status, geographic region and age. Moreover, internet surveys were less affected by non-serious responders and were found to be highly consistent with findings from traditional methods .
Influenza is an important public health problem in the industrialized world. It is associated with high general practice consultation rates, increased hospital admissions and excess deaths, and national networks for clinical and virological surveillance of influenza have existed in Europe since the 1950s. The first European influenza surveillance project was the Eurosentinel scheme, which was followed in 1992 by the ENS-CARE Influenza Early Warning Scheme and in 1996 by the European Influenza Surveillance Scheme (EISS); a total of 21 EU and 3 non-EU states are currently members of EISS . The clinical surveillance of influenza by EISS is predominantly based on reports made by general practitioners (GPs) participating in a national or regional sentinel surveillance system. They usually represent 1–5% of the GPs working in the country or region. In the Netherlands, the Dutch Sentinel Practice Network consists of a representative sample of about 60 GPs covering about 1% of the Dutch population, and is the national provider of data for EISS [6, 7]. In winter, the sentinel GPs are asked to take nose and/or throat swabs from patients with ILI for virological determination and these data function as the official data on which national and international decisions by health policy-makers are based.
An internet-based Great Influenza Survey (GIS) was launched in the Netherlands and the Dutch-speaking part of Belgium (Flanders) in 2003 . The initiative for GIS was taken by one of us (CEK), who is editor in chief of a government-sponsored Dutch Internet Knowledge Portal (Kennislink) . The objective of the portal is to make scientific information accessible to a broad public, and to kindle students' enthusiasm for science. Being a subject with a high general appeal, ILI was chosen as a vehicle to promote participation in an interactive enterprise in which the participant could experience the sensation of being a genuine scientist.
The aim of the present study was to validate and compare the internet-based GIS data with the official ILI data obtained by Dutch GPs participating in the Dutch Sentinel Practice Network during the 2003–2004 influenza season.
Design of GIS
In September 2003 the Dutch Knowledge Portal started its canvassing campaign to encourage a broad section of the public to participate in GIS. A task force that included experts on virology, public health, mathematics, science writing and informatics had been set up earlier that year to give the project a solid basis. The educational and scientific aims of the project were explained in direct mailings to schools and universities, and in repeated interviews on television and radio and in newspapers. Schools were provided with educational material on influenza to promote incorporation of GIS in lessons on biology and science, while a wealth of information on influenza, ILI and the objectives of GIS, written in appealing and accessible language, was provided on the GIS website that was launched in September 2003. Visitors to the website were invited to become GIS participants by providing personal information on gender and age, and by answering questions on health and lifestyle as summarized in table 1.
Intake questions asked of participants in GIS
-Daily occupation? (School/work/home/other)
-Means of transportation? (Car/bike/public)
-How many times per year suffering from cold/ILI? (<2/2–5/>5)
-Recently vaccinated against flu? (Yes/no)
-Do you have asthma or diabetes? (No/asthma/diabetes)
-Do you smoke? (Yes/no)
-Do you follow the recommendations of the Dutch Food and Nutrition Centre? (No/occasionally/daily)
-Do you use vitamins? (No/occasionally/every day)
-Do you follow a diet? (No/vegetarian/veganistic/low-calorie)
-Do you exercise? (<1 hour/week; 1–4 hours; >4 hours per week)
-How many people at home? (Alone/with adults/with adults and children)
-Do the children attend school or nursery? (No/nursery/school)
-Do you have pets? (No/cat(s)/dog(s)/bird(s)/other)
Questions asked about symptoms of ILI
-Sudden fever? (No/yes/don't know)
-How high was the fever? (37C/between 37C and 40C, in steps of 0.5 degree)
Did the GIS population reflect the Dutch population?
To address this question, some key demographic and public health statistics for the Dutch population were compared with corresponding figures calculated from GIS.
The demographic data of the Dutch population provided by Statistics Netherlands as of 1 January 2004  were compared with the demographic data of the GIS participants who responded frequently.
Prevalence of asthma and diabetes and influenza vaccination rates
Data on the prevalence of asthma and diabetes, obtained from the National Information Network of GPs (LINH), were compared with the prevalence of these (self- reported) conditions in the GIS population. LINH is a GP sentinel network consisting of about 80 GPs, equally distributed over the Netherlands and representative of the Dutch population . The network provides encoded patient information extracted from electronic medical records, and data on the prevalence of asthma and diabetes reported by LINH for 2003 [12, 13] were used for the present study. Vaccination uptake rates were also obtained from LINH, since Dutch GPs have been offering influenza vaccination to all their high-risk patients since 1996, in accordance with the guidelines of the Dutch College of General Practitioners. In the present study, the vaccination rates of patients with asthma or diabetes, and persons older than 65 years as supplied by LINH in 2003, were compared with the rates of (self-reported) vaccination of these groups in GIS .
ILI rates during the 2003–2004 influenza season, comparison between GIS and the Dutch Sentinel Practice Network
Fifty-six GPs participated in the Dutch Sentinel Practice Network in 2003–2004. As they have been doing since 1970, participating GPs reported new cases of ILI each week, using specific forms that were sent to our Institute by mail or fax. The number of ILI consultations per week was taken as the numerator and the total number of patients in the participating sentinel practices (1% of the Dutch population) as the denominator to calculate the consultation rates for ILI. The Dutch data are placed weekly on the EISS website, together with the data from other countries joined together in EISS. The published weekly consultation rates (incidence)/100,000 occurring in the 2003–2004 influenza season  were used for the present study. The daily incidences of ILI calculated by the computer for GIS were converted into weekly incidences to enable the comparison between EISS and GIS.
The study was carried out according to Dutch legislation on privacy. The privacy regulation of the study was approved by the Dutch Data Protection Authority. According to Dutch legislation, obtaining informed consent is not obligatory for observational studies.
Comparison of demographic data
Comparison between distribution of gender and age in GIS and the Dutch population
Comparison of the prevalence of asthma and diabetes, and influenza vaccination rates in GIS and LINH population
Comparison between GIS and LINH population with regard to the prevalence of asthma and diabetes, and influenza vaccination rates
Vaccination rate pts. with asthma
Vaccination rate pts. with diabetes
Vaccination rate total population
Vaccination rate 65+
Comparison between ILI rates in GIS and the Dutch Sentinel Practice Network
According to estimates made in 2003 by Statistics Netherlands, 6 out of 10 inhabitants of the Netherlands older than 12 years were regular internet users. Internet usage was high (75%–85%) between the age of 12–45 years, steadily decreasing thereafter by 10–15% per decade. About 5–8% of elderly persons over 75 years still used internet regularly. Internet usage by men was slightly higher than by women in 2003 and the higher the level of education, the higher the use . Where the accrual of persons willing to participate in GIS was concerned, the enterprise was definitely a success. Following substantial recruitment efforts in the media, almost 26,000 persons replied to the requests for participation by answering the intake questions at least, and 53% of these became serious participants. This percentage compares favorably with the 51.2% of serious participants who finished a Swedish internet survey on sexual behavior following an open invitation via a website, and the 50%-plus response rate obtained in a population web survey on lifestyle issues reported by Augustsson et al. [16, 17]. Much lower rates have been reported for online health surveys, however, possibly because they involved less appealing subjects (than sex or influenza), had a poorer design, or were less creative in their motivation and incentive strategy [18, 19].
The serious participants in our study were not equally distributed with regard to gender and age. The Dutch population has an almost equal male-female distribution up to the 71–80 years age group. In the GIS population, however, women between 11 and 50 years were clearly overrepresented, whereas the reverse was observed after the age of 50 and men became overrepresented. Apart from gender, there was, by and large, a clear overrepresentation of GIS participants between 21 and 70 years and a marked under-representation before and after these age limits. The Swedish internet survey mentioned above found an almost identical overrepresentation of participants between 25 and 65 years and under-representation beyond these age limits . A disproportionate distribution of this kind is not a great problem for many health-related issues, however, because there is a good chance that most relevant age groups are included. On the other hand, this imbalance may be a problem for the surveillance of topics such as ILI, because the very young, and the very old to a lesser extent, are largely underrepresented, while these are precisely the age groups that suffer most from ILI. We found, nevertheless, that this concern did not translate into a difference between the course of the ILI incidence curves from the Dutch Sentinel Practice Network and GIS. The incidence of ILI in the very young and old appears to have been less than had been expected, or was outnumbered by the magnitude of the ILI incidence in the rest of the GIS population. This was supported by the fact that the similarities between the 2 curves remained the same when very young and very old patients were excluded from the data provided by the GPs (manuscript in preparation). The major difference between the two curves is that the GIS curve started from a much higher background level and remained at a higher level throughout the season.
The most likely reasons for this difference are that only some patients with ILI complaints will go to their GPs, and that GPs are more critical in making the diagnosis. Part of the difference in amplitude between the GIS curve and the GP curve may also have been due to overrepresentation of GIS participants between 25 and 65 years of age. They represent the majority of the working population, and may be less likely to consult their GP for ILI complaints than the non-working population. On the other hand, they also represent the more healthy proportion of the population. It has been estimated earlier that in the Netherlands the incidence of ILI in the community at least is 6 times higher than presented to the GP , which corresponds with the present observation that the overall difference between GP consultation rate and GIS incidence is 10-fold.
It was a remarkable and encouraging observation that the course of ILI monitored by GIS so closely resembled the official survey of ILI by GPs participating in EISS. The implications of this finding for the surveillance of ILI itself are limited, however. The monitoring of ILI and influenza by EISS is well established and has not only proved to be reliable over the years, but also to function as an important source of scientific information with regard to virology and vaccination . The key role of GPs in the EISS surveillance system ensures a high level of continuity and scrutiny, characteristics that are questionable in GIS. As far as continuity is concerned, GIS is being carried out for the third time in the Netherlands and Flanders in the 2005–2006 season and was started in Portugal for the first time . It remains to be seen how successful these surveys will continue to be in the long run.
A comparable population-based approach for ILI surveillance, in which the GP is bypassed and information is provided by callers to a helpline, (NHS Direct), has been used in the UK. NHS Direct is a nurse-led telephone helpline providing health advice 24 hours a day. On receipt of a call and description of the caller's symptoms, nurses use their judgment to select the appropriate diagnostic algorithm in their computer, leading to self-care advice or a referral to another part of the NHS . In principle, the call data, which can be analyzed daily or even by the hour, can also be used for community surveillance, including ILI. The usefulness of NHS Direct for ILI surveillance has been evaluated for two influenza seasons thus far and, although the results have been reassuring, the system was hindered by insufficient population coverage, lack of uniformity and poor definition of ILI [23, 24].
There was some indication that the incidence of ILI started to increase 1–2 weeks earlier than reported by EISS, but this could not really be substantiated, due to the small number of early events and the high background level of ILI in GIS. For a better understanding of the natural course of ILI it might have been better if GIS had started much earlier than the start of the influenza season. It might have provided us with a badly needed threshold value.
It should be noted that in GIS it took about 4 days before reliable incidence rates could be deduced from real-time data, giving the internet-based approach a head start of about 5 days on the information provided by GPs. This difference between mail and web is likely to disappear as soon as Dutch GPs start reporting their data on a real-time basis. The success of an approach of this kind has been reported for GPs participating in a real-time influenza surveillance projects in other countries [25, 26].
The major finding of the current study is that it is possible to recruit of a high number of persons, willing to participate in on-line health surveillance. This has been previously demonstrated for well-defined groups such as students , but only occasionally for the general population . Our study demonstrates that not only was recruitment successful, but also that the information provided was reliable, because it paralleled the information that was gathered via a different, acknowledged route. In addition, the demographic and health characteristics of the participants were remarkably similar to the general Dutch population, apart from some typical deviations related to internet use. It can therefore be envisioned that this type of interactive on-line surveillance is extended to other health-related issues with, preferentially, a high public appeal such as obesity, life style and stress. Actually, questions about stress and stressful behavior have been asked to participants of the third GIS that took place during the 2005–2006 influenza season .
Bälter et al. recently made some wise remarks on the appropriate use of internet as a tool for epidemiological research , but their remarks were mainly devoted to technical subjects such as length and layout of the questionnaires. The GIS study allows us to tentatively point to another cornerstone that may be important for success: feedback of information to keep the participants involved and motivated.
The current study demonstrates that recruitment of a high number of persons willing to participate in on-line health surveillance is feasible. The information gathered via internet proved to be reliable, because it paralleled the information obtained via an undisputed route. We believe that the interactive nature of GIS and the appealing subject were keys to its success.
We thank the general practitioners participating in the Dutch Sentinel Network and their staff members for providing data on the occurrence of ILI during the influenza-seasons.
- Eysenbach G, Wyatt J: Using the internet for surveys and health research. J Med Internet Res. 2002, 4: e13-View ArticlePubMedPubMed CentralGoogle Scholar
- Kittler AF, Hobbs J, Volk LA, Kreps GL, Bates DW: The internet as a vehicle to communicate health information during a public health emergency: a survey analysis involving the anthrax scare of 2001. J Med Internet Res. 2004, 6: e8-10.2196/jmir.6.1.e8.View ArticlePubMedPubMed CentralGoogle Scholar
- Ritter P, Lorig K, Laurent D, Matthews K: Internet versus mailed questionnaires: a randomized comparison. J Med Internet Res. 2005, 6: e29-View ArticleGoogle Scholar
- Gosling SD, Vazire S, Srivastava S, John OP: Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires. Am Psychol. 2004, 59: 93-104. 10.1037/0003-066X.59.2.93.View ArticlePubMedGoogle Scholar
- The European Influenza Surveillance Scheme: [http://www.eiss.org]
- Heijnen ML, Dorigo-Zetsma JW, Bartelds AI, Wilbrink B, Sprenger MJ: Surveillance of respiratory pathogens and influenza-like illnesses in general practices-The Netherlands, winter 1997–98. Euro Surveill. 1999, 7: 81-4.Google Scholar
- Marquet RL, Bartelds AIM, Kerkhof AJFM, Schellevis FG, van der Zee J: The epidemiology of suicide and attempted suicide in Dutch general practice 1983–2003. BMC Family Practice. 2005, 6: 45-10.1186/1471-2296-6-45.View ArticlePubMedPubMed CentralGoogle Scholar
- Great Influenza Survey (De Grote Griepmeting): [http://www.degrotegriepmeting.nl]
- Internet Knowledge Portal (Kennislink): [http://www.kennislink.nl]
- Statistics Netherlands (Centraal Bureau voor de Statistiek): [http://www.CBS.nl]
- Kroneman MW, van Essen GA, Tacken MA, Paget WJ, Verheij R: Does a population survey provide reliable influenza uptake rates among high-risk groups? A case-study of The Netherlands. Vaccine. 2004, 22: 2163-70. 10.1016/j.vaccine.2003.11.049.View ArticlePubMedGoogle Scholar
- Mohangoo AD, van der Linden MW, Schellevis FG, Raat H: Prevalence estimates of asthma or COPD from a health interview survey and from general practitioner registration: what's the difference?. Eur J Public Health. 2005; Sept 1; (Epub ahead of print)Google Scholar
- National Information Network of GPs: [http://www.LINH.nl]
- Tacken MA, Braspenning JC, Berende A, Hak E, De Bakker DH, Groenewegen PP, Grol RP: Vaccination of high-risk patients against influenza: impact on primary care contact rates during epidemics. Analysis of routinely collected data. Vaccine. 2004, 22: 2985-92. 10.1016/j.vaccine.2004.02.017.View ArticlePubMedGoogle Scholar
- De Jong JC, Rimmelzwaan GF, Bartelds AI, Meijer A, Fouchier RA, Osterhaus AD: The influenza season 2004/'05 with the largest epidemic of the last 5 years caused by the virus variant A/California and the composition of the vaccine for the season 2006/'06. Ned Tijdschr Geneeskd. 2005, 42: 2355-61.Google Scholar
- Augustsson Bälter K, Bälter O, Fondell E, Trolle Lageross Y: Web-based data collection of lifestyle factors in a population-based study. Epidemiology. 2006.Google Scholar
- Ross MW, Månsson SA, Daneback K, Cooper A, Tikkanen R: Biases in internet sexual health samples: Comparison of an internet sexuality survey and a national sexual health survey in Sweden. Soc Sci Med. 2005, 61: 245-52. 10.1016/j.socscimed.2005.01.019.View ArticlePubMedGoogle Scholar
- Bethell C, Fiorillo J, Lansky D, Hendryx M, Knickman J: Online consumer surveys as a methodology for assessing the quality of the United States health care system. J Med Internet Res. 2004, 6: e2-10.2196/jmir.6.1.e2.View ArticlePubMedPubMed CentralGoogle Scholar
- Im EO, Chee W: Recruitment of research participants through the internet. Comput Inform Nurs. 2004, 22: 289-97.View ArticlePubMedGoogle Scholar
- Knottnerus JA: Influenza in the Netherlands. Pharmacoeconomics. 1996, 9 (Suppl 3): 46-9.View ArticlePubMedGoogle Scholar
- Portuguese Great Influenza Survey: [http://www.gripept.net]
- Cooper DL, Smith GE, O'Brien SJ, Hollyoak VA, Baker M: What can analysis of calls to NHS Direct tell us about the epidemiology of gastrointestinal infections in the community?. J Infection. 2003, 46: 101-5. 10.1053/jinf.2002.1090.View ArticleGoogle Scholar
- Harcourt SE, Smith GE, Hollyoak VA, Joseph CA, Chaloner R, Rehman Y, Warburton F, Ejidokun OO, Watson JM, Griffith RK: Can calls to NHS Direct be used for syndromic surveillance?. Commun Dis Public Health. 2001, 3: 178-82.Google Scholar
- Cooper DL, Smith GE, Hollyoak VA, Joseph CA, Johnson L, Chaloner R: Use of NHS Direct calls for surveillance of influenza-a second year's experience. Commun Dis Public Health. 2002, 2: 127-31.Google Scholar
- Lange W, Schöttler M: Real-time influenza surveillance in Germany -Results of a pilot project. Med Microbiol Immunol. 2002, 191: 139-44. 10.1007/s00430-002-0133-2.View ArticlePubMedGoogle Scholar
- Carrat F, Flahault A, Boussard E, Farran N, Dangoumau L, Valleron AJ: Surveillance of influenza-like illness in France. The example of the 1995/1996 epidemic. J Epidemiol Community Health. 1998, 32S-38S. Suppl 1Google Scholar
- Pealer LN, Weiler RM, Morgan Pigg R, Miler D, Dorman SM: The feasibility of a web-based surveillance system to collect health risk behavior data from college students. Health Educ Behav. 2001, 28: 547-59.View ArticlePubMedGoogle Scholar
- Bälter O, Augustsson Bälter K: Demands on web survey tools for epidemiological research. Eur J Epidemiol. 2005, 20: 137-9. 10.1007/s10654-004-5099-5.View ArticlePubMedGoogle Scholar
- Calculating confidence intervals for proportions and their differences. Statistics with confidence. Edited by: Gardner MJ, Altman DG. 1989, British Medical Journal, London, 28-33.Google Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/6/242/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.