This article has Open Peer Review reports available.
The representativeness of a European multi-center network for influenza-like-illness participatory surveillance
- Pietro Cantarelli1, 2, 3,
- Marion Debin1, 2,
- Clément Turbelin1, 2,
- Chiara Poletto1, 2,
- Thierry Blanchon1, 2,
- Alessandra Falchi1, 2,
- Thomas Hanslik1, 2, 4,
- Isabelle Bonmarin5,
- Daniel Levy-Bruhl5,
- Alessandra Micheletti3,
- Daniela Paolotti6,
- Alessandro Vespignani6, 7, 8,
- John Edmunds9,
- Ken Eames9,
- Ronald Smallenburg10,
- Carl Koppeschaar10,
- Ana O Franco11,
- Vitor Faustino11,
- AnnaSara Carnahan12,
- Moa Rehn12 and
- Vittoria Colizza1, 2, 6Email author
© Cantarelli et al.; licensee BioMed Central Ltd. 2014
Received: 3 April 2014
Accepted: 11 September 2014
Published: 20 September 2014
The Internet is becoming more commonly used as a tool for disease surveillance. Similarly to other surveillance systems and to studies using online data collection, Internet-based surveillance will have biases in participation, affecting the generalizability of the results. Here we quantify the participation biases of Influenzanet, an ongoing European-wide network of Internet-based participatory surveillance systems for influenza-like-illness.
In 2011/2012 Influenzanet launched a standardized common framework for data collection applied to seven European countries. Influenzanet participants were compared to the general population of the participating countries to assess the representativeness of the sample in terms of a set of demographic, geographic, socio-economic and health indicators.
More than 30,000 European residents registered to the system in the 2011/2012 season, and a subset of 25,481 participants were selected for this study. All age classes (10 years brackets) were represented in the cohort, including under 10 and over 70 years old. The Influenzanet population was not representative of the general population in terms of age distribution, underrepresenting the youngest and oldest age classes. The gender imbalance differed between countries. A counterbalance between gender-specific information-seeking behavior (more prominent in women) and Internet usage (with higher rates in male populations) may be at the origin of this difference. Once adjusted by demographic indicators, a similar propensity to commute was observed for each country, and the same top three transportation modes were used for six countries out of seven. Smokers were underrepresented in the majority of countries, as were individuals with diabetes; the representativeness of asthma prevalence and vaccination coverage for 65+ individuals in two successive seasons (2010/2011 and 2011/2012) varied between countries.
Existing demographic and national datasets allowed the quantification of the participation biases of a large cohort for influenza-like-illness surveillance in the general population. Significant differences were found between Influenzanet participants and the general population. The quantified biases need to be taken into account in the analysis of Influenzanet epidemiological studies and provide indications on populations groups that should be targeted in recruitment efforts.
KeywordsInfluenza Surveillance Representativeness Internet data collection Participation bias Selection bias
Monitoring influenza epidemics through surveillance is essential for providing public health recommendations in areas including vaccines, antiviral susceptibility and risk assessment . At the national level, general practice (GP) sentinel surveillance schemes collate information on influenza-like-illness (ILI) of visited patients and, in some cases, collect respiratory specimens.
Alongside these well-established schemes, novel opportunities for surveillance in the general population have been opened by the advent of new technologies that promote the participation of individuals through the Internet, creating information in a bottom-up fashion outside of established practices and routines . A participatory system was introduced in The Netherlands in 2003 for ILI surveillance in the general population by means of an online platform , offering a source of disease information generated directly by the users. The system has expanded to other European countries establishing an international participatory surveillance network (Influenzanet). The network has a standardized common framework for data collection [4, 5], thus overcoming possible fragmentations in case definitions and systems design of GP surveillance across countries.
To be of value in providing information to guide health policy, the collected data need to be related to the epidemic situation in the underlying population. In agreement with recommendations for GP surveillance networks , here we evaluate the quality of the collected data by assessing the representativeness of the participating (i.e. monitored) individuals in the Influenzanet cohort. The advantage with respect to other surveillance schemes (e.g. GPs or other digital approaches of unsupervised nature, such as web search records [7, 8], online news [9, 10], or tweets ) is the ability to ask users about themselves– including geographic, demographic, mobility, socio-economic and health indicator questions; this information can be compared with national statistics. The aim is to identify possible biases to be taken into account for epidemiological analyses. Furthermore, the comparison of representativeness results across countries may guide informed strategies to improve coverage and participation of underrepresented population groups in the following seasons.
Influenzanet is a European multicenter network  for ILI surveillance in the general population through online systems. Starting the 2011/2012 season, Influenzanet was launched with a uniform and standardized data collection approach in seven European countries (The Netherlands [3, 12], Belgium (Flemish region only) [12, 13], Portugal [14, 15], Italy , United Kingdom (UK) [17, 18], Sweden , France [20, 21]), leveraging on pre-existing participatory surveillance activities . In each country, this surveillance system is coordinated by local research and public health teams and Institutions (see the Additional file 1 for further details).
Focusing on the 2011/2012 Influenzanet season, we analyzed seven national data collection campaigns that started in November 2011 and ended in April or May 2012, with few exceptions (Additional file 1: Table S1). Differences were mainly related to country-specific practical issues (e.g. launch following the Ethical approval in France, or to coincide with public health events or communications for the upcoming influenza season).
Influenzanet consists of a website with centralized information on the network and results from each participating country  that links to the national online platforms, each in the national language and with a country-specific name, but characterized by a common website template. National platforms are used to register participants, to give them access to their account where they can upload information, and to publish summary surveillance results in real time.
Participation is voluntary and anonymous, and open to all residents of the countries composing the multi-center network (in France, overseas territories and French individuals under 18 years old were not considered, the latter due to regulatory constraints applied to the first season only). Recruitment occurred with the help of press releases of the supporting institutions, media communications, specific advertising events (e.g. schools activities or science fairs), and through emails and word of mouth. More details can be found on the national platforms [12, 14, 16, 18–20]. In some countries, weekly reports on Influenzanet results were also published within the official national surveillance bulletins [22, 23].
For sensitivity analysis, we also performed the same analyses on the two following influenza seasons, 2012/2013 and 2013/2014.
Privacy and ethical approval
This study was conducted in agreement with country-specific regulations on privacy and data collection and treatment. Informed consent was obtained from all participants enabling the collection, storage, and treatment of data, and their publication in anonymized, processed, and aggregated forms for scientific purposes. In addition, approvals by Ethical Review Boards or Committees were obtained, where needed according to country-specific regulations. In The United Kingdom, the Flusurvey study was approved by the London School of Hygiene and Tropical Medicine Ethics Committee (Application number 5530). In Sweden, the Influensakoll study was approved by the Stockholm Regional Ethical Review Board (Dnr. 2011/387-31/4). In France, the Grippenet.fr study was approved by the Comité consultatif sur le traitement de l’information en matière de recherche (CCTIRS, Advisory committee on information processing for research, authorization 11.565) and by the Commission Nationale de l’Informatique et des Libertés (CNIL, French Data Protection Authority, authorization DR-2012-024). In Portugal, the Gripenet project was approved by the National Data Protection Committee and also by the Ethics Committee of the Instituto Gulbenkian de Ciência.
To join the network, users registered on their national platform. Upon registration, the user was asked to complete an intake survey, covering demographic factors (age, gender), geographic factors (location of home and work/school expressed at the municipality or zipcode level), socio-economic factors (household size and composition, occupation, educational level, daily transportation means), and health-related factors (including vaccination status against influenza in the 2011/2012 and previous season, diet, pregnancy status, smoking habits, and medical conditions associated with higher risk of influenza complications). The intake survey was standardized and translated whilst preserving the same type and content of questions and possible answers, as well as the same order of questions within the survey, and accounting for the differences related to specific national standards (e.g. schooling structure and associated age/degrees). A few additional questions were added by some platforms due to differences in national public health regulations or to gather additional profiling information. The survey is available in English in the Additional file 2.
A multi-user account was also available to allow the registration of multiple individuals through a single account. The aim was to facilitate group participation (e.g. family members) and also to access groups who otherwise would be unlikely to participate (e.g. children or elderly not familiar with the Internet).
All users were asked to fill in the intake survey at least once, prior to participating to the surveillance. The intake survey could be updated throughout the season (e.g. because of change of residence, vaccination or pregnancy status). When multiple intake surveys were available for a user, in the present study we used the most recently completed one. In the sensitivity analysis, we quantified the type of changes made in the updated surveys and tested the effect of discarding the updates.
All intake questionnaires filled in between the start date and the closure date of the data collection campaign for the 2011/2012 season were considered in the analysis. Following previous work [13, 15, 21, 24], we included in our sample only active participants (defined as those who completed an intake survey and at least three weekly symptoms surveys, to avoid results being skewed by sporadic participation). We will refer to these as Influenzanet active participants or Influenzanet participants. We tested different inclusion criteria and performed a sensitivity analysis with the stricter inclusion criterion that each participant filled in at least one weekly symptoms survey per calendar month.
Users who did not specify age/gender details were additionally removed from the sample, as demographic biases could not be assessed nor accounted for in a sample weighting procedure.
Census and health data sources
We collected national data from a number of socio-demographic datasets and health datasets for all participating countries. In absence of data for the years 2011 or 2012, we relied on the most recent available sources.
Demographic and geographic data were taken from the European Commission portal for European Statistics  and from national institutes of statistics. Georeferenced census data were obtained from the Nomenclature of Territorial Units for Statistics (NUTS), a standard geocode for referencing the subdivisions of countries for statistical purposes, developed by the European Union . We considered the NUTS2 level, corresponding to basic regions for the application of regional policies.
All other socio-economic data were taken from European Statistics and national sources: household size and composition [27, 28]; education data [29–31]; employment data ; transport habits ; vaccination coverage data [34–41]; diabetes prevalence data [42–48]; asthma prevalence data [44, 49–53]; smoking prevalence data ; body mass index (BMI) data for France .
Commuting data was collected for all countries from national institutes of statistics or departments of transportation . Namely, we used data on the number of daily commuters from location of origin to location of destination.
The representativeness of the Influenzanet population was assessed through the comparison of its characteristics with those of the general population for each country. We used χ2-test for non-continuous sociodemographic variables, and Student’s t-test for mean comparisons. All comparisons used 2-tailed tests and a 5% cutoff point. To assess whether differences in participation rates between countries were associated with differences in Internet coverage (access and usage ), a test for association between paired samples was considered, using Pearson’s product moment correlation coefficient, Kendall’s τ or Spearman’s ρ. Statistical analyses were performed using the R software version 2.13.2 (R Development Core Team, R Foundation for Statistical Computing, Vienna, Austria, http://www.r-project.org).
Age data were analyzed in 10-years age categories up to an aggregated 70+ class. For France we had a category of 18-19 years old individuals, because of the absence of younger participants during the data collection campaign here analyzed. We additionally split the 60-69 class into two categories, 60-64 and 65-69 years of age, to account for the age definition (65+) of individuals at risk for developing flu-related complications.
Georeferenced data from Influenzanet were mapped from zip codes or municipality resolution to NUTS2 level for comparison with national data. Apart from the geographic and demographic characteristics, all other variables were adjusted by age (10-years categories) and gender.
The household composition question offered a list of age groups to be ticked, next to open fields where to indicate the number of individuals in the household for each selected age group (Intake Q6 in Additional file 2). When no number was indicated, we assumed that one individual belonged to the selected age group.
Commuting data, extracted from countries’ census and from Influenzanet population, were mapped to NUTS2 level. Data were analyzed in terms of networks of nodes and links [58, 59], with nodes representing the NUTS2 regions and directed links the commuting movement between regions. A weight w OD was also assigned to each link from origin O to destination D to indicate the number of commuters on that connection. Adjusted analyses by geographic distribution of the population were performed (Additional file 1). We assessed whether the Influenzanet links reproduce the backbone of the census commuting network defined by extracting for each country a portion of census network of the same size of the Influenzanet commuting network containing the highest traffic links. An alternative definition of backbone was tested for sensitivity analysis using the disparity filter algorithm  (Additional file 1). We quantified the overlap between the Influenzanet commuting network and the census one through the Jaccard index, measuring the ratio between the number of common links in the two networks and the total number of links. The index is defined in the range [0,1] where 0 indicates that no common link is observed and 1 indicates that the two sets are identical. We calculated the probability of occurrence of the directed links in the Influenzanet commuting (P OD ), given the probability of commuting from O to D computed from national census data and the sample of the Influenzanet participants in region O. Details on the computation are reported in the Additional file 1.
Participation to Influenzanet in the 2011/2012 season
No. registered individuals
No. active** participants
% active in sample
No. active in country (per 100,000)
Participation rates to Influenzanet per country compared to 2011 Internet access and usage statistics
No. Influenzanet participants per 100,000 (rank)
% individuals using the internet at least once a week (rank)
% internet access in households (rank)
% broadband internet connections in households (rank)
% individuals who never used the Internet (rank)
Among the sample of active participants, 83% had a single membership account (variation from 69% for Italy to 89% for Belgium), 9% belonged to a multiple account with 2 active participants (from 7% for Belgium to 12% for the UK), and 8% belonged to an account with 3 or more participants.
Overall, 89.1% of participants never updated their intake survey (variation from 78.7% for Italy to 93.5% for Sweden), 8.8% updated it twice, and 2.1% updated it at least three times.
Geographic and demographic characteristics
Average age of Influenzanet participants and comparison with the national statistics (all p < 10 –5 , except * p = 0.5, †0.01 < p < 0.03, ††0.001 < p < 0.006)
Years (95% CI)
52.8 (52.3 – 53.3)
52.0 (51.5 – 52.5)
45.9 (45.0 – 46.9)
51.6 (51.3 – 51.9)
44.9 (44.0 – 45.9)
43.7 (42.8 – 44.5)
47.0 (46.2 – 47.8)
49.0 (48.3 – 49.7)
50.8 (50.2 – 51.4)
43.7 (42.3 - 45.2)
49.7 (49.3 – 50.0)
42.4 (41.0 – 43.8)
44.3 (43.3 – 45.3)
45.5 (44.6 – 46.5)
56.0 (55.4 – 56.7)
54.3 (53.4 – 55.2)
47.5 (46.3 – 48.8)
54.3 (53.9 – 54.8)
47.5 (46.1 – 48.9)
42.5 (40.8 – 44.2)
49.4 (48.0 – 50.7)
The class of young adults, from 30 to 39 years of age, showed different results depending on gender (Figure 3, when all countries are considered) and on the country (Figure 4 and Additional file 1: Figure S4).
Gender-specific differences in the representativeness of Influenzanet participants are also found in the older classes. Each country reported an underrepresentation of the 70+ class when all participants are considered, with the male class being however overrepresented in the majority of countries (Belgium, France, The Netherlands and UK, Additional file 1: Figure S4). This gender disproportion is also confirmed if we consider all Influenzanet countries aggregated (Figure 3b).
Average ratio between the number of individuals commuting outside and within their region of residence (all p > 0.1) and comparison with national statistics
Ratio between across-regions and within-regions commuters (95% CI)
Ratio between across-regions and within-regions commuters (95% CI)
0.429 (0.021 - 1.000)
0.053 (0.000 - 0.213)
0.025 (0.000 - 0.135)
0.189 (0.102 - 0.343)
0.164 (0.000 - 0.806)
0.028 (0.0 - 0.102)
0.478 (0.0 - 2.746)
Census backbones and Influenzanet networks showed an overlap ranging from 0.18 (Sweden) to 0.85 (Belgium) (Additional file 1: Table S4). The adoption of an alternative definition of network backbone displays a lower similarity between the two networks (Figure A5).
Average household size of Influenzanet participants and comparison with national statistics (all p < 10 –3 )
Household size (95% CI)
3.4 (3.3 – 3.5)
2.9 (2.8 – 3.0)
2.8 (2.7 – 3.0)
3.2 (3.2 – 3.3)
4.0 (3.2 – 4.8)
3.9 (3.3 – 4.5)
4.0 (3.1 – 5.0)
Employment rate in the [15-64]y class of age and comparison with national statistics (all p < 10 –3 , except * p = 0.09, †0.01 < p < 0.05
% (95% CI)
68.6 (66.7 – 70.4)
70.9 (68.8 – 73.0)
66.2 (61.2 – 71.0)
72.6 (71.7 – 73.6)
68.2 (64.6 – 71.5)
71.3 (68.4 – 74.0)
68.1 (65.4 – 70.7)
Education level of Influenzanet participants and comparison with national statistics (all p < 10 –6 )
Influenzanet country: indicator
% of individuals
% of individuals
FR: individuals with at least high-school level
No qualification ([15-64]y; 65+) 65+)
GCSE ([15-64]y; 65+)
A-level ([15-64]y; 65+)
Higher ([15-64]y; 65+)
SE: individuals in [20-64] age class
No qualification (female; male)
GCSE (female; male)
A-level (female; male)
Bachelor (female; male)
Higher (female; male)
Health factors and vaccination
The percentage of Influenzanet participants reporting asthma is significantly lower than in the general population of Portugal (3.04% vs. 6.80%,p < 10–5), Italy (4.2% vs. 6.1%, p < 10–2), and the UK (9.2% vs. 11%, p = 0.02). The opposite trend is obtained for The Netherlands (8.4% vs. 3.2%, p < 10–6) and Belgium (3.95% vs. 2.8%, p < 10–4). No significant difference was found in France (5.9% vs. 6%, p = 0.8). Results are reported in Figure 9.
Influenzanet diabetes prevalence was in agreement with national data for three countries (The Netherlands, Belgium, and France, with p = 0.8, p = 0.2, and p = 0.7, respectively), and lower for the others (p = 0.02 for Sweden, p = 0.01 for Portugal, p ≤ 10–3 for Italy and UK, Figure 9).
Vaccination coverage against influenza in the 65+ age class during the 2011/2012 season was larger in the Influenzanet participants of France, Portugal and Sweden, whereas it was statistically representative in Italy (57.2% vs. 62.7%, p = 0.8) and UK (74.21% vs. 74%, p = 0.98) (Figure 9). In the 2010/2011 season, vaccination coverage was higher among Influenzanet participants in all countries (p < 10–4), except in Italy where vaccinated 65+ individuals were strongly underrepresented (35% vs. 62%, p < 10–4), and in UK where vaccination coverage was in agreement with national data (75% vs. 73%, p =0.55, see Additional file 1: Table S6). Dutch data were not available for comparison for the 2011/2012 season and Belgian data were not available for both seasons.
Repeating the analysis with a stricter inclusion criterion produced no qualitative differences in the results presented.
The updates of the intake survey for 10.9% of the total number of participants most frequently concerned the participant’s job (place of work; occupation; main activity), her weight (in the French survey only, where a question on weight and height was added to evaluate the participant’s BMI), her mean of transport (main mean of transport; time spent daily on public transportation), and her place of living. These changes do not affect the results obtained for the representativeness in terms of age, gender, household, and health indicators. The changes in the geographic and job indicators produced no qualitative differences in the results presented.
The representativeness analysis on the following two seasons (2012/2013 and 2013/2014) showed that the obtained results are robust in time (Additional file 1). No qualitative difference was observed, except for the Influenzanet vaccination coverage in France that was found to be representative of the corresponding value in the general population, differently from the 2011/2012 season. Differences in the participation of specific age groups were observed in some countries (e.g. in the UK where a higher participation of school-aged children was reported in the 2013/2014 season thanks to school-specific activities and communication campaigns), without altering the overall picture of lack of representativeness in terms of age observed for all participating countries.
31,674 residents in 7 European countries joined the online surveillance study in the first season (2011/2012) where a standardized and uniform data collection approach was adopted by the Influenzanet Consortium. Active participation was observed for 80% of the participants and covered all NUTS2 regions included in the project. Participation varied widely across countries, geographic regions, gender groups, and age classes. This is most likely related to different factors, namely: the reachability of a given portion of the population obtained through communication campaigns; the availability, usage of and familiarity with the Internet (which is used in this study as the mean to collect data); and the self-selection of participants, or ‘volunteer effect’, and the underlying interest towards the object of the study .
Results seem to indicate that coverage biases due to the Internet may partly explain the observed variability in participation per country, however all tests were non-significant likely due to the small number of data points. Belgium and Portugal showed a better ranking in participation rates with respect to the various Internet indicators, pointing to a larger participation than expected based on country ranking for Internet usage only, which is likely due to the longer history of the national platforms (Belgium from 2003 together with The Netherlands, and Portugal from 2005). The Netherlands, France, and Italy ranked in participation approximately as expected by Internet access and usage statistics. Conversely, Sweden and the United Kingdom were ranked lower in participation rates (3rd and 6th, respectively) than according to Internet statistics. It is important to note that for France and Sweden it was their first season in the project.
Geographic distribution within each country was not representative, and a larger participation was generally observed in those regions hosting the laboratory/Institution conducting the study, likely reflecting a more powerful effect of communication campaigns at the local level. Other initiatives, geographically limited, appear to be responsible for large participation rates in the population. This is for example the case of the Corsica region, with a participation rate of 3.5 per 100,000 vs. 2.4 per 100,000 observed in the region of the Ile de France (hosting the Supporting Institution), following the diffusion of localized communication campaigns and Influenzanet activities at schools in the region supported by a regional project .
An unbalance in the participation by gender was observed, except in the case of Portugal. Two opposed aspects may be at play in the gender imbalance. On one hand, previous studies suggest that women are on average more interested in health-related topics and also exhibit a more active information-seeking behavior [63, 64]. Such gender-specific behavior may thus lead to a more likely voluntary female participation in a health-related project like Influenzanet, as observed in The Netherlands, UK, Sweden, and France. Results showing a higher tendency of participation of larger households in the study may further support this hypothesis, as possibly driven by women’s interest in family and children care .
On the contrary, Belgium and Italy showed a larger fraction of male participants with respect to the national partition by gender. This might be explained by another gender-specific aspect, regarding the usage of and familiarity with technology in general. Internet usage differs by gender across all countries, with a larger fraction of men accessing the Internet at least once per week compared to women . Interestingly, the countries with the largest relative difference in the gender-specific access to the Internet (Italy, Portugal and Belgium, with a relative difference of 18%, 11%, and 7%, respectively) were also the countries with a larger prevalence of male Influenzanet participants (Belgium and Italy) or displaying a representative population by gender (Portugal). A larger disproportion in men’s vs. women’s Internet access appears therefore to balance out the female volunteering effect due to health-interest, family care, and information-seeking behavior.
The Influenzanet population was not representative in terms of age, with an overrepresentation of the [40-69]y class (for each gender), an underrepresentation of the younger age classes, [0-29] (for each gender), and of the elderly (age ≥ 70y, for all countries when both genders are considered together). Internet usage statistics report a decreasing dependence on age , with larger (e.g. Italy and Portugal) to smaller (e.g. The Netherlands and Sweden) variations by age classes. This decreasing rate by age may explain the low rates of participation observed in the 70+. To achieve a better representativeness of individuals in this class, the surveillance system will need to design targeted communication campaigns for this group and, most importantly, facilitate the accessibility to the project. It is interesting to note that individuals in the [60-69] age class are largely overrepresented. We tested whether this may be induced by a specific interest and concern of individuals of 65+ years of age for whom influenza vaccination is recommended in Europe, but found no major difference in the representativeness of [60-64] vs. [65-69] class to support this hypothesis.
Underrepresentation in the [0-9] and [10-19] classes of age may be due to the impossibility to access the Internet in an unsupervised way for the youngest children, and to the unlikelihood of being exposed to the project for the older ones. The system already incorporates the possibility of adding multiple users to an account managed by a single participant who is supposed to facilitate the input of data for individuals who cannot or are not familiar with Internet tools. The results of this study for the 2011/2012 season indicate, however, that more specific efforts in reaching out to younger age classes are needed, for instance through projects and communication/entertainment actions at schools. Such actions may be for example responsible for the increase in participation rates in school-aged children observed in the UK in the 2013/2014 season.
A lack of interest in influenza or health-related topics may be at the basis of the underrepresentation of the [20-29] age class, since this is the group having the most largely diffused usage of new technologies, with an at least weekly Internet access reported for more than 88% of individuals between 16 and 34 years old for all countries studied, with the exception of Italy (81% for [16-24]y and 70% for [25-34]y) and Portugal (89% and 77%, respectively). The class of [30-39] years old instead showed a different participation behavior depending on the gender (overrepresentation of females and underrepresentation of males) and on the country (underrepresentation in Belgium, France and The Netherlands, when both genders are considered, opposite trend elsewhere). This age class may represent the transition between young-specific lack of interest for the project and the raise of family-specific interest for health-related information. The average age at first childbirth is indeed found between 28 years (Belgium) and 30 years (Italy) in 2010 . Other possible mechanisms may clearly come into play, such as e.g. a more general increased responsibility towards society and public good.
Once the non-representative nature of the Influenzanet population in terms of age and gender was adjusted for, commuting patterns registered by Influenzanet reproduced well the ratio between the within-region and the across-region number of commuters, recovering a feature that is relevant for the spatial spread of influenza. The proportion of census links represented in the Influenzanet network was larger for the countries with higher number of active participants, showing that a better representativeness of the topology of the network can be reached with higher levels of participation. When only a small fraction of links was represented, those were in general the ones with higher census traffic, i.e. the network backbone.
The analysis of transportation modes showed that the Influenzanet sample, despite being non representative for 6 countries out of 7, reproduced some of the aspects of the general population transport behavior, like the top three transportation modes, that were the same in the Influenzanet and in the general population for all countries except Sweden.
Participants in general had higher education levels compared to the general population, which is in agreement with previous studies employing web-based surveys [67, 68], and is likely induced by the non-representative nature of Internet users (Internet usage dramatically increases with education level ) and of the sample of individuals highly engaged in the survey’s topic.
Our interpretation of partially incomplete data for the household composition (see Methods) offers a lower boundary for the household size, therefore it does not qualitatively alter our findings on larger household sizes found for Influenzanet participants. Other assumptions adopted for the study were tested for sensitivity (i.e. stricter inclusion criteria and consideration of the first intake only neglecting following updates) and no qualitative differences were observed in the results.
The Influenzanet sample contained fewer smokers than expected from national statistics, with few exceptions (representativeness for France for both genders and for Portugal and Italy for the female sample only). International comparability on such statistics is however limited due to the lack of standardization in the measurements of smoking habits in health interview surveys across EU member states (see e.g. differences found across different sources, Refs.  and ). For example, there are variations in the wording of questions and in the response categories used in surveys for smoking behaviors (e.g. smoking daily vs. regularly). Our results consider the Influenzanet responses for daily smoking habits (i.e. less than 10, 10 or more cigarettes per day, excluding occasional smokers) compared to the national statistics defined in terms of ‘daily smoking’ .
Vaccination coverage against influenza in the 65+ age class was statistically representative of national coverage in Italy and the UK, and it was higher in the samples of the other countries. Vaccination coverage reported for Italy for the previous season (participants were asked about their vaccination status in the previous season too) was much smaller than what has been declared for the season under study (the latter being also statistically representative of national data). No clear explanation is available, given that the sample of individuals declaring the vaccination status is the same. It may be due either to memory biases in the reporting of previous season vaccination behavior, or to change of vaccination behavior from one season to another. The 2010/2011 season was indeed the first influenza season following the 2009 H1N1 influenza pandemic, and the reported coverage may be the result of the negative impact of the controversies related to the pandemic vaccination campaign of 2009/2010 on subsequent seasonal influenza vaccination coverage. While this hypothesis was explicitly tested in some countries where no association was found , we are not aware of similar studies being conducted in Italy, and we argue that the large variability observed in the attitudes towards vaccination uptake during the H1N1 pandemic  may possibly lead to different results that are country-specific.
Overall, health-related results further indicate a tendency of Influenzanet participants towards better health and towards health care, with few exceptions. Furthermore, an analysis on the Body Mass Index of French participants have shown that they were less frequently found to be overweight and obese than the French population , further supporting such tendency.
For sensitivity analyses we also tested the robustness of our findings for the two influenza seasons following the standardization of the Influenzanet platform. The only qualitative difference was found in the vaccination coverage of the 65+ Influenzanet population in France that was representative of the corresponding national value for both 2012/2013 and 2013/2014 seasons, whereas in the 2011/2012 season a marginal overrepresentation was observed. It is to be noted however that some statistics for the general population for some of the indicators considered here were not available for all countries at the time of the study (e.g. vaccination coverage for the last influenza season, or asthma and diabetes incidences). Other differences, however not altering the findings reached for the 2011/2012 season, included an increase in the participation rate of school-aged children in the UK, following targeted communications in the country. Larger quantitative differences that may alter the conclusions of this study may be found on longer timeframes of data collection, induced by population changes in some of the indicators that may drive the participation to the surveillance scheme. For example, in the longitudinal study of eight seasons of the Belgian platform, Vandendijck and collaborators found a marked increase in participation in the [60-69] age class, likely attributable to the growing internet usage in this age group during that timeframe .
In addition to the self-selection bias, another potential limitation of the study is induced by the employed data collection methodology that may have an effect on data reliability when participants self-report inaccurate information. This may happen unconsciously, e.g. due to the fact that participants mistakenly introduce wrong data or might forget to report an information, or as the result of a deliberate action. In the first case, while simple mistakes in completing the surveys may be automatically checked by the system (as e.g. a date of birth in the future) or avoided with design improvements, all errors related to misunderstandings, subjective interpretation or memory effects in the reporting would go undetected. Studies have found that Web participants’ responses contained less random and systematic error than their telephone counterparts . This was explained as an effect of the lack of social compliance towards the interviewer and the availability of a longer time to process the information at the individual’s own pace . Moreover, memory effects leading to a systematic error known as recall bias are expected to occur when surveying participants’ behavior on a large set of indicators regarding events or experiences from the past. We evaluate that such bias is unlikely to occur in the intake survey of Influenzanet, as the questions asked refer to standard demographic information and everyday habits or conditions (e.g. smoking behavior, main mean of transportation, presence of allergies, etc.). For the same reason, also misunderstandings and wrong interpretations of the questions are unlikely to occur.
The only question referring to a particular event in time contained in the Influenzanet intake survey is the one on the vaccination status. If the vaccination occurs after the completion of the intake survey, the participant may forget to update the information on her personal space, thus inducing a bias in our results. We evaluate that such cases, if present, would represent a small fraction of the total, as the Influenzanet surveillance campaign typically starts after the vaccination campaign in each of the countries. Nonetheless, a simple reminder concerning the update of the vaccination status can be easily implemented to overcome this issue.
Concerning deliberate actions of providing inaccurate data on online surveys, the probability of filling in fraudulent data in a web-based survey, though possible, is expected to be very limited due to the absence of specific incentives, and the time resources needed to perform the fraudulent action.
The analysis of the characteristics of approximately 25,000 participants in the Influenzanet network of online platforms for influenza-like-illness surveillance showed a large variability across countries in terms of representativeness. The youngest and oldest age classes were all underrepresented, and gender representativeness was reached only for one country out of seven. Participants’ households were found to be larger than those of the general population, and participants’ health indicators overall indicated a higher concern for health-related issues.
The advantage of the system is to allow the evaluation of representativeness along a large set of population aspects. The study indicated areas in which specific strategies and updates in future surveillance may be envisioned for the recruitment of undersampled groups of the general population. The evaluation findings will be used to correctly interpret epidemiological data and assess risk factors to inform public health policy.
We particularly thank all European participants who took part in the Influenzanet project in the 2011/2012 season. We also thank Niel Hens for useful discussions and for sharing Flemish general population health data, and Patricia Soares for help extracting the Portuguese general population data. This work is partly supported by the U707/InVS partnership contract n°12-N-MIP20-04, and the ANR contract no. ANR-12-MONU-0018 (HARMSFLU).
- World Health Organization: Global Influenza Surveillance and Response System (GISRS). http://www.who.int/influenza/gisrs_laboratory/en/,
- OECD: Participative Web and User-Created Content. Web 2.0, Wikis and Social Networking. 2007, OECD PublishingGoogle Scholar
- Marquet RL, Bartelds AIM, van Noort SP, Koppeschaar CE, Paget J, Shellevis SG, van der Zee J: Internet-based monitoring of influenza-like illness (ILI) in the general population of the Netherlands during the 2003-2004 influenza season. BMC Public Health. 2006, 6: e242-10.1186/1471-2458-6-242.View ArticleGoogle Scholar
- Influenzanet. http://www.influenzanet.eu/,
- Paolotti D, Carnahan A, Colizza V, Eames K, Edmunds J, Gomes G, Koppeschaar C, Rehn M, Smallenburg R, Turbelin C, Van Noort S, Vespignani A: Web-based participatory surveillance of infectious diseases: the Influenzanet participatory surveillance experience. Clin Microbiol Infect. 2014, 20: 17-21. 10.1111/1469-0691.12477.View ArticlePubMedGoogle Scholar
- Centers for Disease Control and Prevention: Updated guidelines for evaluating public health surveillance systems. Recommendations from the Guidelines Working Group. MMWR. 2001, 50 (RR–13): 1-35.Google Scholar
- Ginsberg J, Mohebbi MH, Patel RS, Brammer L, Smolinski MS, Brilliant L: Detecting influenza epidemics using search engine query data. Nature. 2009, 457: 1012-1014. 10.1038/nature07634.View ArticlePubMedGoogle Scholar
- Pelat C, Turbelin C, Bar-Hen A, Flahault A, Valleron AJ: More diseases tracked by using Google Trends. Emerg Infect Dis. 2009, 15: 1327-1328. 10.3201/eid1508.090299.View ArticlePubMedPubMed CentralGoogle Scholar
- Brownstein JS, Freifeld CC, Reis BY, Mandl KD: Surveillance Sans Frontieres: internet-based emerging infectious disease intelligence and the HealthMap project. PLoS Med. 2008, 5: e151-10.1371/journal.pmed.0050151.View ArticlePubMedPubMed CentralGoogle Scholar
- Brownstein JS, Freifeld CC, Madoff LC: Digital disease detection–harnessing the Web for public health surveillance. N Engl J Med. 2009, 360: 2153-2157. 10.1056/NEJMp0900702.View ArticlePubMedPubMed CentralGoogle Scholar
- Signorini A, Segre AM, Polgreen PM: The use of Twitter to track levels of disease activity and public concern in the U.S. during the influenza A H1N1 pandemic. PLoS One. 2011, 6: e19467-10.1371/journal.pone.0019467.View ArticlePubMedPubMed CentralGoogle Scholar
- deGroteGriepMeting. https://www.degrotegriepmeting.nl/,
- Vandendijck Y, Faes C, Hens N: Eight years of the great influenza survey to monitor influenza-like illness in Flanders. PLoS One. 2013, 8 (5): e64156-10.1371/journal.pone.0064156.View ArticlePubMedPubMed CentralGoogle Scholar
- gripenet. http://www.gripenet.pt/,
- van Noort SP, Muehlen M, Rebelo de Andrade H, Koppeschaar C, Lima Lourenco JM, Gomes MG: Gripenet: an internet-based system to monitor influenza-like illness uniformly across Europe. Euro Surveill. 2007, 12: 229-233.Google Scholar
- Influweb. https://www.influweb.it/,
- Tilston NL, Eames KTD, Paolotti D, Ealden T, Edmunds WJ: Internet-based surveillance of Influenza-like-illness in the UK during the 2009 H1N1 influenza pandemic. BMC Public Health. 2010, 10: e650-10.1186/1471-2458-10-650.View ArticleGoogle Scholar
- Flusurvey. http://flusurvey.org.uk/,
- influensakoll. https://www.influensakoll.se/,
- GrippeNet.fr. http://www.grippenet.fr/,
- Debin M, Turbelin C, Blanchon T, Bonmarin I, Falchi A, Hanslik T, Levy-Bruhl D, Poletto C, Colizza V: Evaluating the representativeness of an online nationwide surveillance system for influenza in France. PLoS One. 2013, 8 (9): e73675-10.1371/journal.pone.0073675.View ArticlePubMedPubMed CentralGoogle Scholar
- InVS, Institut de veille sanitaire. http://www.invs.sante.fr/,
- Réseau Sentinelles Bulletin hebdomadaire. http://websenti.u707.jussieu.fr/sentiweb/,
- Friesema IHM, Koppeschaar CE, Donker GA, Dijkstra F, van Noort SP, Smallenburg R, van der Hoek W, van der Sande MA: Internet-based monitoring of influenza-like illness in the general population: experience of five influenza seasons in the Netherlands. Vaccine. 2009, 27: 6353-6357. 10.1016/j.vaccine.2009.05.042.View ArticlePubMedGoogle Scholar
- Eurostat, European Commission. http://epp.eurostat.ec.europa.eu/portal/page/portal/eurostat/home,
- NUTS, Nomenclature of Territorial Units for Statistics. http://epp.eurostat.ec.europa.eu/portal/page/portal/nuts_nomenclature/introduction,
- Eurostat: Distribution of households by household size. http://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=ilc_lvph03&lang=en,
- Eurostat: Average household size. http://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=ilc_lvph01&lang=en,
- Institut national de la statistique et des études économiques: Les estimations de population. http://www.insee.fr/fr/publications-et-services/default.asp?page=dossiers_web/population/estimations_pop.htm#p2,
- Pordata, Base de dados Portugal contemporaneo: Resident population aged 15 to 64 and 65 and over: by highest level of educational qualifications obtained. http://www.pordata.pt/en/Portugal/Resident+population+aged+15+to+64+and+65+and+over+by+highest+level+of+educational+qualifications+obtained+%28percentage%29-2266,
- Statistics Sweden: Educational attainment and employment of the population 2010. http://www.scb.se/Pages/PublishingCalendarViewInfo____259924.aspx?PublObjId=17126,
- OECD: “Employment rate”, Employment and Labour Markets: Key Tables from OECD, No. 4. doi:10.1787/emp-table 2012-1-enGoogle Scholar
- European Commission: Future of transport. analytical report. 2011, http://ec.europa.eu/public_opinion/flash/fl_312_en.pdf,Google Scholar
- Caisse Nationale d’Assurance Maladie, Ministère des affaires sociales et de la santé, Agence nationale de sécurité du médicament et des produits de santé, Institut de veille sanitaire: Campagne de vaccination contre la grippe saisonnière 2012-2013. http://www.sante.gouv.fr/IMG/pdf/dossier_presse_grippe_2012_28092012.pdf,
- Institut national de la statistique et des études économiques: Les variables issues de la Caisse Nationale d’Assurance Maladie des Travailleurs Salariés (CNAMTS). http://www.insee.fr/fr/ppp/bases-de-donnees/donnees-detaillees/duicq/accueil.asp?page=doc/cnam.htm,
- Tuppin P, Choukroun S, Samson S, Weill A, Ricordeau P, Allemand H: Vaccination against seasonal influenza in France in 2010 and 2011: decrease of coverage rates and associated factors. Presse Med. 2012, 41: 568-576. 10.1016/j.lpm.2012.05.017.View ArticleGoogle Scholar
- Dutch National Public Health Compass: Influenza. http://www.nationaalkompas.nl/preventie/van-ziekten-en-aandoeningen/influenza/bereik/,
- Nunes B, Branco MJ: Vacinação antigripal da população portuguesa, em. 2011, Lisboa: Instituto Nacional de Saúde Dr. Ricardo Jorge. Departamento de Epidemiologia, Documento interno, http://repositorio.insa.pt/handle/10400.18/1010, -2012: cobertura e algumas características do acto vacinal,
- Italian Ministry of Health: Coperture vaccinali medie per regione. http://www.salute.gov.it/influenza/paginaInternaMenuInfluenza.jsp?id=679&menu=strumentieservizi,
- UK Health Protection Agency: Weekly Report 31st March 2011. http://www.hpa.org.uk/webc/HPAwebFile/HPAweb_C/1296684494218,
- UK Health Protection Agency: Weekly Report 22nd March 2012. http://www.hpa.org.uk/webc/HPAwebFile/HPAweb_C/1317133280485,
- Ecosante. http://www.ecosante.fr,
- Dutch National Public Health Compass: Diabetes. http://www.nationaalkompas.nl/gezondheid-en-ziekte/ziekten-en-aandoeningen/endocriene-voedings-en-stofwisselingsziekten-en-immuniteitsstoornissen/diabetes-mellitus/omvang/,
- Belgian health interview survey 2008. https://his.wiv-isp.be/SitePages/Home.aspx,
- Portuguese National Institute of Health: Relatório Anual da Diabetes. http://www.insa.pt/sites/INSA/Portugues/ComInf/Noticias/Paginas/RelatorioAnualDiabetes.aspx,
- Italian National Institute of Health: La patologia diabetica: dati di prevalenza a livello nazionale. http://www.epicentro.iss.it/igea/PrimoPiano/prevalenza.asp,
- Diabetes UK: Reports and statistics. diabetes prevalence 2011. http://www.diabetes.org.uk/Professionals-old/Publications-reports-and-resources/Reports-statistics-and-case-studies/Reports/Diabetes-prevalence-2011-Oct-20111/,
- Diabetes Sweden: Reports and statistics. http://www.diabetes.se,
- Dutch National Public Health Compass: Asthma. http://www.nationaalkompas.nl/gezondheid-en-ziekte/ziekten-en-aandoeningen/ademhalingswegen/astma/omvang/,
- Sa-Sousa A, Morais-Almeida M, Azevedo LF, Carvalho R, Jacinto T, Todo-Bom A, Loureiro C, Bugalho-Almeida A, Bousquet J, Fonseca JA: Prevalence of asthma in Portugal - the Portuguese National Asthma Survey. Clin Transl Allergy. 2012, 2 (1): 15-10.1186/2045-7022-2-15.View ArticlePubMedPubMed CentralGoogle Scholar
- Italian National Institute of Health: Persone con presenza di alcune malattie croniche. http://dati.istat.it/Index.aspx?DataSetCode=DCCV_MALCRON,
- Simpson CR, Sheikh A: Trends in the epidemiology of asthma in England: a national study of 333,294 patients. J R Soc Med. 2010, 103 (3): 98-106. 10.1258/jrsm.2009.090348.View ArticlePubMedPubMed CentralGoogle Scholar
- Swedish Heart-Lung Foundation: Research on Asthma. http://www.hjart-lungfonden.se/Forskning/Aktuell-forskning/Lungforskning/Forskning-om-astma/,
- OECD, health at a glance: Europe 2012. determinants of health. smoking among adults. http://www.oecd-ilibrary.org/sites/9789264183896-en/02/05/index.html?contentType=&itemId=/content/chapter/9789264183896-24-en&containerItemId=/content/serial/23056088&accessItemIds=/content/book/9789264183,
- Eschwege E, Charles M-A, Basdevant A, Moisan C, Bonnélye G, Touboule C, Goulhamousen N, Crine A: ObEpi Roche 2012 - Enquête épidémiologique nationale sur le surpoids et l’obésité. http://www.roche.fr/content/dam/corporate/roche_fr/doc/obepi_2012.pdf,
- Balcan D, Colizza V, Goncalves B, Hu H, Ramasco JJ, Vespignani A: Multiscale mobility networks and the spatial spreading of infectious diseases. Proc Natl Acad Sci U S A. 2009, 106: 21484-21489. 10.1073/pnas.0906910106.View ArticlePubMedPubMed CentralGoogle Scholar
- Eurostat: Information society statistics. http://epp.eurostat.ec.europa.eu/statistics_explained/index.php/Information_society_statistics,
- Barabasi AL, Albert R: Emergence of scaling in random networks. Science. 1999, 286: 509-512. 10.1126/science.286.5439.509.View ArticlePubMedGoogle Scholar
- Barrat A, Barthelemy M, Vespignani A: Dynamical Processes on Complex Networks. 2008, Cambridge: University PressView ArticleGoogle Scholar
- Serrano MA, Boguna M, Vespignani A: Extracting the multiscale backbone of complex weighted networks. Proc Natl Acad Sci U S A. 2009, 106: 6483-6488. 10.1073/pnas.0808904106.View ArticlePubMedPubMed CentralGoogle Scholar
- Friedman C, Wyatt J, Smith A, Kaplan B: Evaluation Methods in Medical Informatics. 1997, New York: SpringerView ArticleGoogle Scholar
- GrippeNet.fr in Corsica. https://grippenet.fr/fr/grippenet-en-corse/,
- Centre de Recherche pour l’Etude et l’Observation des Conditions de Vie: Les opinions des femmes et des hommes sont-elles semblables ou différentes ?. Cahier de recherche. 195: http://www.credoc.fr/pdf/Rech/C195.pdf,
- Fox S, Rainie L: The online health care revolution: how the web helps Americans take better care of themselves. http://www.pewinternet.org/~/media/Files/Reports/2000/PIP_Health_Report.pdf.pdf,
- Crespi I, Miller T: Family, Care and Work in Europe: An Issue of Gender?. 2013, Italy: Eum, Edizioni Università di MacerataGoogle Scholar
- UNECE (United Nations Economic Commission for Europe) Statistical Database. http://w3.unece.org/,
- Couper MP: Web surveys: a review of issues and approaches. Publ Opin Q. 2000, 64: 464-494. 10.1086/318641.View ArticleGoogle Scholar
- Gosling SD, Vazire S, Srivastava S, John OP: Should we trust web-based studies? a comparative analysis of six preconceptions about internet questionnaires. Am Psychol. 2004, 59 (2): 93-104.View ArticlePubMedGoogle Scholar
- Guthmann J-P, Fonteneau L, Bonmarin I, Levy-Bruhl D: Influenza vaccination coverage one year after the A(H1N1) influenza pandemic, France, 2010-2011. Vaccine. 2012, 30: 995-997. 10.1016/j.vaccine.2011.12.011.View ArticlePubMedGoogle Scholar
- Tizzoni M, Bajardi P, Poletto C, Ramasco JJ, Balcan D, Goncalves B, Perra N, Colizza V, Vespignani A: Real-time numerical forecast of global epidemic spreading: case study of 2009 A/H1N1 pdm. BMC Med. 2012, 10: 165-10.1186/1741-7015-10-165.View ArticlePubMedPubMed CentralGoogle Scholar
- Krosnick J, Chang L: The American Association for Public Opinion Research (AAPOR) 56th Annual Conference. A Comparison of Random Digit Dialing Telephone Survey Methodology with Internet Survey Methodology as Implemented by Knowledge Networks and Harris Interactive. 2001, Montreal, Quebec, CanadaGoogle Scholar
- Dillman DA, Phelps G, Tortora R, Swift K, Kohrell J, Berck J, Messer BL: Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the Internet. Soc Sci Res. 2009, 38: 1-18. 10.1016/j.ssresearch.2008.03.007.View ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/14/984/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.