Perceived utility and feasibility of pathogen genomics for public health practice: a survey among public health professionals working in the field of infectious diseases, Belgium, 2019
BMC Public Health volume 20, Article number: 1318 (2020)
Pathogen genomics is increasingly being translated from the research setting into the activities of public health professionals operating at different levels. This survey aims to appraise the literacy level and gather the opinions of public health experts and allied professionals working in the field of infectious diseases in Belgium concerning the implementation of next-generation sequencing (NGS) in public health practice.
In May 2019, Belgian public health and healthcare professionals were invited to complete an online survey containing eight main topics including background questions, general attitude towards pathogen genomics for public health practice and main concerns, genomic literacy, current and planned NGS activities, place of NGS in diagnostic microbiology pathways, data sharing obstacles, end-user requirements, and key drivers for the implementation of NGS. Descriptive statistics were used to report on the frequency distribution of multiple choice responses whereas thematic analysis was used to analyze free text responses. A multivariable logistic regression model was constructed to identify important predictors for a positive attitude towards the implementation of pathogen genomics in public health practice.
146 out of the 753 invited public health professionals completed the survey. 63% of respondents indicated that public health agencies should be using genomics to understand and control infectious diseases. Having a high level of expertise in the field of pathogen genomics was the strongest predictor of a positive attitude (OR = 4.04, 95% CI = 1.11 – 17.23). A significantly higher proportion of data providers indicated to have followed training in the field of pathogen genomics compared to data end-users (p < 0.001). Overall, 79% of participants expressed interest in receiving further training. Main concerns were related to the cost of sequencing technologies, data sharing, data integration, interdisciplinary working, and bioinformatics expertise.
Belgian health professionals expressed favorable views about implementation of pathogen genomics in their work activities related to infectious disease surveillance and control. They expressed the need for suitable training initiatives to strengthen their competences in the field. Their perception of the utility and feasibility of pathogen genomics for public health purposes will be a key driver for its further implementation.
Sequence information from viruses, bacteria, and other infectious organisms can be used to identify a pathogen and its specific characteristics, and compare its genetic relatedness to other pathogens . Advances in sequencing technologies, especially the shift to next-generation sequencing (NGS), have made it possible to analyze pathogen genomes in much greater detail. Compared to Sanger sequencing, NGS technologies allow a faster and cheaper way to sequence larger lengths of nucleotides. As such, NGS makes microbial pathogen whole-genome sequencing (WGS) accessible in high throughput within a matter of days . During the last decade, NGS has expanded beyond the research settings and is being rapidly applied into routine practice for public health and food safety [1,2,3,4,5,6].
In public health, integrating pathogen genomics with epidemiology provides many opportunities for improving the population-level risk assessment and management of infectious diseases [1,2,3,4,5,6,7,8]. The main applications of WGS include (1) retrospective (or near real-time) comparisons of pathogens’ relatedness to test epidemiological transmission hypotheses of suspected outbreaks (i.e. outbreak investigations); (2) WGS-based prospective surveillance by monitoring of cases generating alerts when clusters of pathogens with similar genomes are identified in a limited geographical area or time period or when virulent clones emerge (outbreak detection by control-oriented surveillance); and (3) cross-sectional genomic epidemiology surveys to monitor long-term changes in epidemiology over larger geographic and population scales to inform prevention strategies (strategy-oriented surveillance) . The main added value of implementing WGS during surveillance activities or outbreak investigations is inherent in the higher resolution of the WGS output itself, leading to an increased sensitivity and specificity to identify transmission clusters compared to conventional subtyping methods . As such, there are numerous success stories of outbreak investigations applying WGS that were able to identify to the source of infection and implement targeted control measures to stop further spread, saving resources at the health protection and local authority level [10,11,12,13,14,15,16,17]. Other concrete examples of the utility of WGS for national surveillance and local infection control include the guidance of vaccination strategies [18,19,20,21] and antibiotic stewardship [22, 23]. Besides transforming the public health approach to infectious diseases monitoring, analysis of pathogen genomics can advance the accuracy of infection diagnostics and guide the treatment of individual patients [4, 24,25,26,27,28,29]. For several pathogens, NGS is able to replace current time-consuming and/or labor-intensive conventional methods with a single, all-in-one diagnostic test [30,31,32,33].
Public health professionals play a key role in protecting the population against communicable disease threats. This requires them to give effective responses in a limited time frame, supported by adequate information resulting from applying the most appropriate tools adapted to the specific public health threat scenario. Infectious disease surveillance systems build upon the cooperation between: clinicians, who are at the frontline through identification of infected patients; microbiologists, who are involved in testing specimens; molecular biologists, who study organisms at the molecular level; bioinformaticians, who develop computational approaches/algorithms to analyze genomic data; epidemiologists, who use the data to understand patterns in disease occurrence at the population level; infection control practitioners, who are responsible for local prevention and control of infectious diseases in the community; hospital hygienists, who are involved in the prevention and control of healthcare-associated infections; food safety inspectors, who monitor food products; etc. The activities of these public health experts operating at different levels in the information cycle will be impacted by the introduction of pathogen genomics as they are all connected to each other. This ranges from microbiologists adapting their laboratory workflows to epidemiologists rethinking their current data analysis approaches.
Typically, new laboratory technologies are adopted by data providers first, while data end-users might not be familiar enough with the new methods to effectively translate the output data into public health actions. Expertise with pathogen genomics and its applications for public health practice might also differ between those in charge of national surveillance of infectious diseases and those involved in local infection control and patient management, as well as between different fields (i.e. human, animal, food, and the environment) within the One Health spectrum [34, 35].
Differences in perceptions and needs between these different profiles should be taken into account before we can build a strategy that engages all the stakeholders in an effective collaboration. The key to success in translating pathogen genomics into public health practice is to demonstrate an added value by better addressing the needs and expectations of the whole range of public health experts. An effective exchange of expertise across disciplines (e.g., clinicians, microbiologists, epidemiologists, and bioinformaticians) is key for enabling the smooth implementation of NGS into routine public health activities. If such coordination of joint efforts cannot be accomplished, the technology shift, which is currently ongoing, might not realize its full potential [5, 8].
Previous surveys in the field of public health genomics focused on: human genomics [36,37,38]; specific aspects such as proficiency testing , the design of WGS clinical reports  or data sharing ; or specific target groups such as National Microbiology Focal Points  or food safety laboratories . In this study, by organizing an online survey, we aimed to perform a wide landscape analysis of all potentially involved stakeholders in order to appraise the level of genomic literacy and to gather the opinions of public health experts and allied professionals working in the field of infectious diseases in Belgium concerning the implementation of NGS in routine public health activities, in terms of its utility, feasibility, implementation, and translation into actionable results for public health decision making.
An electronic questionnaire survey (see Additional file 1) was developed for this study using LimeSurvey (Version 2.71.1)  for the collection of relevant information from public health professionals working in the field of infectious diseases in Belgium. For the purposes of this study, a ‘public health professional in the field of infectious diseases’ was defined as a person with professional expertise in the field of infectious diseases and who directly or indirectly contributes to the population-level management of infectious diseases. To provide a complete picture of all involved stakeholders, the survey aimed to reach different subgroups based on professional qualification (i.e. microbiologists, molecular biologists, bioinformaticians, epidemiologists, clinicians, clinical biologists, infection control practitioners, and hospital hygienists), employing institution (i.e. governmental, private, hospital, and university), health field (i.e. human, animal, food, and environment), expertise in pathogens (i.e. bacteria, viruses, parasites, fungi, and yeasts), and level of action (i.e. national surveillance and local infection control). To identify all actors in the field of public health activities for infectious diseases, an overview was made of existing surveillance systems (i.e. data sources) in Belgium (see Additional file 2). The set of questions was compiled based on the literature, including several review articles [1,2,3, 5, 7, 24,25,26, 44, 45]. Existing items from previous survey questionnaires [27, 36,37,38,39,40,41,42, 46]) were used and adapted when relevant. Most of the existing questionnaires from which some questions were adapted to be used for this survey were not validated, except for Chow-White et al. 2017 as mentioned in the respective publication . The construction of the survey was discussed during several feedback rounds within a multidisciplinary team including epidemiologists, microbiologists, and molecular biologists. As a result, the survey instrument was vetted by subject matter experts. The questionnaire eventually contained eight main topics comprising background questions, general attitude towards pathogen genomics for public health practice and main concerns, genomic literacy, current and planned NGS activities, place of NGS in the diagnostic hierarchy of microbiology, data sharing obstacles, end-user requirements, and key drivers for the implementation of NGS. Based on a filter question where participants indicated their level of familiarity with pathogen genomics, the respondents were redirected to different sets of questions with different levels of technicity and detail. The filter question gave access to a reduced version of the questionnaire for those participants judging themselves as not at all familiar with pathogen genomics. The responses were mainly collected as single/multiple options from a set of pre-defined answers, but also included the optional entry of free text. These qualitative open questions were included to add context to the quantitative responses. The survey tool was pre-tested by three researchers not directly involved in the development phase to ensure the acceptability and clarity of the questionnaire.
Participants were contacted individually by an email invitation containing a personal token to complete the survey. No monetary or other incentive was offered. The participant information statement at the beginning of the survey informed the respondents about the objective and design of the study and their rights before participation to the survey, and explained that responses are anonymized and will be kept confidential. The approval from an Ethical Committee was not considered necessary due to national regulations (legislation 4 April 2014), as this study was not medical in nature and as participants were not subject to any actions and/or rules of conduct. The survey was available online during a two months’ period during which three reminders were sent to those who had not yet responded. The first invitations were sent on the 6th of May 2019 and the survey remained active until the 1st of July 2019. Participants were invited to send any questions, feedback or comments for the survey to the organizers. Only completed questionnaires were used for analysis.
Descriptive statistics were reported by analyzing categorical response frequencies. Differences in viewpoints between the stakeholders were described using subgroup analyses and compared using a Fisher’s exact test. Subgroups were compiled on the basis of the level of action (national vs. local), the position in the information cycle (data providers vs. data end-users), and the level of expertise in the field of pathogen genomics. The level of action was considered national when the main affiliation of the respondent concerned a national institute involved in national public health activities, whereas the local level included professionals who mainly operate at the community, hospital, or university level. Subgroups based on the position in the information cycle were defined as data providers, defined here as experts in wet and dry lab procedures and (potentially) generating NGS data (including microbiologists, molecular biologists, clinical biologists, and bioinformaticians), and data end-users defined here as using NGS data to improve their activities and implementing infection control measures (including epidemiologists, local infection control practitioners, hospital hygienists, and clinicians). The level of expertise was categorized as high, middle or low, and was based on respondents’ self-reported familiarity with pathogen genomics, training level, and current use of NGS. Multiple logistic regression was performed to identify predictors of a positive attitude towards the implementation of pathogen genomics from a public health perspective. Enthusiasm about public health agencies using genomics to understand and control infectious diseases was defined directly through a question with multiple options, each containing a clear statement (see Additional file 1). For the purpose of this analysis, the question asking about their enthusiasm originally consisting of multiple categories was collapsed into two levels: very enthusiastic versus all others. The following predictor variables were initially tested in the model: level of action; position in the information cycle; level of expertise; current use of NGS; institution; age group; years of professional experience; and position in their institution. Model building involved a univariate analysis to select variables to be included in the multivariable model based on a χ2-test (cut-off, P = 0.25), and variable selection from the multivariable model using backward stepwise regression based on the Akaike Information Criterion (AIC). Adjusted odds ratios (ORs) and 95% confidence intervals (CIs) were calculated. Quantitative analyses were performed using R software (R Studio version 1.0.153) . Answers to open-ended survey questions were summarized and analyzed using Nvivo qualitative data analysis software (Nvivo Version 12) . This was done by identifying themes (codes) within the data, which were derived both deductively and inductively. Following the thematic analysis framework, the text was compared and contrasted with the identified codes. The qualitative findings were summarized as a mind map linking the identified major and minor themes and a word cloud visualizing the word frequency from the qualitative responses. Simultaneously, quotes were selected for the sake of illustration.
Profile of the respondents
Out of the 753 invited participants, 465 did not respond at all, 142 partially filled in the survey, and a total of 146 participants delivered a completed survey which represents an overall survey response rate of 19% (Fig. 1). From these, 135 participants continued after the filter question and delivered answers to all questions (116 subject were redirected to a technical version of the survey and 19 subjects to a basic version, based on the filter question). The data from the 11 participants who preferred to quit after the filter question were only used to describe the background characteristics of the study population. The 142 subjects who partially filled in the survey were dropped completely from the analysis. Full responses to all questions as they appeared in the questionnaire are provided as an appendix to this report (see Additional file 3).
Background characteristics of the 146 participants are presented in Table 1. The majority of respondents had their main affiliation in the public sector (53%), followed by hospitals (including university hospitals) (36%), private sector (8%), and university (3%). The public sector was primarily represented by Sciensano (Belgian Institute for Health), comprising 42% of all survey participants (61/146). 53% of the respondents indicated that they had more than 10 years of professional experience within the field of infectious diseases. The reported roles of respondents within their institutions included: microbiologists/molecular biologists/bioinformaticians/clinical biologists (45%); epidemiologists (21%); clinicians (13%); hospital hygienists/infection control practitioners (10%); and policy makers (3%).
The 146 survey respondents were asked to describe their level of familiarity with sequencing technologies and pathogen genomics using following classification: ‘Very - I am involved in the generation and/or use of NGS data’ (22%), ‘Somewhat - I have a general sense of the applications of NGS’ (58%), or ‘Not at all - I don't know anything about NGS and its applications’ (21%). Of those participants answering ‘very familiar’, most of them (88%; 28/32) indicated that they mainly used NGS in the context of WGS. Of those ‘not at all familiar’, 11 preferred to quit the survey and 19 continued the survey to answer some general questions, leaving a total of 135 participants for the remainder of the survey (Fig. 1). Subgroup analysis showed differences in familiarity with pathogen genomics between data providers and end-users (Fig. 2). Data providers indicated significantly more frequently that they were ‘very familiar’ compared to data end-users (p < 0.001).
The majority of respondents (63%; 85/135) indicated that they were very enthusiastic (i.e. ‘we should be using genomics now’) about public health agencies using genomics to understand and control infectious diseases, 30% (40/135) did not have an opinion or did not know enough of the topic to be able to give an opinion, and 7% (10/135) indicated that they did not see clear applications and/or an added value for public health. Subgroup analysis pointed out differences in enthusiasm according to the level of expertise in the field of pathogen genomics (Fig. 3). Important predictors, as identified by the best fitting model, of a positive attitude related to the implementation of pathogen genomics from a public health perspective were the level of expertise, the level of action, and the position in the information cycle (Table 2). Participants classified as having a high level of expertise based on their self-reported familiarity with the topic, their training level, and/or the current use of NGS were significantly more likely to be enthusiastic about the implementation of pathogen genomics in a public health context compared to their peers with a low expertise (adjusted OR = 4.04, 95% CI = 1.11 – 17.23). Further, public health professionals operating at the national level were more often ‘very enthusiastic’ about the implementation of pathogen genomics (71%) compared to those at the local level (53%). Similarly, data providers were more often ‘very enthusiastic’ (76%) compared to data end-users (50%).
A large majority of respondents considered the following public health activities as likely to be most impacted by pathogen genomics in the next five years: identifying an outbreak (clusters of related isolates) (78%; 105/135), nosocomial and food/waterborne outbreak investigations (76%; 103/135), and monitoring the spread of antimicrobial resistance (71%; 96/135). In contrast, only 44% (60/135) of respondents thought that pathogen genomics would have a major impact on making a diagnosis and selecting an appropriate treatment (individual patient management). Other public health activities that will benefit from the implementation of pathogen genomics mentioned by the participants are presented in Table 3.
The most frequent concerns among participants being ‘very’ or ‘somewhat’ familiar with NGS technologies and pathogen genomics (n=116) regarding feasibility of its routine use for public health purposes, were the cost of sequencing technologies and the existing barriers to timely and open sharing of pathogen sequence data and accompanying metadata (Table 4). All participants exclusively working with respiratory infections (e.g. influenza) and/or vaccine-preventable diseases (e.g. measles) (n=7) were very concerned about the cost, whereas this was only true for 38% of participants exclusively working with invasive bacterial diseases (e.g. Neisseria meningitidis), food- and waterborne diseases (e.g. Salmonella), and/or healthcare-associated infections (e.g. Clostridium difficile) (n=21). Further, other concerns shared by a large proportion of the participants were interdisciplinary cooperation, integration of pathogen sequence data with contextual data, access to bioinformatics expertise, and availability of typing schemes and databases. Participants indicating to be ‘not at all’ familiar with pathogen genomics were mainly concerned about the cost of the sequencing technologies (see full responses in Additional file 3). Other concerns provided by the participants as free text are presented in Table 3.
Two-thirds of the participants (88/135) indicated that they had followed training in the fields of genomics/genetics/molecular biology/bioinformatics. There were marked differences by position in the information cycle: 56% (38/68) of data end-users indicated that they had never followed any training in the field, whereas this was stated by only 13% (9/67) of data providers (p < 0.001). Further breakdown of training experience by professional category is shown in Fig. 4.
The main reasons for not taking a training/course in this field (yet) were the lack of available and/or suitable trainings (40%; 19/47) and the lack of time (30%; 14/47). Other reasons indicated as free text are presented in Table 3. The vast majority of participants (79%, 106/135) indicated that they felt the need and/or would be interested in following (additional) courses/training/workshops covering a topic related to pathogen genomics.
Current and planned NGS activities
Overall, 36% (42/116) participants being ‘very’ or ‘somewhat’ familiar with NGS technologies and pathogen genomics indicated that they are currently using or generating NGS data for at least one pathogen. Differences between professional groups are presented in Fig. 5. Among the microbiologists, those from a National Reference Centre (NRC) were more likely to be currently using NGS (12/18, i.e. 67%) compared to those from other laboratories (1/7, i.e. 14%), however this difference was not significant (p=0.71). From the public health professionals exclusively involved in human infectious disease activities, 30% (21/70) were currently using NGS technologies, whereas this was the case for 44% (11/25) of those exclusively involved in the food, animal or environmental sector (p=0.23).
Looking forward, 44% (51/116) of participants indicated that they were planning to use or generate NGS data for any (additional) pathogen(s) within three years. Details on the specified pathogens can be found in the appendix (see Additional file 3). Reasons provided by participants indicating that they did not plan to implement pathogen genomics were mainly related to the cost and the lack of expertise.
Key drivers for implementation
Participants being ‘very’ or ‘somewhat’ familiar with NGS technologies and pathogen genomics (n=116) were asked to assign a score from 1 to 5 to the different criteria based on their increasing relative importance to decide whether or not NGS should be implemented for a particular pathogen (Fig. 6). Clinical and/or public health significance of the pathogen were scored as the most important drivers. The different subgroups scored the different criteria similarly (see Additional file 3). Comments provided by the participants to provide context to their scores are presented in Table 3.
Laboratory workflow integration
Centralization of sequencing and bioinformatics at NRCs organized per pathogen or per group of pathogens was most often (34%; 40/116) selected by respondents being ‘very’ or ‘somewhat’ familiar with pathogen genomics as the preferred WGS provision model in the Belgian context. Excluding participants working at NRCs slightly lowered this proportion to 29 out of 98 (i.e., 30%). There were no marked differences according to the level of action of the participants (Fig. 7). Illustrative quotes for the need for centralization are presented in Table 3.
A majority (59%; 68/116) of the participants being ‘very’ or ‘somewhat’ familiar with pathogen genomics considered technical barriers (lack of data standardization, poor data quality, missing metadata, etc.) as a major obstacle for sharing pathogen sequence data and associated metadata, whereas only 38% (44/116) cited ethical issues and concerns, 35% (41/116) political sensitivities, and 27% (31/116) priority to scientific publication. The proportion of participants indicating priority to scientific publication as a major obstacle for data sharing did not significantly differ between those primarily affiliated to a university (including university hospitals) compared to the other participants. Concerns about organizational aspects are presented in Table 3.
Qualitative analysis results
Major themes identified within the qualitative data are utility (applications), feasibility (including capacity building, multi-disciplinary working, contextual data, costs, data sharing, ethics, timeliness, wet and dry lab), One Health context, and routine implementation (including organization and translation into action). A mind map linking the identified major and minor themes is presented in Fig. 8. A full list of identified themes and the coded text is available in the appendix (see Additional file 4), as well as a word cloud constructed based on the free text responses (see Additional file 5).
This survey sought the opinion of Belgian public health professionals working in the field of infectious diseases concerning the implementation of pathogen genomics in public health activities. To successfully translate pathogen genomics into public health practice, the needs and expectations of the different stakeholders should be taken into account. Other questionnaire surveys related to knowledge and attitudes towards public health genomics in specific health expert categories have been published [27, 36,37,38,39, 41, 42, 46, 49]. However, to the best of our knowledge this survey is the first that aimed to perform a wide landscape analysis of all potentially involved stakeholders. Therefore, a strength of the current study is that it took into account a wide range of stakeholders with diverse backgrounds (epidemiologists, microbiologists, bioinformaticians, clinicians, infection control practitioners, etc.), health domains (human, food, environmental, etc.), pathogen expertise (bacteria, viruses, parasites, fungi, etc.), activity sectors (public, private, university, hospital, etc.), work positions (employee and lower/middle/high management), and degree of familiarity with genomics. Besides seeking the general attitude of the participants towards the implementation of pathogen genomics in their professional activities and investigating the current and future use, this explorative study was able to touch upon multiple key topics, such as genomic literacy, data sharing obstacles, place of NGS in the diagnostic hierarchy of microbiology, and end-user requirements.
Familiarity with sequencing technologies and pathogen genomics varied between the different professional groups, with data providers being more familiar than data end-users. As shown before, one of the largest barriers to acceptability from the public health unit is the capacity to understand and use the data . Possibly, there is a positive association between genomic literacy and (critically) predicting the added value in public health. Having a high level of expertise, was the strongest predictor for a positive attitude, as was also shown in other surveys [51,52,53]. Epidemiologists and infection control practitioners should be informed about the benefits and limitations of NGS technologies in order to contribute in identifying tangible field application in public health, allowing the use of WGS output to appropriately guide public health actions [38, 51]. Another important challenge related to the interpretation of WGS data is the capacity to interpret signals, and thereby separating noise from public health events that require specific actions. Consequently, integrating genomics into infection control and surveillance is critically linked to human resource development [8, 24]. In the survey, the main reasons stated for not training in the field of genomics were lack of time or access to suitable trainings “…adapted to public health needs”. However, the participants of this survey generally expressed a positive attitude towards following (additional) training courses, or workshops in pathogen genomics. Educational workshops should be applied to a public health context and bring together the expertise of microbiologists, molecular experts, bioinformaticians, epidemiologists, infection control practitioners, and clinicians. The development of a new discipline called ‘genomic epidemiology’ integrating information on epidemiological and pathogen sequence characteristics by public health microbiologists, epidemiologists, and risk managers was recommended in the expert opinion on WGS for public health surveillance by the European Centre for Disease Prevention and Control (ECDC) in 2016 . ECDC has initiated public health genomics training workshops that bring together experts with epidemiology, microbiology and pathogen genomics backgrounds from European Union (EU) member states with interest in implementing the technology in surveillance and outbreak investigations. Besides, the zoonotic origin of many clinically relevant pathogens and antimicrobial resistance determinants stresses the importance of a cross-sectoral One Health approach. The implementation of WGS should be synchronized and integrated between the human health and veterinary sectors  allowing a better monitoring of the emergence and spread of zoonotic pathogens and antimicrobial resistance-related threats.
Lack of financial resources was often indicated as a principal reason for not using or planning to use WGS by the respondents of this survey, which was also reported by the European surveys conducted by ECDC  and the European Food Safety Authority (EFSA) . Operational costs will be influenced by the processes used in current laboratory practice and differs between viruses and bacteria. Whereas drug susceptibility testing and epidemiological typing are commonly performed for bacteria, this is often not the case for viruses detected in the routine laboratory . Therefore, cost-effectiveness of NGS for many bacteria potentially follows from the replacement of conventional characterization methods, whereas for viruses NGS is considered as a tool providing additional complementary information without replacement of the existing methods. Further, an important consideration is the added value of NGS for routine diagnostics. As long as NGS is more expensive than the conventional methods and when there is no direct benefit for the individual patient, it will not be used in routine. Then the fields of application for surveillance purposes should be clearly defined to be able to justify the additional financial resources needed to perform WGS beside the diagnostic activities.
To translate pathogen sequence data into truly useful and actionable information, it needs to be integrated with other types of information (i.e. clinical and epidemiological data). In Belgium, most data end-users were concerned about the challenges encountered with the integration of pathogen sequence data with clinical and epidemiological data. Indeed, the public health usability of any kind of lab results, including WGS data, is highly dependent on the cross-linkage with contextual epidemiological and clinical information [8, 55, 56]. Data integration is often hampered by the incomplete and/or unstandardized nature of the contextual data . The ongoing digitalization of health data such as laboratory and clinical records may represent an opportunity to review and upgrade traditional data collection processes for communicable disease surveillance.
According to World Health Organization’s (WHO) 2016 guidance on managing ethical issues in outbreaks , rapid data sharing is crucial during an unfolding health emergency. This suggests that pathogen sequence data should be rapidly and openly shared at the start of an outbreak, in many cases before scientific publication. However, many barriers for data sharing remain including authorship/attribution for publications, results dissemination, ethical considerations, data ownership, database access agreements, etc. . In our survey, practical barriers (lack of data standardization, poor data quality, missing metadata, etc.) seemed to be the major obstacles in Belgium for sharing pathogen sequence data and associated metadata for public health purposes. Participants mainly mentioned the lack of a central database and clear guidelines. This reflects a lack of information on the effective data sharing through EU-wide genomic surveillance and cross-border outbreak analysis systems managed by ECDC and EFSA in support of the member states [9, 60, 61]. Finally, 27% of participants considered the priority to publication as a major bottleneck for sharing pathogen sequence data. Publication priority is linked to the importance of guaranteeing reputational returns to research efforts [40, 62]. The challenge here is to find a balanced arrangement that allows data sharing in real time and the acknowledgement of research work by giving to researchers who have been involved in data generation the possibility to use and publish their own results in priority. As the use of NGS shifts from research to routine laboratory practices, this data sharing barrier will slowly be alleviated.
Regarding expertise and availability of personnel, wet and dry lab experts were more concerned about the analysis of pathogen sequence data than the sequencing itself. As was mentioned in a review article of Aarestrup et al. and documented in a recent European survey by Revez et al., the most important limiting factor in many countries is the lack of access to bioinformatics expertise, especially when used as part of frontline diagnostics  or national public health reference laboratory service .
Another point of discussion is the potential impact of NGS on the diagnostic microbiology pathway. Traditionally, frontline clinical laboratories perform standard identification, antimicrobial susceptible testing and occasionally typing. Isolates may then be referred to reference laboratories based on the need (e.g. diagnostic confirmation) or for surveillance purposes. These reference laboratories perform confirmation testing and advanced characterization. NGS was first implemented at the level of academic or reference laboratories, because of the need for investments, operational costs, and requirements for expertise  while having limited added value for individual patient care. Samples must be multiplexed (batching) for cost-effectiveness, which is easier to achieve in large reference laboratories with high volume of sample throughput [26, 45]. However, processing delays may be present when samples are shipped to a reference center. These processing delays may result in longer turnaround times rendering this centralized approach inappropriate to support a fast response when needed. The reduced costs of sequencing facilitated the introduction of NGS technology to frontline clinical laboratories. This shift towards a decentralized use may reduce turnaround times, empower hospital-based microbiology, and strengthen local infection control efforts . This decentralized capacity will allow the inclusion of these data in the surveillance network coordinated by the epidemiologists what will compensate the reduced referral of isolates to reference centers. Consequently, the implementation of NGS in routine labs is an important driver to reconsider the future role of NRCs. Molecular typing for public health surveillance is undergoing a stepwise transition to NGS . Current and future NGS activities represented in this national survey were mainly in the context of food- and waterborne outbreak detections and investigations, reflecting the priority for these diseases across Europe and beyond [63, 64]. Several criteria should be considered in the process of integrating WGS in a routine laboratory setting  in order to know in which situations and for which pathogens it is worthwhile to use NGS. Identifying a set of key drivers that cover all aspects related to the implementation of NGS (utility and feasibility) can help to guide prioritization of pathogens and to efficiently allocate resources. Clinical and/or public health significance of the pathogen was scored as the most important driver during the implementation of pathogen genomics in routine public health activities, followed by availability of expertise to generate, analyze and interpret WGS data, and priority of the pathogen with respect to preventing the spread of antimicrobial resistance.
Qualitative responses revealed the opinion of several participants that the assessment of the added value of new technologies for individual patient care is paramount. If pathogen genomics is routinely used to guide patient management (diagnosis and/or treatment options), the pathogen sequence data gathered for diagnostic purposes can be accumulated for public health activities . If there is no added value for routine diagnosis, the cost of WGS will have to be covered by limited public health budgets.
As a limitation, the relatively low response rate induced a potential volunteer bias as those public health experts being more interested and/or experienced in the field could be more likely to participate in the survey. Yet, 21% of the participants indicated that they were ‘not at all’ familiar with sequencing technologies and pathogen genomics. Further, we noticed a possible underrepresentation of the food, animal and environmental field in comparison to the human field, as well as a low number of bioinformaticians in the survey. In addition, public health professionals from the Belgian Institute for Health (Sciensano) might be overrepresented. The majority of microbiologists participating in the survey are based in a NRC, emphasizing surveillance activities and hence less weight to routine diagnostics. Given this potential imbalance, it is important to take into account the distribution of profiles within the study population while interpreting the results. However, it is difficult to ascertain the true underlying distribution of the different professional groups within the target population. Another limitation of the study is that the specific terminology used in the questions may not have been uniformly understood or consistently interpreted by stakeholders with different professional backgrounds .
Public health professionals working in the field of infectious diseases in Belgium were in general enthusiastic about public health agencies implementing pathogen genomics for the surveillance and control of infectious diseases. However, introducing genomic methods into public health practice is inevitably linked to the decrease in cost, the introduction in routine activities of frontline clinical labs, the identification of field applications in public health, and the necessary development of new competencies. The results of the survey confirm the need to increase genomic literacy by offering dedicated training opportunities among public health professionals, especially for the data end-users including epidemiologists, clinicians, and infection control practitioners, enabling them to critically assess the utility and feasibility of implementing pathogen genomics in their work activities. As such, those at the forefront (i.e. end-users) may act as “honest brokers” responsible for evaluating the added value of genomic application. In the end, the main driver for the advancement of pathogen genomics in public health practice depends on the added value of this information for the different clinical and public health needs. Further, inter-disciplinary (between epidemiologists, microbiologists and bioinformaticians) and inter-sectoral (One Health context) collaboration should be improved in the future to pool expertise and to ensure an integrated and cohesive system for the management of infectious diseases. In terms of feasibility, respondents in this survey were mainly concerned, like their peers in similar European surveys, about data integration, data sharing, and the cost of sequencing technologies. Overall, this survey helps to better understand the perceived utility and feasibility of pathogen genomics according to public health professionals and can inform further guidance to facilitate its implementation in Belgium. Future challenges can be anticipated by performing a similar survey among public health experts based in a country that already progressed further in the process of implementing pathogen genomics within their public health surveillance system.
Availability of data and materials
The raw data analyzed during the current study are available from the corresponding author on reasonable request.
Akaike Information Criterion
Belgian Antibiotic Policy Coordination Committee
Belgian Infection Control Society
Belgian Society for Food Microbiology
Belgian Society of Infection Specialists and Clinical Microbiologists
European Centre for Disease Prevention and Control
European Food Safety Authority
Federal Agency for the Safety of the Food Chain
Next generation sequencing
National Reference Centre
Single nucleotide polymorphism
World Health Organization
Kan B, Zhou H, Du P, Zhang W, Lu X, Qin T, et al. Transforming bacterial disease surveillance and investigation using whole-genome sequence to probe the trace. Front Med. 2018;12:23–33.
Gwinn M, MacCannell DR, Khabbaz RF. Integrating Advanced Molecular Technologies into Public Health. J Clin Microbiol. 2017;55:703–14.
Besser J, Carleton HA, Gerner-Smidt P, Lindsey RL, Trees E. Next-generation sequencing technologies and their application to the study and control of bacterial infections. Clin Microbiol Infect. 2018;24:335–41.
Gwinn M, MacCannell D, Armstrong GL. Next Generation Sequencing of Infectious Pathogens. JAMA. 2019;321:893–4.
Aarestrup FM, Brown EW, Detter C, Gerner-Smidt P, Gilmour MW, Harmsen D, et al. Integrating Genome-based Informatics to Modernize Global Disease Monitoring, Information Sharing, and Response. Emerg Infect Dis. 2012;18:e1.
Van Goethem N, Descamps T, Devleesschauwer B, Roosens NHC, Boon NAM, Van Oyen H, et al. Status and potential of bacterial genomics for public health practice: a scoping review. Implementation Sci. 2019;14:79.
Struelens MJ, Brisse S. From molecular to genomic epidemiology: transforming surveillance and control of infectious diseases. Eurosurveillance. 2013;18:20386.
Struelens M, Rupnik M. Expert opinion on whole genome sequencing for public health surveillance strategy to harness whole genome sequencing to strengthen EU outbreak investigations and public health surveillance. Stocholm: European Centre for Disease Prevention and Control; 2016. https://ecdc.europa.eu/sites/portal/files/media/en/publications/Publications/whole-genome-sequencing-for-public-health-surveillance.pdf. Accessed 26 Jun 2020.
European Centre for Disease Prevention and Control. ECDC strategic framework for the integration of molecular and genomic typing into European surveillance and multi-country outbreak investigations – 2019–2021; 2019. p. 54.
Hassan R. Multistate Outbreak of Salmonella Anatum Infections Linked to Imported Hot Peppers — United States, May–July 2016. MMWR Morb Mortal Wkly Rep. 2017;66. https://doi.org/10.15585/mmwr.mm6625.
Mair-Jenkins J, Borges-Stewart R, Harbour C, et al. Investigation using whole genome sequencing of a prolonged restaurant outbreak of Salmonella Typhimurium linked to the building drainage system, England, February 2015 to March 2016. Euro Surveillance : Bulletin Europeen sur les Maladies Transmissibles = European Communicable Disease Bulletin. 2017;22(49). https://doi.org/10.2807/1560-7917.es.2017.22.49.17-00037.
Davis RJ, Jensen SO, Van Hal S, Espedido B, Gordon A, Farhat R, et al. Whole Genome Sequencing in Real-Time Investigation and Management of a Pseudomonas aeruginosa Outbreak on a Neonatal Intensive Care Unit. Infect Control Hosp Epidemiol. 2015;36:1058–64.
Lapierre P, Nazarian E, Zhu Y, Wroblewski D, Saylors A, Passaretti T, et al. Legionnaires’ Disease Outbreak Caused by Endemic Strain of Legionella pneumophila, New York, New York, USA, 2015. Emerging Infect Dis. 2017;23:1784–91.
Quick J, Ashton P, Calus S, Chatt C, Gossain S, Hawker J, et al. Rapid draft sequencing and real-time nanopore sequencing in a hospital outbreak of Salmonella. Genome Biol. 2015;16:114.
Weiss D, Boyd C, Rakeman JL, Greene SK, Fitzhenry R, McProud T, et al. A Large Community Outbreak of Legionnaires’ Disease Associated With a Cooling Tower in New York City, 2015. Public Health Rep. 2017;132:241–50.
Gillesberg Lassen S, Ethelberg S, Björkman JT, Jensen T, Sørensen G, Kvistholm Jensen A, et al. Two listeria outbreaks caused by smoked fish consumption—using whole-genome sequencing for outbreak investigations. Clin Microbiol Infect. 2016;22:620–4.
Inns T, Lane C, Peters T, et al. A multi-country Salmonella Enteritidis phage type 14b outbreak associated with eggs from a German producer: 'near realtime'application of whole genome sequencing and food chain investigations, United Kingdom, May to September 2014. Euro Surveill. 2015;20(16):21098.Published; 2015. https://doi.org/10.2807/1560-7917.es2015.20.16.21098.
Dyson ZA, Thanh DP, Bodhidatta L, Mason CJ, Srijan A, Rabaa MA, et al. Whole Genome Sequence Analysis of Salmonella Typhi Isolated in Thailand before and after the Introduction of a National Immunization Program. PLOS Neglected Trop Dis. 2017;11:e0005274.
Hill DMC, Lucidarme J, Gray SJ, Newbold LS, Ure R, Brehony C, et al. Genomic epidemiology of age-associated meningococcal lineages in national surveillance: an observational cohort study. Lancet Infect Dis. 2015;15:1420–8.
Sealey KL, Harris SR, Fry NK, Hurst LD, Gorringe AR, Parkhill J, et al. Genomic analysis of isolates from the United Kingdom 2012 pertussis outbreak reveals that vaccine antigen genes are unusually fast evolving. J Infect Dis. 2015;212:294–301.
Rodrigues CMC, Maiden MCJ. A world without bacterial meningitis: how genomic epidemiology can inform vaccination strategy. F1000Res. 2018;7. https://doi.org/10.12688/f1000research.13793.1.
Ellington MJ, Reuter S, Harris SR, Holden MTG, Cartwright EJ, Greaves D, et al. Emergent and evolving antimicrobial resistance cassettes in community-associated fusidic acid and meticillin-resistant Staphylococcus aureus. Int J Antimicrob Agents. 2015;45:477–84.
Hughes J, Stabler R, Gaunt M, Karadag T, Desai N, Betley J, et al. Clonal variation in high- and low-level phenotypic and genotypic mupirocin resistance of MRSA isolates in south-east London. J Antimicrob Chemother. 2015;70:3191–9.
Rossen JWA, Friedrich AW, Moran-Gilad J. Practical issues in implementing whole-genome-sequencing in routine diagnostic microbiology. Clin Microbiol Infect. 2018;24:355–60.
Tang P, Croxen MA, Hasan MR, Hsiao WWL, Hoang LM. Infection control in the new age of genomic epidemiology. Am J Infect Control. 2017;45:170–9.
Balloux F, Brønstad Brynildsrud O, van Dorp L, Shaw LP, Chen H, Harris KA, et al. From Theory to Practice: Translating Whole-Genome Sequencing (WGS) into the Clinic. Trends Microbiol. 2018;26:1035–48.
Kwong JC, McCallum N, Sintchenko V, Howden BP. Whole genome sequencing in clinical and public health microbiology. Pathology. 2015;47:199–210.
Tanca A, Deligios M, Addis MF, Uzzau S. High throughput genomic and proteomic technologies in the fight against infectious diseases. J Infect Dev Ctries. 2013;7:182–90.
Lecuit M, Eloit M. The potential of whole genome NGS for infectious disease diagnosis. Expert Rev Mol Diagn. 2015;15:1517–9.
Pankhurst LJ, Del Ojo EC, Votintseva AA, Walker TM, Cole K, Davies J, et al. Rapid, comprehensive, and affordable mycobacterial diagnosis with whole-genome sequencing: a prospective study. Lancet Respir Med. 2016;4:49–58.
Didelot X, Bowden R, Wilson DJ, Peto TEA, Crook DW. Transforming clinical microbiology with bacterial genome sequencing. Nat Rev Genet. 2012;13:601–12.
Durand G, Javerliat F, Bes M, Veyrieras J-B, Guigon G, Mugnier N, et al. Routine Whole-Genome Sequencing for Outbreak Investigations of Staphylococcus aureus in a National Reference Center. Front Microbiol. 2018;9:511.
Holmes A, Allison L, Ward M, Dallman TJ, Clark R, Fawkes A, et al. Utility of Whole-Genome Sequencing of Escherichia coli O157 for Outbreak Detection and Epidemiological Surveillance. J Clin Microbiol. 2015;53:3565–73.
Johnson I, Hansen A, Bi P. The challenges of implementing an integrated One Health surveillance system in Australia. Zoonoses Public Health. 2018;65:e229–36.
Mackenzie JS, Jeggo M. The One Health Approach—Why Is It So Important? Trop Med Infect Dis. 2019;4. https://doi.org/10.3390/tropicalmed4020088.
Geskin A, Legowski E, Chakka A, Chandran UR, Barmada MM, LaFramboise WA, et al. Needs Assessment for Research Use of High-Throughput Sequencing at a Large Academic Medical Center. Plos One. 2015;10:e0131166.
Chow-White P, Ha D, Laskin J. Knowledge, attitudes, and values among physicians working with clinical genomics: a survey of medical oncologists. Hum Resour Health. 2017;15:42.
Rosso A, D’Andrea E, Marco MD, Pitini E, Unim B, Vito CD, et al. European survey on knowledge and attitudes of public health professionals on public health genomics: Pilot Study. Epidemiol Biostatistics Public Health. 2017;14. https://doi.org/10.2427/12531.
Crisan A, McKee G, Munzner T, Gardy JL. Evidence-based design and evaluation of a whole genome sequencing clinical report for the reference microbiology laboratory. PeerJ. 2018;6:e4218.
Ribeiro C dos S, Roode MY van, Haringhuizen GB, Koopmans MP, Claassen E, Burgwal LHM van de. How ownership rights over microorganisms affect infectious disease control and innovation: A root-cause analysis of barriers to data sharing as experienced by key stakeholders. Plos One. 2018;13:e0195885.
Revez J, Espinosa L, Albiger B, Leitmeyer KC, Struelens MJ, ECDC, National Microbiology Focal Points and Experts Group. Survey on the Use of Whole-Genome Sequencing for Infectious Diseases Surveillance: Rapid Expansion of European National Capacities, 2015-2016. Front Public Health. 2017;5:347.
Fierro RG, Thomas-Lopez D, Deserio D, Liebana E, Rizzi V, Guerra B. Outcome of EC/EFSA questionnaire (2016) on use of Whole Genome Sequencing (WGS) for food- and waterborne pathogens isolated from animals, food, feed and related environmental samples in EU/EFTA countries. EFSA Support Publ. 2018;15:1432E.
LimeSurvey Project Team / Carsten Schmitz. LimeSurvey: An Open Source survey tool. Hamburg: LimeSurvey Project; 2012. http://www.limesurvey.org.
Aarestrup FM, Koopmans MG. Sharing Data for Global Infectious Disease Surveillance and Outbreak Detection. Trends Microbiol. 2016;24:241–5.
Arnold C. Considerations in centralizing whole genome sequencing for microbiology in a public health setting. Expert Rev Mol Diagn. 2016;16:619–21.
Moran-Gilad J, Sintchenko V, Pedersen SK, Wolfgang WJ, Pettengill J, Strain E, et al. Proficiency testing for bacterial whole genome sequencing: an end-user survey of current capabilities, requirements and priorities. BMC Infect Dis. 2015;15:174.
R Core Team. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2017. https://www.R-project.org/.
QSR International Pty Ltd. NVivo (Version 12); 2018. https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home.
European Centre for Disease Prevention and Control. Monitoring the use of whole-genome sequencing in infectious disease surveillance in Europe 2015–2017. European Centre for Disease Prevention and Control. 2018. https://www.ecdc.europa.eu/en/publications-data/monitoring-use-whole-genome-sequencing-infectious-disease-surveillance-europe. Accessed 28 Jun 2020.
Ford L, Carter GP, Wang Q, Seemann T, Sintchenko V, Glass K, et al. Incorporating Whole-Genome Sequencing into Public Health Surveillance: Lessons from Prospective Sequencing of Salmonella Typhimurium in Australia. Foodborne Pathog Dis. 2018;15:161–7.
Marzuillo C, Vito CD, D’Addario M, Santini P, D’Andrea E, Boccia A, et al. Are public health professionals prepared for public health genomics? A cross-sectional survey in Italy. BMC Health Serv Res. 2014;14:239.
Chen L-S, Goodson P. Public Health Genomics knowledge and attitudes: A survey of public health educators in the United States. Genet Med. 2007;9:496–503.
Chen L-S, Kwok O-M, Goodson P. US health educators’ likelihood of adopting genomic competencies into health promotion. Am J Public Health. 2008;98:1651–7.
Köser CU, Ellington MJ, Cartwright EJP, Gillespie SH, Brown NM, Farrington M, et al. Routine Use of Microbial Whole Genome Sequencing in Diagnostic and Public Health Microbiology. Plos Pathogens. 2012;8:e1002824.
Muellner P, Stärk KDC, Dufour S, Zadoks RN. “Next-Generation” Surveillance: An Epidemiologists’ Perspective on the Use of Molecular Information in Food Safety and Animal Health Decision-Making. Zoonoses Public Health. 2016;63:351–7.
Franz E, Gras LM, Dallman T. Significance of whole genome sequencing for surveillance, source attribution and microbial risk assessment of foodborne pathogens. Curr Opin Food Sci. 2016;8:74–9.
Hill AA, Crotta M, Wall B, Good L, O’Brien SJ, Guitian J. Towards an integrated food safety surveillance system: a simulation study to explore the potential of combining genomic and epidemiological metadata. R Soc Open Sci. 2017;4:160721.
World Health Organization. Guidance for managing ethical issues in infectious disease outbreaks. World Health Organization; 2016. https://apps.who.int/iris/bitstream/handle/10665/250580/9789241549837-eng.pdf;jsessionid=FF77AAE57960850C51129D423FD20AA8?sequence=1.
World Health Organization. WHO R&D Blueprint meeting on pathogen genetic sequence data (GSD) sharing in the context of public health emergencies, 28-29 September 2017. 2017.
European Centre for Disease Prevention and Control and European Food Safety Authority. Multi-country outbreak of Listeria monocytogenes clonal complex 8 infections linked to consumption of cold-smoked fish products – 4 June 2019. 2019.
European Centre for Disease Prevention and Control and European Food Safety Authority. Collection and analysis of whole genome sequencing data from food-borne pathogens and other relevant microorganisms isolated from human, animal, food, feed and food/feed environmental samples in the joint ECDC–EFSA molecular typing database. 2019. https://www.ecdc.europa.eu/sites/default/files/documents/food-borne-pathogens-collection-analysis-whole-genome-sequencing-data.pdf.
Kaye J, Heeney C, Hawkins N, de Vries J, Boddington P. Data Sharing in Genomics – Re-shaping Scientific Practice. Nat Rev Genet. 2009;10:331–5.
Cassini A, Colzani E, Kramarz P, Kretzschmar M, Takkinen J. Impact of food and water-borne diseases on European population health. Curr Opin Food Sci. 2016;12:21–9.
World Health Organization. WHO estimates of the global burden of foodborne diseases. WHO. 2015. http://www.who.int/foodsafety/publications/foodborne_disease/fergreport/en/. Accessed 28 Jun 2020.
Neher RA, Bedford T. Real-Time Analysis and Visualization of Pathogen Sequence Data. J Clin Microbiol. 2018;56. https://doi.org/10.1128/JCM.00480-18.
We would like to thank all study participants who took the time to complete the survey and provided valuable insights to reach to objectives of our study. These include the scientists from the Belgian Institute for Health (Sciensano) and the Federal Agency for the Safety of the Food Chain (FASFC) working in infectious disease departments; physicians and infection control practitioners of the regional infectious disease control teams from the three regions in Belgium; microbiologists from National Reference Centers (NRCs) and from sentinel laboratories; members from the Belgian Society of Infection Specialists and Clinical Microbiologists (BVIKM), the Belgian Antibiotic Policy Coordination Committee (BAPCOC), the Belgian Infection Control Society (BICS), and the Belgian Society for Food Microbiology (BSFM); clinicians, clinical biologists and hospital hygienists participating in sentinel surveillance networks; and public health experts within the ministry of health.
We also would like to thank Jérome Ambroise, Jimmy Van Den Eynden, and Boudewijn Catry for pre-testing the survey, and Vera Cantaert, Yves Dupont, Chloé Wyndham-Thomas, and Karl Mertens for their contributions in recruiting participants.
This research was supported by the .Be READY project financed by Sciensano.
Ethics approval and consent to participate
The first page of the questionnaire explained the purpose of the research, the measures taken to protect respondents’ confidentiality and the voluntary nature of participation. Questionnaire respondents were asked to tick ‘agree’ at the start of the questionnaire to indicate that they consented to take part in the study and were willing to complete an anonymized questionnaire. Ethics approval was deemed unnecessary according to national regulations (law of 4 April 2014; http://www.ejustice.just.fgov.be/mopdf/2014/05/16_1.pdf#Page97), as participation was anonymous and no medical data were processed.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
“Questionnaire”. Description of data: “List of questions included in the online survey”.
“Selection of target groups for the survey”. Description of data: “An overview of existing surveillance systems to identify all public health professionals who (would potentially) generate or use NGS data for the surveillance of infectious diseases based in different institutes and organizations in Belgium.”
“Full responses to questionnaire”. Description of data: “Full responses to all questions included in the survey”.
“Thematic analysis of open questions”. Description of data: “A full list of identified themes and the coded text”.
“Word cloud based on open questions”. Description of data: “Word cloud constructed based on the free text responses visualizing the word frequency”.
About this article
Cite this article
Van Goethem, N., Struelens, M.J., De Keersmaecker, S.C.J. et al. Perceived utility and feasibility of pathogen genomics for public health practice: a survey among public health professionals working in the field of infectious diseases, Belgium, 2019. BMC Public Health 20, 1318 (2020). https://doi.org/10.1186/s12889-020-09428-4