- Research article
- Open Access
The evidence base of primary research in public health emergency preparedness: a scoping review and stakeholder consultation
BMC Public Health volume 15, Article number: 432 (2015)
Effective public health emergency preparedness and response systems are important in mitigating the impact of all-hazards emergencies on population health. The evidence base for public health emergency preparedness (PHEP) is weak, however, and previous reviews have noted a substantial proportion of anecdotal event reports. To investigate the body of research excluding the anecdotal reports and better understand primary and analytical research for PHEP, a scoping review was conducted with two objectives: first, to develop a thematic map focused on primary research; and second, to use this map to inform and guide an understanding of knowledge gaps relevant to research and practice in PHEP.
A scoping review was conducted based on established methodology. Multiple databases of indexed and grey literature were searched based on concepts of public health, emergency, emergency management/preparedness and evaluation/evidence. Inclusion and exclusion criteria were applied iteratively. Primary research studies that were evidence-based or evaluative in nature were included in the final group of selected studies. Thematic analysis was conducted for this group. Stakeholder consultation was undertaken for the purpose of validating themes and identifying knowledge gaps. To accomplish this, a purposive sample of researchers and practicing professionals in PHEP or closely related fields was asked to complete an online survey and participate in an in-person meeting. Final themes and knowledge gaps were synthesized after stakeholder consultation.
Database searching yielded 3015 citations and article selection resulted in a final group of 58 articles. A list of ten themes from this group of articles was disseminated to stakeholders with the survey questions. Survey findings resulted in four cross-cutting themes and twelve stand-alone themes. Several key knowledge gaps were identified in the following themes: attitudes and beliefs; collaboration and system integration; communication; quality improvement and performance standards; and resilience. Resilience emerged as both a gap and a cross-cutting theme. Additional cross-cutting themes included equity, gender considerations, and high risk or at-risk populations.
In this scoping review of the literature enhanced by stakeholder consultation, key themes and knowledge gaps in the PHEP evidence base were identified which can be used to inform future practice-oriented research in PHEP.
For more than a decade, there has been substantial attention to and investment in emergency preparedness and response capacity for emergencies with health impacts. The importance of robust emergency preparedness and response systems for health emergencies is highlighted by recent incidents such as the Ebola outbreak in West Africa, the emergence of Middle East Respiratory Syndrome Coronavirus, natural disasters such as floods, and industrial-technical incidents like train derailments [1-4]. Preparedness for the diversity of events that may impact health is described as an ‘all-hazards’ approach, highlighting the importance of ensuring the system is prepared for a variety of potential threats. While there are multiple sectors with responsibilities for emergency preparedness and response, actions taken to prepare and respond to the population health consequences of emergencies fall to public health emergency preparedness (PHEP). PHEP has been defined as “the capability of the public health and health care systems, communities, and individuals, to prevent, protect against, quickly respond to, and recover from health emergencies, particularly those whose scale, timing, or unpredictability threatens to overwhelm routine capabilities” . Examples of actions of PHEP related to health emergencies include surveillance and epidemiologic activities to monitor, detect, and investigate potential health threats, and the development and communication of information to the public . In the context of the Severe Acute Respiratory Syndrome (SARS) outbreak of 2003, gaps were noted in the Canadian public health system including coordinated surveillance systems for detecting outbreaks and communication to the public .
Despite the investments into operational capacity for PHEP, a persistent challenge is a paucity of evidence to inform and guide practice [7-11]. Evidence-based approaches to public health practice have been recognized as crucial , alongside established practice in related fields of evidence-based medicine and health decision-making [13-16]. Evidence-based or informed practice in public health encompasses incorporating scientific evidence in selecting and implementing programs, developing policies, and evaluating progress . In the PHEP field, however, anecdotal responses to recent events are typically used as the best available science . A review of the emergency planning literature focused on the United Kingdom (UK) noted that the published literature for emergency planning is dominated by event reports . This can be problematic in that outcomes and conclusions may not be generalizable to other populations, settings, exposures or interventions [12-14]. Public health in general aims to be informed by the best available evidence and there is an appetite to have more rigorous forms of evidence to inform PHEP practice [12,16,17].
While the PHEP knowledge base is dominated by event reports and narrative reviews , PHEP research of a more analytical nature has been undertaken in some recent emergencies . This review focuses on primary and analytical research in PHEP in order to better inform research priorities for the field. Our objectives in undertaking this scoping review are therefore twofold: first, to develop a thematic map of the PHEP evidence base, focused on primary and analytical research; and second, to use this map to inform and guide an understanding of knowledge gaps relevant to research and practice in PHEP.
We conducted a scoping review of published research literature in the field of public health emergency preparedness. The scoping review methodology is particularly relevant to fields which have an emerging and diverse knowledge base, which makes the method well-suited to the PHEP literature [18-20]. Scoping reviews may be undertaken for four reasons: to examine the extent, range and nature of research activity; to determine the value of undertaking a full systematic review; to summarize and disseminate research findings; and to identify research gaps in the existing literature . This scoping review addresses three of the four reasons: to examine the extent, range and nature of research for PHEP; to summarize and disseminate research findings; and finally to identify existing gaps in the field that can be addressed through ongoing research. The six steps of the framework articulated by Arksey and O’Malley and further described by Levac et al. were applied in the review protocol [18-20]. As outlined by Levac et al., to maximize the rigor of the review, we implemented an iterative process throughout the six steps that incorporated knowledge from expert stakeholders.
Step 1: Identifying the research purpose and scope of inquiry
The first purpose of the review is to create a map of the major themes in the PHEP evidence base to inform an understanding of the extent, range and nature of PHEP research. The second purpose is to identify potential knowledge gaps relevant to research and practice in PHEP by summarizing research findings and identifying existing gaps in the field. ‘Public health’ was defined for this review as the scope of public health activities relevant to Canadian public health agencies and departments; this scope includes activities such as surveillance, epidemiology, food and water safety, public information and communication, and laboratory services but notably excludes the provision of mass health care or hospital and institution-based emergency preparedness that may be included in PHEP in other countries . Emergency preparedness research focused on all aspects of the emergency management cycle (prevention/mitigation, preparedness, response and recovery) was considered. An all-hazards approach was used including literature related to a range of hazards such as infectious disease, natural disasters, terrorism and technological events. The settings of complex humanitarian emergencies and conflict were out of scope. The focus was on peer-reviewed or grey literature and included studies of primary research, either quantitative or qualitative , and research examining standards or best practices (Table 1). Secondary research or knowledge syntheses such as literature reviews were excluded from the final group. Computer simulation studies were excluded for feasibility purposes and in order to focus on the scope of applied, real-world PHEP research.
Step 2: Identifying relevant studies
A library specialist-assisted search was employed that addressed the concepts of public health, emergencies or disasters, emergency preparedness or emergency management and evidence or evaluation. The databases searched were Medline, Embase, BIOSIS, PsycInfo and EBSCO (CINAHL, Academic Search Premier, Health Business Elite, Environment Complete and SocINDEX) from 1998 to 2013 to include important events in the emergency management field such as SARS, 9/11 and Hurricane Katrina internationally and the 1998 ice storm in Canada (Additional file 1). In addition, reference lists of included articles were reviewed. Non-indexed sources of evidence were examined (grey literature) and a custom Google search was created to examine key government, agency and non-governmental organization websites for an international sample of organizations practicing in the PHEP field.
Step 3: Study selection
Study selection was an iterative process that included screening first by title only, second by abstract, and finally, by full-text review. Initial search results were screened by title for their relevance to: (1) health, health consequences or health systems and (2) emergency preparedness or management actions. We conducted an initial pilot using 10% of studies reviewed by two reviewers to ensure consistency in applying the title screening criteria. After this pilot, the remaining initial search results were reviewed for relevance by title by one reviewer.
The second phase of study selection applied the inclusion criteria for relevance by abstract (Table 1). These criteria were developed in relation to the research purpose and refined based on consensus by members of the research team. The inclusion criteria were applied by two reviewers to the abstracts of title-screened search results and differences resolved by consensus.
Table 1 criteria were applied again to the screening of full text articles that were examined by two reviewers. A final group of selected articles was identified after full-text review; this group comprised the ‘primary research’ studies of interest. Themes identified from excluded simulation studies were cross-referenced with the themes from the final group of articles to ensure there were no notable elements or themes missed.
Step 4: Charting the data
The descriptive-numerical summary approach described by Levac et al. was used for charting the data . The data charting form was developed by consensus with the research team, piloted with two team members, and adapted iteratively in refining the final categories. The final table categories included were: research question/objective, category of study design (e.g., descriptive or analytical), specific study design, study population, intervention or exposure of interest, control group, study outcomes, study conclusions, hazard category, specific hazard, year of emergency event, scope of emergency event (e.g., local, provincial/state, federal or international support required), phase of emergency management cycle (e.g., prevention and/or mitigation, preparedness, response, recovery) , country of emergency event, country of study authors, and finally, additional notable elements or themes. If fields could not be completed or were not applicable for a given study, they were left blank. Assessment of quality of evidence beyond research design was out of scope for this review. The data charting form was completed by two reviewers and differences resolved by team consensus.
The category of study design allowed for the identification of a final group of articles that were primary research and evaluative or evidence-based in nature. This group was assessed using a thematic analysis approach which allowed for the capture of emergent themes in a more inductive method, rather than classifying the studies deductively based on previously described categories . Thematic analysis was initiated by two reviewers and iteratively developed with the research team. Specifically, each reviewer examined papers by full-text and noted key themes for the study. Themes identified were discussed with the larger research team and noted in the final data charting form based on consensus.
Steps 5 and 6: Collating, summarizing, reporting results and consultation
The consultation step is described by Levac et al. as essential and adding methodological rigor . The objectives of the consultation phase were to share preliminary findings with stakeholders as knowledge translation, validate the evidence map of themes, and identify additional sources of information, meaning and applicability [18,19]. Collating and summarizing results was therefore done in phases related to these objectives. In the first phase of data collation and summary, numerical or quantitative in nature were summarized using Microsoft Excel™ 2010. The key themes from the final group of primary research studies were used to create groupings of similar studies by theme and summarized in table format. These initial results and themes were sent to stakeholders to elicit their feedback through a survey.
For the consultation step process, we used a sample of stakeholders invited to participate in a working group meeting funded by a Canadian Institutes of Health Research (CIHR) Planning Grant awarded to conduct a scoping review and engage stakeholders in setting research priorities for PHEP. The sample was identified in a purposive manner to be international and interdisciplinary in nature and represent a diversity of perspectives from within PHEP as well as closely related fields responsible for working with the public health system in emergencies. Participants represented the fields of public health, health care, emergency medical services, government, and were distributed across research, academic and policy environments. The number of participants was 26. Consultation was carried out via electronic survey administered using FluidSurveys™. Ethics approval was obtained by the Public Health Ontario Ethics Review Board. For each theme, the survey participants were asked whether additional evaluative literature exists, and if they were aware of any gaps relating to practice, based on their knowledge and expertise. In addition, participants were asked to note any additional themes or topics not captured in the review (Additional file 2).
The second phase of data collation and summary involved analysis of the results from the survey. Dichotomous (yes/no) responses were summarized using Microsoft Excel™ 2010. Qualitative data from open-ended questions were coded and summarized based on themes and relevant sub-themes. The results of the survey for suggested additional studies were assessed for inclusion using the same criteria from Table 1 to determine whether they met consistent criteria for inclusion in the review; while all additional suggested themes were included in the final list of themes.
The last aspect of consultation occurred during an in-person working group meeting with the stakeholders. The themes resulting from the review and survey were presented to participants who were given the opportunity to offer final comment and confirm the themes and sub-themes.
Study selection and characteristics of the studies
The total number of citations retrieved through the search of indexed databases after duplicates removed was 3015. A flowchart of the article selection process is displayed in Figure 1. Three hundred articles met our criteria after title and abstract screening including additional sources from reference lists and grey literature. Reasons for exclusion of citations were predominantly related to emergency preparedness or management activities that did not meet the criteria of public health actions, based on the established definition. For example, hospital-based preparedness; mass casualty preparedness, triage or health care; and preparedness for institutions such as long-term care.
Data charting by descriptive numerical summary was undertaken for the 300 articles that were selected after abstract screening. The types of disaster were: all hazards or disaster type unspecified (39%); infectious natural hazards (31%); and non-infectious natural hazards or natural disasters (24%) (Figure 2).
The largest proportion of articles related to the response phase of the emergency management cycle (42%) and the second largest to the preparedness phase (34%). The prevention/mitigation phase accounted for 21%, while recovery was represented by only 3% of articles. The majority of studies were published by United States (US)-based authors (197, 66.3%), with Canada ranking second (23, 7.7%). The largest number of emergency events described in the published literature was from the US (126, 42%), with the second largest from Canada (18, 6.0%).
The majority of studies, 186 (62%), were non-primary research or descriptive articles. Thirty-seven (12%) were analytic epidemiology studies and 45 (15%) were descriptive primary research studies, such as surveys (Figure 3). Of the group of non-primary research or descriptive articles, 50% were emergency event-related descriptions such as “lessons learned”, event reports or case studies; 37% were literature reviews that did not employ systematic review methodology; 6% were commentaries; 3% were systematic reviews and another 3% were practice guidelines. A final 1% consisted of media or magazine reports that were captured in the search.
Eighty-two primary research and analytic epidemiology studies were selected for further review; these were reduced to 58 articles after exclusion of computer simulation studies. The final group of 58 articles related specifically to the research question. No grey literature met the criteria of being evidence-based or evaluative in nature. Within the final group, the most common study design was survey methodology (60%) including cross-sectional population surveys, surveys using qualitative methods such as key informant interviews, and surveys conducted after training exercises as an evaluation tool. The cohort studies were largely retrospective administrative database analyses. Studies with an evaluation purpose were largely evaluations of a process change or of training.
Identification of themes
The first iteration of thematic analysis resulted in the identification of ten major themes (Table 2). The key findings for the initial themes are described below, with additional detail provided in Additional file 3.
Attitudes and beliefs [23-25]: A key sub-theme in the literature relating to attitudes and beliefs is the willingness of workers to respond to a public health emergency. In identified studies, willingness to respond to particular emergencies varies by type of disaster and practice setting [23,25]. In addition, studies concluded that willingness as an attitude is important to consider in emergency response training [23,24].
Capacity assessment and capacity-building [26-31]: A study in this theme reports that surveys at the local health department level can provide information to guide development of capacity . Evaluative information that relates to emergency preparedness and response can inform a deeper understanding of the capabilities important to preparedness . Another study notes the importance of funding on the preparedness capacity of local public health .
Collaboration and system integration [32-35]: Several studies highlight the importance of collaboration and integration in relation to public health emergencies. Collaborating on exchanging plans and protocols was identified as an important and feasible goal for agencies . Roles exist in the health system that can facilitate linkages between health care and public health settings; for example, epidemiologists that link with hospitals . Newer methodologies, like network analysis, present opportunities for examining the role of inter-agency networks in promoting prepared public health systems .
Communicable disease control [36,37]: This theme is a well-understood core function of public health. Studies on this theme found relating to PHEP research indicate the utility of specific emergency plans to inform public health strategies such as vaccination and examining the role of interventions such as school closure as mitigating measures in responding to a public health incident [36,37].
Communication [38-45]: The third highest number of studies was identified in this theme. Sub-themes noted for this theme relate to the focus of communication as externally-facing (to the public), internally-facing (within the system), and high-risk populations. Externally-facing communication examined aspects of the adoption of public health guidance by the public. For example, concern or risk perception may not translate automatically to higher uptake of guidance and skepticism regarding institutions may influence public uptake of recommended actions [39,45]. Internally-facing communication that occurs within the system included findings that stakeholders have preferences on how information is received and note the importance of trust in the agency delivering the message [40,43]. Principles of trust are also important when communicating with high-risk populations .
Education, training and exercises [47-55]: This theme contained the second highest number of studies. Sub-themes were developed that related to the audience of the education or training intervention such as public health practitioner, clinician and leader. Public health practitioner studies found that blended learning approaches had good outcomes for both satisfaction and knowledge . Studies also found that criteria or tools can be used to facilitate evaluation of training and exercises [51,53]. Leadership-specific training was reported to result in increased overall knowledge and positive change .
Public health considerations for sheltering and evacuation [56,57]: Studies identified in this theme describe the importance of the planning process to address the health needs of sheltered populations . In addition, public health training in shelter operations was noted as a gap .
Quality improvement and performance standards [58-62]: This theme included studies that explore models for improving public health preparedness as well as frameworks and tools that can be used to measure preparedness and promote quality improvement [61,62]. Processes for team debriefing were also found to be helpful to evaluate performance .
Surveillance, epidemiology and public health information [63-75]: This theme was associated with the largest proportion of studies (13/58, 22%). Within this theme, methodologies applied in epidemiological data collection and analysis were identified. Specific sub-themes included studies on rapid needs assessments, which is a method that can provide valuable information to authorities on the needs and health status of affected communities [65,70,75]. Risk assessments using surveys can provide information about exposure risk to populations . In addition, other studies described the use of a diversity and adaptability of data sources that can be applied to public health surveillance; for example, medical dispatch data and mobile phone data [63,73].
Vulnerable populations [76-80]: Sub-themes identified for this theme included methods for vulnerability assessment, such as heat vulnerability index, and high-risk populations in general. For high-risk populations, the challenges of identifying and prioritizing vulnerable populations were noted and strategies for integrating diverse communities into emergency planning and response were explored [76,77].
A list of the initial themes along with summarized key findings from the relevant articles was disseminated to stakeholders. Of 26 participants invited to participate in the survey, the response rate was 62%.
Four additional themes were noted by stakeholders: ethics, gender considerations, psychosocial considerations, resilience and planning. Ethics, gender, resilience and psychosocial considerations had been noted in the evidence reviewed but had not been captured as themes. Planning was a theme that was identified as important based on experience, rather than literature. In addition, wording refinements for themes were suggested based on participants’ expertise. For example, “high risk” or “at-risk” populations was suggested instead of “vulnerable populations”. Although the majority of the studies were conducted in the US, 88% of survey respondents felt the themes were applicable to PHEP in Canada.
There were 29 total additional citations that were suggested by stakeholders as potential evidence for inclusion in the review. It was determined by the research team that 2 additional articles would have met the inclusion criteria applied in the review [81,82]. Reasons for exclusion included: articles that were not related to public health actions (E.g. hospital or emergency responder actions), not representing primary research, or outside the date criteria. The additional articles and survey results were used to revise the original PHEP themes. A number of themes were cross-cutting in nature, while others stood on their own. The synthesized list of themes included 4 cross-cutting themes and 12 stand-alone themes (Table 3).
In the final stage of stakeholder consultation, the synthesized and revised results consisting of cross-cutting and stand-alone themes were presented to the survey participants in a face-to-face format. This step resulted in the confirmation of the themes as presented in Table 3 without further suggested revisions.
This review aimed to identify knowledge gaps for PHEP relevant to research and practice in the field. Primary research from non-US settings was identified as a general gap, consistent with previous reviews . The results of the survey questions pertaining to participants’ sense of knowledge gaps, for the initial ten themes, is described below with additional detail provided in Additional file 4.
Attitudes and beliefs: Gaps pertained to the ethical values involved in assessing willingness to respond; the importance of psychosocial supports to foster willingness to work; and volunteer characteristics.
Capacity assessment and capacity-building: Gaps for this initial theme related to building capacity to address a need for psychosocial supports for workers and building capacity for hospital surge. General capacity-building also noted the importance of poverty reduction in mitigating the impact of emergencies on populations.
Collaboration and system integration: Gaps pertaining to this theme related to cross-jurisdictional collaboration as well as across sectors of the health system.
Communicable disease control: A gap that was noted relating to SARS was the labour issues involved for front line health care workers in the setting of communicable disease outbreaks.
Communication: Communication as a strategy to promote integration with other sectors such as primary care was noted as a gap. In addition, emerging information technologies such as social media have had limited exploration to date.
Education, training and exercises: A number of gaps were identified in the survey pertaining to this initial theme. These ranged from specific capabilities such as Incident Management System implementation to more complex concepts such as flexibility and emergency decision-making for public health practitioners.
Public health considerations for sheltering and evacuation: Gaps in this initial theme related to understanding the logistics and implementation challenges for public health measures such as evacuation. In addition, the potential for public health to benefit from skillsets found in other disciplines, which could promote improved public preparedness levels.
Quality improvement and performance standards: Within this theme, the challenge of understanding the components of preparedness was noted. Psychosocial supports was again noted as a gap in this theme.
Surveillance, epidemiology and public health information: Elements of integration to promote public health information were noted, such as the interfaces of public health with primary and emergency care. The value of technologies such as electronic health records was described as a gap in relation to this theme. The concept of resilience was also raised as a gap.
Vulnerable populations: A number of gaps pertaining to vulnerable populations were noted. The ethical values and considerations inherent in exploring issues of “vulnerable” populations were described. In addition, methods for promoting community engagement for public health preparedness were noted and for addressing the needs of specific populations and potential vulnerabilities.
The following specific quotes relate to qualitative data collected on knowledge gaps, with relevant associated theme:
“What are the main methods and coping strategies used by individuals, stakeholders and their organizations to get back to normal life?” (resilience);
“How do we answer the frequent question: “Are we prepared?” (quality improvement and performance standards);
“There is more focus on recovery needed in the literature, whether physical or psychosocial impacts” (resilience, psychosocial impacts, capacity assessment and capacity building);
“There is a need for collaboration and networking across disciplines to scope beyond PHEP/public health” (collaboration and system integration).
Based on thematic analysis and synthesis of the initial themes with survey results, we identified five key knowledge gaps that were important areas of interest to stakeholders but where evidence is sparse or non-existing. These are depicted along with the cross-cutting themes identified from the consultation (Table 4).
Previous reviews of PHEP evidence base documented the largely descriptive and anecdotal nature of the literature and found it challenging to draw conclusions as a result [7,11]. Rigorous and analytical research needs to be conducted to strengthen the PHEP evidence base to respond to practice and policy-related questions for the field [9,10]. This review focused on the body of primary research that has been conducted in the PHEP field and aimed to map important themes, for the purpose of furthering understandings of existing PHEP science and revealing potential knowledge gaps for future research. This scoping review is novel for the PHEP field in that it utilized published scoping review methodology with the identification of themes as they emerged, without applying a pre-existing framework . In addition, the study included expert stakeholder consultation to validate and confirm the themes identified by the review, a knowledge translation step that is sometimes omitted in scoping reviews but considered essential .
The iterative identification of themes using the literature and survey methods resulted in the recognition of important cross-cutting themes for PHEP. To our knowledge, this is the first time that the cross-cutting nature of some PHEP themes has been documented. Based on these results, consideration of the themes of at-risk populations, gender and equity should be recognized as integral to planning and implementing PHEP research . Although these themes may be viewed as similar they each have unique characteristics: at-risk populations vary depending on the hazard and risk, gender differences cut across virtually all population health issues, and equity speaks more broadly about disparities that may pre-exist within a given community. Resilience was the fourth cross-cutting theme . Resilience is described as the intrinsic capacity of individuals, populations and infrastructure to resist and rebound from shocks and to then rebuild to a stronger state . In national-level emergency management policy, resilience is established as the overall objective of all-hazards preparedness and capacity-building activities in order to reduce the impact of emergencies on a given population [22,85]; hence its inclusion in this category.
In addition to cross-cutting themes, the stand-alone themes with the largest number of papers included in the review were Surveillance, Epidemiology and Public Health Information, Communication and Education, Training and Exercises. Surveillance, Epidemiology and Public Health Information was the theme with the largest number of studies. This is not surprising given the integral nature of surveillance and epidemiology to public health, as well as the manner in which quantitative data are used for analytic processes to assess risk or measure outcomes of public health emergency events. Efforts in the US to identify key domains for PHEP are consistent and have similarly identified ‘health surveillance’ . Communication is an area that has been noted to be problematic in post-emergency event assessments; thus, it is positive that research is emerging under this theme [6,87]. The social behavioural facets of communication relating to PHEP have recently been explored in the literature. For example, concepts of trust and preferences for certain characteristics of messaging highlight the contextual aspects that are emerging from PHEP communication research [40,43,45]. Important knowledge gaps were identified for communication in PHEP relevant to the use of emerging technologies. Education, Training and Exercises was another theme in which preliminary study on approaches to training for ‘hard skills’ such as technical skills and ‘soft skills’ such as leadership has been undertaken [53,54]. Nonetheless, attention to training as it relates to emergency decision-making in public health was highlighted as a gap in this review.
The complexity of PHEP is reflected in the knowledge gaps that emerged. For example, Collaboration and System Integration sub-themes pertain to multiple health-care stakeholders. Multi-sectoral partnerships and engagement has also been noted as important domains for PHEP operations in the US . Opportunities for generating knowledge to examine complexities such as system integration could address knowledge gaps and inform operational activities and policy. Another notable knowledge gap relates to the theme of Quality Improvement and Performance Standards. Multiple publications have noted the importance of quality improvement and development of standards or metrics for PHEP [5,9,88]. Despite this, little research was identified that has empirically examined the development of standards and metrics. Research in PHEP that advances empirical knowledge in this area would be valuable.
This review has several limitations. First, this study specifically focused on research that was relevant to the actions and activities of public health as defined for the Canadian setting. The findings therefore may not translate to other countries that have differing scope of public health responsibilities. As was noted in the emergent themes, the importance of integration and collaboration within the health system is key for PHEP, and it is possible that this review did not capture the extent of public health connectedness to the rest of the health system, as found in the literature. This focus on public health was carefully defined, however, with the purpose of identifying themes and relevant knowledge gaps that specifically pertain to public health agencies and departments. Second, as with all systematic reviews, there is a possibility that literature was missed, which may have resulted in a lack of recognition of key themes or gaps. We used a rigorous search strategy for the indexed and grey literature to minimize this. In addition, we conducted stakeholder consultation in order that experts in the field would provide information and contribute knowledge that was missing. As a result of this iterative approach, additional articles were included in the final thematic analysis. Third, computer simulation studies were excluded due to scope of the current study, with a focus on the field-derived primary research studies. An attempt to account for themes that resulted from data-charting of this group was done and compared with the final emergent themes. In general, these studies pertained to vaccination and outbreak management, which were captured under themes of Communicable Disease Control or Surveillance, Epidemiology and Public Health Information. Finally, quality of research studies was not assessed beyond research design in this review and for this reason, the themes identified in the literature and the knowledge gaps do not take into account specific methodological strengths and weaknesses of the included studies. While quality assessment is ideal to include, we addressed the varying quality in the PHEP literature by focusing on identifying a group of articles that did not include event reports or ‘lessons learned’ commentaries, and focused on primary research studies.
A question that has been posed in a review of the emergency planning literature focused on the UK is whether “the traditional hierarchy of evidence applied to clinical research is appropriate in this field” . The challenge of using the traditional evidence hierarchy to inform evidence-based decision-making in health has been raised outside the PHEP field . The hierarchical approach to evidence is quantitatively oriented and does not elaborate on how context may modify the relevance of evidence . Upshur et al. suggest that an inclusive definition of evidence includes both quantitative research that involves measurement as well as qualitative research that examines context and meaning . This review included the breadth of evidence described in this taxonomy, and identified qualitative research in PHEP that attempts to capture meaning. The themes identified in this review for PHEP would support that both quantitative and qualitative methodological approaches will be valuable in generating knowledge to understand the complexities of gaps such as communication, collaboration and integration, and attitudes and beliefs. However, as noted by Upshur et al., such research will benefit from having a sufficient theoretical base that will allow it to have solid ‘scientific’ standing in the eyes of proponents of evidence-based practice. This is important given the high proportion of anecdotal event reports which share experiential and contextual knowledge but generally lack a sufficient theoretical base. In addition, there are recent studies highlighting that participatory and community-based approaches to research may have an important role in addressing some of the cross-cutting and stand-alone themes noted in this review [89,90]. Participatory methods hold promise for promoting inclusiveness across a community and examining the meaning that may underlie the complexities of the emergency . These methods also model a collaborative approach that may promote resilience, enabling both research and community-building, thereby fostering the translation of knowledge to action [89,91].
Global health threats, which include emerging infectious diseases, technological events, and natural disasters, appear to be increasing in frequency and impact. This scoping review identifies the key cross-cutting and stand-alone themes that characterize the current state of the evidence base of primary research in PHEP. The identification of themes and related knowledge gaps represents a leaping-off point from which to identify research priorities and inform the further generation of practice-oriented research to advance PHEP. Based on the diversity of themes and knowledge gaps identified, collaboration in inter-disciplinary teams such as was utilized in the consultation step of this review and an inclusive definition of evidence, will be valuable in developing research that is both rigorous and relevant to the field.
Canadian Institutes of Health Research
Public health emergency preparedness
Severe acute respiratory syndrome
Marquis M. Lac-megantic pollution downplayed by Quebec government. The Canadian Press; 2013. http://www.ctvnews.ca/health/health-headlines/quebec-gov-t-downplays-concerns-about-lac-megantic-pollution-1.1409486.
Fouchier R, Zaki AM, van Boheemen S, Bestebroer T, Osterhaus A. Isolation of a novel coronavirus from a man with pneumonia in Saudi Arabia. N Engl J Med. 2012;367(19):1814–20.
Alberta mobilizes to fight public health threats in flooding aftermath. [http://crofsblogs.typepad.com/h5n1/2013/06/alberta-mobilizes-to-fight-public-health-threats-in-flooding-aftermath.html]
Ebola Response Team WHO. Ebola virus disease in West Africa — the first 9 months of the epidemic and forward projections. N Engl J Med. 2014;371(16):1481–95.
Nelson C, Lurie N, Wasserman J, Zakowski S. Conceptualizing and defining public health emergency preparedness. Am J Public Health. 2007;97(S1):S9–11.
Naylor D, Basrur S, Bergeron MG, Brunham RC, Butler-Jones D, Dafoe G, et al. Learning from SARS: Renewal of Public Health in Canada; 2003. p. 1–224.
Savoia E, Massin-Short SB, Rodday AM, Aaron LA, Higdon MA, Stoto MA. Public health systems research in emergency preparedness: a review of the literature. Am J Prev Med. 2009;37(2):150–6.
Nelson C, Lurie N, Wasserman J. Assessing public health emergency preparedness: concepts, tools, and challenges. Annu Rev Public Health. 2007;28:1–18.
Smith K, Jarris P, Inglesby T, Hatchett R, Kellermann A. Public health preparedness research. J Public Health Manag Pract. 2013;19(2):S6–8.
Lurie N, Manolio T, Patterson AP, Collins F, Frieden T. Research as a part of public health emergency response. N Engl J Med. 2013;368(13):1251–4.
Challen K, Lee A, Booth A, Gardois P, Woods H, Goodacre S. Where is the evidence for emergency planning: a scoping review. BMC Public Health. 2012;12:542–50.
Brownson RC, Fielding JE, Maylahn CM. Evidence-based public health: a fundamental concept for public health practice. Annu Rev Public Health. 2009;30:175–201.
Evidence-Based Medicine Working Group. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA. 1992;268(17):2420–5.
Levin A. The Cochrane collaboration. Ann Intern Med. 2001;135(4):309–12.
McCormack L, Sheridan S, Lewis M, Boudewyns V, Melvin CL, Kistler C, et al. Communication and dissemination strategies to facilitate the use of health-related evidence. Evid Rep Technol Assess. 2013;213:1–520.
Greenhalgh T, Howick J, Maskrey N, Evidence Based Medicine Renaissance Group. Evidence based medicine: a movement in crisis? BMJ. 2014;348:g3725.
Lee A, Phillips W, Challen K, Goodacre S. Emergency management in health: key issues and challenges in the UK. BMC Public Health. 2012;12:884.
Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19.
Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5:69–77.
Armstrong R, Hall BJ, Doyle J, Waters E. Cochrane update ‘scoping the scope’ of a cochrane review. J Public Health. 2011;33(1):147–50.
Upshur REG, VanDenKerkhof EG, Goel V. Meaning and measurement: an inclusive model of evidence in health care. J Eval Clin Pract. 2001;7(2):91.
Emergency Management Policy Directorate. Public safety Canada: an emergency management framework for Canada. 2nd ed. 2011.
Barnett DJ, Balicer RD, Thompson CB, Storey JD, Omer SB, Semon NL, et al. Assessment of local public health workers’ willingness to respond to pandemic influenza through application of the extended parallel process model. PLoS One. 2009;4(7), e6365.
Barnett DJ, Thompson CB, Errett NA, Semon NL, Anderson MK, Ferrell JL, et al. Determinants of emergency response willingness in the local public health workforce by jurisdictional and scenario patterns: a cross-sectional survey. BMC Public Health. 2012;12:164.
DeSimone CL. Response of public health workers to various emergencies. AAOHN J. 2009;57(1):17–23.
Borell J, Eriksson K. Improving emergency response capability: an approach for strengthening learning from emergency response evaluations. Int J Emerg Manag. 2008;5(3-4):ate of Pubaton: 2008.
Davis MV, MacDonald PD, Cline JS, Baker EL. Evaluation of public health response to hurricanes finds North Carolina better prepared for public health emergencies. Public Health Rep. 2007;122(1):17–26.
Day T, Challen K, Walter D. Major incident planning in primary care trusts in northwest England: a cross-sectional survey. Health Serv Manage Res. 2010;23:25–9.
Hyde J, Kim B, Martinez LS, Clark M, Hacker K. Better prepared but spread too thin: the impact of emergency preparedness funding on local public health. Disaster Manag Response. 2006;4(4):106–13.
Isaac-Renton JL, Chang Y, Prystajecky N, Petric M, Mak A, Abbott B, et al. Use of Lean response to improve pandemic influenza surge in public health laboratories. Emerg Infect Dis. 2012;18(1):57–62.
Porten K, Faensen D, Krause G. SARS outbreak in Germany 2003: workload of local health departments and their compliance in quarantine measures–implications for outbreak modeling and surge capacity? J Public Health Manag Pract. 2006;12(3):242–7.
Hunter JC, Crawley AW, Petrie M, Yang JE, Aragon TJ. Local Public Health System Response to the Tsunami Threat in Coastal California following the Tohoku Earthquake; 2012.
Moore D, Shiell A, Noseworthy T, Russell M, Predy G. Public health preparedness in Alberta: a systems-level study. BMC Public Health. 2006;6:313.
Olson DR, Paladini M, Lober WB, Buckeridge DL: Applying a new model for sharing population health data to national syndromic influenza surveillance: DiSTRIBuTE project proof of concept, 2006 to 2009. PLoS Curr. 2011, (September 12, 2011).
Markiewicz M, Bevc CA, Hegle J, Horney JA, Davies M, MacDonald PD. Linking public health agencies and hospitals for improved emergency preparedness: North Carolina's public health epidemiologist program. BMC Public Health. 2012;12:141.
DiBiase LM, Davis SE, Rosselli R, Horney J. Evaluation of the implementation of the H1N1 pandemic influenza vaccine in local health departments (LHDs) in North Carolina. Vaccine. 2011;29(23):3969–76.
Chowell G, Echevarria-Zuno S, Viboud C, Simonsen L, Tamerius J, Miller MA, et al. Characterizing the epidemiology of the 2009 influenza A/H1N1 pandemic in Mexico. PLoS Med. 2011;8(5), e1000436.
Aburto NJ, Pevzner E, Lopez-Ridaura R, Rojas R, Lopez-Gatell H, Lazcano E, et al. Knowledge and adoption of community mitigation efforts in Mexico during the 2009 H1N1 pandemic. Am J Prev Med. 2010;39(5):395–402.
Balkhy HH, Abolfotouh MA, Al-Hathlool R, Al-Jumah M. Awareness, attitudes, and practices related to the swine influenza pandemic among the Saudi public. BMC Infect Dis. 2010;10(1):42–9.
Blanchard JC, Haywood Y, Bradley S, Tanielian TL, Stoto M, Lurie N. In their own words: lessons learned from those exposed to anthrax. Am J Public Health. 2005;95(3):489.
Harris JK, Clements B. Using social network analysis to understand Missouri's system of public health emergency planners. Public Health Rep. 2007;122(4):488–98.
Hiam R, Guest C, Isaac-Toua G. An evaluation of public health interventions during a pertussis outbreak in the ACT, 2003. Aust NZ J Public Health. 2005;29(6):552–4.
Staes CJ, Wuthrich A, Gesteland P, Allison MA, Leecaster M, Shakib JH, et al. Public health communication with frontline clinicians during the first wave of the 2009 influenza pandemic. J Public Health Manag Pract. 2011;17(1):36.
Taylor MM, Stokes WS, Bajuscak R, Serdula M, Siegel KL, Griffin B, et al. Mobilizing mobile medical units for hurricane relief: the United States Public Health Service and Broward County Health Department response to hurricane Wilma, Broward County, Florida. J Public Health Manag Pract. 2007;13(5):447–52.
Teasdale E, Yardley L. Understanding responses to government health recommendations: public perceptions of government advice for managing the H1N1 (swine flu) influenza pandemic. Patient Educ Couns. 2011;85(3):413–8.
TaylorClark KA, Viswanath K, Blendon RJ. Communication inequalities during public health disasters: Katrina's wake. Health Commun. 2010;25(3):221–9.
Alfano-Sobsey E, Kennedy B, Beck F, Combs B, Kady W, Ramsey S, et al. Evaluation of a model training program for respiratory-protection preparedness at local health departments. J Environ Health. 2006;68(8):38–42.
Kerby DS, Brand MW, Johnson DL, Ghouri FS. Self-assessment in the measurement of public health workforce preparedness for bioterrorism or other public health disasters. Public Health Rep. 2005;120(2):186–91.
Qureshi KA, Gershon RR, Merrill JA, Calero-Breckheimer A, Murrman M, Gebbie KM, et al. Effectiveness of an emergency preparedness training program for public health nurses in New York City. Fam Community Health. 2004;27(3):242–9.
Savoia E, Biddinger PD, Fox P, Levin DE, Stone L, Stoto MA. Impact of tabletop exercises on participants' knowledge of and confidence in legal authorities for infectious disease emergencies. Disaster Med Public Health Prep. 2009;3(2):104–10.
Savoia E, Testa MA, Biddinger PD, Cadigan RO, Koh H, Campbell P, et al. Assessing public health capabilities during emergency preparedness tabletop exercises: reliability and validity of a measurement tool. Public Health Rep. 2009;124(1):138–48.
Chandler T, Qureshi K, Gebbie KM, Morse SS. Teaching emergency preparedness to public health workers: use of blended learning in web-based training. Public Health Rep. 2008;123(5):676–80.
Gebbie KM, Valas J, Merrill J, Morse S. Role of exercises and drills in the evaluation of public health in emergency response. Prehosp Disaster Med. 2006;21(3):173–82.
Wang C, Wei S, Xiang H, Xu Y, Han S, Mkangara OB, et al. Evaluating the effectiveness of an emergency preparedness training programme for public health staff in China. Public Health. 2008;122(5):471–7.
Wang C, Wei S, Xiang H, Wu J, Xu Y, Liu L, et al. Development and evaluation of a leadership training program for public health emergency response: results from a Chinese study. BMC Public Health. 2008;8:377.
Brahmbhatt D, Chan J, Hsu E, Mowafi H, Kirsch T, Quereshi A, et al. Public health preparedness of post-Katrina and Rita shelter health staff. Prehosp Disaster Med. 2010;24(6):500–7.
Broz D, Levin E, Mucha A, Pelzel D, Wong W, Persky V, et al. Lessons learned from Chicago’s emergency response to mass evacuations caused by hurricane Katrina. Am J Public Health. 2009;99(8):1496–504.
Flabouris A, Nocera A, Garner A. Efficacy of critical incident monitoring for evaluating disaster medical readiness and response during the 2000 Sydney Olympic Games. Prehosp Disaster Med. 2004;19(2):164–8.
Lotstein D, Seid M, Ricci K, Leuschner K, Margolis P, Lurie N. PREPARE for Pandemic I: Using quality improvement methods to improve public health emergency preparedness: PREPARE for Pandemic Influenza. Health Aff (Millwood). 2008;27(5):w328–39.
Nelson C, Chan E, Chandra A, Sorensen P, Willis HH, Dulin S, et al. Developing national standards for public health emergency preparedness with a limited evidence base. Disaster Med Public Health Prep. 2010;4(4):285–90.
Parker CL, Barnett DJ, Fews AL, Blodgett D, Links JM. The Road Map to Preparedness: a competency-based approach to all-hazards emergency readiness training for the public health workforce. Public Health Rep. 2005;120(5):504–14.
Sarpy SA, Chauvin SW, Anderson AC. Evaluation of the effectiveness of the South Central Center for Public Health Preparedness training. Public Health Rep. 2003;118(6):568–72.
Bassil KL, Cole DC, Moineddin R, Gournis E, Schwartz B, Craig AM, et al. Development of a surveillance case definition for heat-related illness using 911 medical dispatch data. Can J Public Health. 2008;99(4):339–43.
Betancourt JA, Hakre S, Polyak CS, Pavlin JA. Evaluation of ICD-9 codes for syndromic surveillance in the electronic surveillance system for the early notification of community-based epidemics. Mil Med. 2007;172(4):346–52.
Chen KT, Chen WJ, Malilay J, Two SJ. The public health response to the Chi-Chi earthquake in Taiwan. Public Health Rep. 2003;118(6):493–9.
Fan C, van Dijk A, Fernando D, Hall JN, Wynn A, Gemmill I, et al. Automated mortality surveillance in south-eastern Ontario for pandemic influenza preparedness. Can J Public Health. 2010;101(6):459–63.
Ghosh TS, Vogt RL. Active influenza surveillance at the local level: a model for local health agencies. Am J Public Health. 2008;98(2):213–5.
Groenewold MR. Enhancing local health department disaster response capacity with rapid community needs assessments: validation of a computerized program for binary attribute cluster sampling. Prehosp Disaster Med. 2006;21(1):32–9.
Maguire H, Fraser G, Croft J, Bailey M, Tattersall P, Morrey M, et al. Assessing public health risk in the London polonium-210 incident, 2006. Public Health. 2010;124(6):313–8.
McNeil M, Goddard J, Henderson A, Phelan M, Davis S, Wolkin A, et al. Rapid community needs assessment after Hurricane Katrina – Hancock County, Mississippi, September 14-15, 2005. MMWR. 2006;55(9):234–6.
Schnall AH, Wolkin AF, Noe R, Hausman LB, Wiersma P, Soetebier K, et al. Evaluation of a standardized morbidity surveillance form for use during disasters caused by natural hazards. Prehosp Disaster Med. 2011;26(2):90–8.
Schnitzler J, Benzler J, Altmann D, Mucke I, Krause G. Survey on the population’s needs and the public health response during floods in Germany 2002. J Public Health Manag Pract. 2007;13(5):461–4.
Yang C, Yang J, Luo X, Gong P. Use of mobile phones in an emergency reporting system for infectious disease surveillance after the Sichuan earthquake in China. Bull World Health Organ. 2009;87(8):619–23.
Yzermans CJ, Donker GE, Kerssens JJ, Dirkzwager AJE, Soeteman RJH, tenVeen PMH. Health problems of victims before and after disaster: a longitudinal study in general practice. Int J Epidemiol. 2005;34(4):820–6.
Zane DF, Bayleyegn TM, Haywood TL, Wiltz-Beckham D, Guidry HM, Sanchez C, et al. Community assessment for public health emergency response following hurricane Ike – Texas, 25-30 September 2008. Prehosp Disaster Med. 2010;25(6):503–10.
Andrulis DP, Siddiqui NJ, Purtle JP. Integrating racially and ethnically diverse communities into planning for disasters: the California experience. Disaster Med Public Health Prep. 2011;5(3):227–34.
Abrahamson V, Raine R. Health and social care responses to the Department of Health Heatwave Plan. J Public Health. 2009;31(4):478–89.
Reid CE, O’Neill MS, Gronlund CJ, Brines SJ, Brown DG, Diez-Roux AV, et al. Mapping community determinants of heat vulnerability. Environ Health Perspect. 2009;117:1730–6.
Reid CE, Mann JK, Alfasso R, English PB, King GC, Lincoln RA, et al. Evaluation of a heat vulnerability index on abnormally hot days: an environmental public health tracking study. Environ Health Perspect. 2012;120(5):715–20.
Stein BD, Tanielian TL, Eisenman DP, Keyser DJ, Burnam MA, Pincus HA. Emotional and behavioral consequences of bioterrorism: planning a public health response. Milbank Q. 2004;82(3):413–55.
Krewski D, Lemyre L, Turner MC, Lee JEC, Dallaire C, Bouchard L, et al. Public Perception of Population Health Risks in Canada: Health Hazards and Sources of Information. Human Ecol Risk Asses. 2006;12:626.
Garoon J, Duggan P. Discourses of disease, discourses of disadvantage: a critical analysis of National Pandemic Influenza Preparedness Plans. Soc Sci Med. 2008;67(7):1133.
Public Health Agency of Canada. What Makes Canadians Healthy or Unhealthy? Ottawa: 2013.http://www.phac-aspc.gc.ca/ph-sp/determinants/determinants-eng.php November 21, 2014.
Castleden M, McKee M, Murray V, Leonardi G. Resilience thinking in health protection. J Public Health. 2011;33(3):369–77.
Presidential Policy Directive / PPD-8: National Preparedness. [http://www.dhs.gov/xlibrary/assets/presidential-policy-directive-8-national-preparedness.pdf]
National Health Security Preparedness Index. [http://www.nhspi.org/]
Walker D, Expert Panel on SARS and Infectious Disease Control. For the public's health: a plan of action. Final report of the Ontario Expert Panel on SARS and Infectious Disease Control; 2004.
Seid M, Lotstein D, Williams VL, Nelson C, Leuschner K, Diamant A, et al. Quality improvement in public health emergency preparedness. Annu Rev Public Health. 2007;28:19.
O’Sullivan T, Corneil W, Kuziemsky C, Toal-Sullivan D. Use of the structured interview matrix to enhance community resilience through collaboration and inclusive engagement. Systems Research and Behavioral Science; 2014.
Chandra A, Williams M, Plough A, Stayton A, Wells K, Horta M, et al. Getting actionable about community resilience: the Los Angeles county community disaster resilience project. Am J Public Health. 2013;103(7):1181–9.
O’Sullivan T, Kuziemsky C, Toal-Sullivan D, Corneil W. Unraveling the complexities of disaster management: a framework for critical social infrastructure to promote population health and resilience. Soc Sci Med. 2013;93:238–46.
The authors wish to acknowledge CIHR (Funding reference number 122651) which provided funding to support in part the work of this project. The authors also wish to acknowledge the assistance of Library Services at Public Health Ontario and the contributions of Sandy Bogucki, David Etkin, John Lindsay, Laurie Mazurik, James Shultz and Charles Stewart to the study development and implementation.
The authors declare that they have no competing interests.
YK participated in study conception and design, participated in data collection and analysis and drafted the manuscript. GF and CT contributed to study design, analysis and manuscript development. BH, EV and MG participated in study design and analysis. BS conceived of the study, and participated in its design and coordination. All authors critically reviewed the manuscript for important intellectual content and read and approved the final manuscript.
Yasmin Khan, Ghazal Fazli and Brian Schwartz contributed equally to this work.
Additional file 1:
Example of Search Strategy as Applied in Medline.
Additional file 2:
Example of Survey Questions.
Additional file 3:
Summary table of key findings from studies selected by initial theme.
Additional file 4:
Summarized survey results for applied knowledge gaps by initial theme.
Rights and permissions
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Khan, Y., Fazli, G., Henry, B. et al. The evidence base of primary research in public health emergency preparedness: a scoping review and stakeholder consultation. BMC Public Health 15, 432 (2015). https://doi.org/10.1186/s12889-015-1750-1
- Severe Acute Respiratory Syndrome
- Severe Acute Respiratory Syndrome
- Emergency Preparedness
- Stakeholder Consultation
- Middle East Respiratory Syndrome