Towards defining core principles of public health emergency preparedness: scoping review and Delphi consultation among European Union country experts

Abstract

Background

European Member States, the European Commission and its agencies work together to enhance preparedness and response for serious cross-border threats to health such as Ebola. Yet, common understanding of public health emergency preparedness across EU/EEA countries is challenging, because preparedness is a relatively new field of activity and is inherently fraught with uncertainty. A set of practical, widely accepted and easy to use recommendations for generic preparedness that bundles the activities described in separate guidance documents supports countries in preparing for any possible health threat. The aim of this consensus procedure was to identify and seek consensus from national-level preparedness experts from EU/EEA countries on key recommendations of public health emergency preparedness.

Methods

To identify key recommendations and to prioritize the recommendations we started with a literature consensus procedure, followed by a modified Delphi method for consultation of public health emergency preparedness leaders of EU/EEA countries. This consisted of six consecutive steps: a questionnaire to achieve consensus on a core set of recommendations, a face-to-face consultation, preselection of prioritized recommendations, a questionnaire to achieve consensus on the prioritized set and a face-to-face consensus meeting to further prioritize recommendations.

Results

As a result, EU/EEA experts selected 149 recommendations as core preparedness principles and prioritized 42. The recommendations were grouped in the seven domains: governance (57), capacity building and maintenance (11), surveillance (19), risk-assessment (16), risk- and crisis management (35), post-event evaluation (6) and implementation of lessons learned (5).

Conclusions

This prioritised set of consensus principles can provide a foundation for countries aiming to evaluate and improve their preparedness for public health emergencies. The recommendations are practical, support generic preparedness planning, and can be used by all countries irrespective of their current level of preparedness.

Background

Cross-border outbreaks demonstrate that, in an interconnected world, all countries are potentially vulnerable [1]. The likelihood of outbreaks is exacerbated by factors such as urbanization, the rapid growth and mobility of the world population, and the speed of travel [2, 3]. COVID-19, Ebola, as well as other recent threats and outbreaks, highlighted the need for countries to better prepare for threats and outbreaks and the need for a coherent view of preparedness for international purposes. The COVID-19 pandemic shows that countries are not prepared enough for this type of situations and better preparedness is needed.

In Europe, Member States and the European Commission work together with the aim of coordinating their efforts in enhancing preparedness and response for serious cross-border threats to health [4]. Yet, common understanding of public health emergency preparedness across EU/EEA countries is challenging, because preparedness is a relatively new field of activity and is inherently subject to uncertainty [5]. In general, the evidence level supporting preparedness actions/recommendations is low and most of them are produced by consensus, case studies, or outbreak descriptions [6]. Several knowledge gaps have been identified for the preparedness evidence base [7], such as the lack of instruments for improving public health preparedness as well as instruments that can be used to measure and promote preparedness quality [8, 9].

There are a large number of guidance documents available, each describing preparedness recommendations and activities for specific situations or specific stakeholders. When countries want to prepare themselves in general for a wide range of threats it is very difficult to assimilate the generic elements that are described across these different guidance documents. A set of practical, widely accepted and easy to use recommendations for generic preparedness that integrates guidance across all aspects of the preparedness emergency cycle supports countries in preparing for any possible health threat. Until now, there is no overview of recommendations for generic preparedness planning at operational level to support countries in preparedness planning at both the local and the national level.

The aim of this consensus procedure was to seek consensus from national-level experts from EU/EEA countries on identification of core principles of public health emergency preparedness. The findings can be used in a systematic and integrated approach to public health emergency preparedness planning.

Methods

A multistep approach was developed to achieve consensus on key recommendations for planning for public health emergency preparedness from a national perspective and to prioritize the recommendations (Fig. 1). A literature review was conducted, followed by a modified Delphi method [10] that consists of a series of consecutive steps: a questionnaire to achieve consensus on core set of recommendations, a face-to-face consultation, preselection of prioritized recommendations by the researchers, a questionnaire to achieve consensus on the prioritized set and a face-to-face consensus meeting on the prioritized recommendations.

Fig. 1

Flowchart of recommendations through all steps

Step 1: Literature study

For the literature study grey literature was identified relating to public health emergency preparedness applicable to all European countries. As information sources the websites of the WHO and ECDC were scanned and via Google websites of other public health institutes and organizations (performed by EB and DR). Documents produced by international organizations were included, because their guidance documents apply to more than one country. Documents were selected as a source of evidence if they described public health emergency planning from a national perspective, provided guidance for preparedness, described tools or checklists to assess the level of preparedness, or described lessons learnt for emergency planning from a national perspective. In addition, country-level public health emergency preparedness plans were included that were available for review and developed after the 2009 pandemic (H1N1) influenza outbreak. Documents were systematically assessed for data items: all relevant information, recommendations or questionnaire items extracted that expressed a recommendation on preparedness from a national perspective, see Fig. 1. The extraction was done by two researchers (EB and DR) independently. After the extraction all recommendations were refined by four researchers (EB, DR, CS and AT together) to make them generically applicable. This means that disease specific recommendations were altered textually to make them applicable for a wide range of infectious diseases, recommendations that were solely applicable in a disease specific context were excluded and recommendations that contained the same message were excluded. A common set of domains were identified in the extracted recommendations, based on the expertise of the team and the extracted recommendations. The recommendations were processed in an online questionnaire (EU Survey), see Additional File 3 for the recommendations included in the questionnaire and Additional file 1 (Questionnaire core set of recommendations) for the questionnaire.

Step 2: Questionnaire core set of recommendations

In step 2 we presented the preselected recommendation to a panel by means of a digital questionnaire. We invited the National Focal Points for Preparedness and Response (NFP P&R) of 31 EU/EEA countries, their alternates, or another expert from their country with at least 3 years of experience in preparedness planning for our panel. The NFPs P&R are experts designated by their member states to represent them in international meetings. We aimed for one response per country. The invitations were sent by ECDC. Non-responders received at least one reminder.

The expert panel was asked to appraise the relevance of each of the recommendations in an online survey on a 9-point Likert Scale (1 = not relevant, 9 = very relevant). Additionally, an open textbox was provided for any comments or adding new recommendations, for each main section of the questionnaire.

Analysis

Median relevance scores were calculated for each recommendation. Recommendations with a median score > 7 and > 70% of the scores in the highest tertile indicated that the recommendation were regarded as accepted by the panel, whereas recommendations with a median score > 7 and < 70% of the scores in the highest tertile were identified as requiring further consultation. Recommendations with a median score of 7 and > 70% of the scores in the highest tertile were labelled to be discussed by the research team, whereas recommendations with the score of 7 and < 70% of the scores in the highest tertile were excluded [10]. Finally, a recommendation with a median score < 7 indicated that the recommendation was rejected by the expert panel. This analysis is derived from the RAND/UCLA Appropriateness Method User’s Manual [10]. In our analysis we use a median that is more strict then in the RAND manual to ensure we only select recommendations that are valued highly by the entire group of experts.

If in the open text box comments were added concerning a recommendations, the team (EB, DR, CS, AT) assessed the comments and a decision was made whether panel consultation was needed on that recommendation even if the analysis of the scores concluded otherwise.

Step 3: Face-to-face consultation core set of recommendations

In step 3 we conducted a face-to-face meeting to achieve consensus on the recommendations that were identified as requiring further consultation. For the face-to-face consultation, we sent invitations to the same experts as invited for the questionnaire. It was possible for countries to send a delegate if necessary. The 2-day meeting was organized on the 10th and 11th of April 2017 and was held in Utrecht (the Netherlands). Prior to the meeting all experts received a personal feedback report, which included details on the analysis of the scores, an overview of the group scores and their personal score for each recommendation. The aim of the face-to-face meeting was to discuss the recommendations that were not directly accepted or rejected by the expert panel during the online survey, in order to reach consensus on a complete set of recommendations. After the plenary discussion the experts voted for the in- or exclusion of recommendations needing further consultation, recommendations with suggested textual adjustments, as well as proposals of new recommendations. We used a threshold of 70% for acceptance. Furthermore, criteria for the preselection of priority recommendations were discussed with the expert group. After the set of recommendations was selected, the set was assessed by the research team and completed where necessary.

Step 4: Preselection prioritization

In step 4, 5 and 6 we aimed to prioritize the selected recommendations, as the set resulting from step 3 could be too extensive for practical use in preparedness evaluation and planning. A preselection of the selected recommendations from step 3 was done by five experts (EB, DR, CS, AT, AJ). The preselection was performed based on the following criteria: the recommendation is essential for PHEP, the recommendation is feasible and accessible for all EU countries, the recommendation functions as enabler for other recommendations. All recommendations had to meet all criteria to be preselected. The experts discussed all recommendations and determined per recommendation whether the recommendation met the criteria. The preselected recommendations served as input for step 5.

Step 5: Questionnaire prioritization

We invited the experts as in step 2 for the panel. The expert group was invited per e-mail by the ECDC. Non-responders received at least one reminder. See Additional file 2 (Questionnaire baseline set) for the questionnaire.

We asked the experts in an online questionnaire (EU Survey) to indicate on a 9-point Likert scale (1 = not appropriate, 9 = very appropriate) the appropriateness of the preselected recommendations as a baseline set needed to achieve preparedness, applicable for all countries. There was the option for ‘no opinion’ and an open text box for comments to add recommendations. The questionnaire was structured according to the seven domains.

Analysis

The analysis for step 4 was the same as the analysis for step 2.

Step 6: Face-to-face consultation prioritization

The face-to-face expert meeting was organized to discuss the recommendations that were not directly accepted or rejected as a prioritized recommendation, or were added by the expert panel during the online survey. The purpose of the discussion was to achieve consensus on the prioritized set of recommendations. After the plenary discussion the experts voted for the in- or exclusion of recommendations needing further consultation, recommendations with suggested textual adjustments, as well as proposals of new recommendations. We used a threshold of 70% for acceptance. The meeting took place during a regular NFP P&R meeting in Stockholm, organized by the ECDC on May 18th, 2017_.

Results

Step 1: Literature review

In total, the search identified twenty documents of grey literature (sources of evidence). The documents were characterised as follows: six were ECDC guidance documents, nine were WHO documents and one was developed by CDC and one by UNISDR. Three EU member state preparedness plans were identified that met our inclusion criteria, see Table 1. The 20 documents (sources of evidence) resulted in 253 extracted recommendations, see Fig. 1. All extracted recommendations were made generic by the researchers so they apply to any disease or country. This synthesis of results led to 147 recommendations to be included in the questionnaire, see Additional file 3. The recommendations were grouped in the following domains: governance, capacity building and maintenance, surveillance, risk-assessment, risk- and crisis management, post-event evaluation and implementation of lessons learned, see Fig. 2. The domain risk- and crisis management also includes risk communication.

Table 1 Included literature

Fig. 2

Public Health Emergency Preparedness cycle

The seven domains are grouped into three phases: The pre-event phase spans all activities related to planning and anticipation, whereas the event phase focuses on the execution of existing preparedness plans in response to a (potential) public health threat. The post-event phase takes place after the recovery from a public health threat and emphasises the continuous improvement of all domains and elements represented in the PHEP process.

Step 2: questionnaire core set of recommendations

Out of the 31 invited countries, 27 responded to the questionnaire. Two countries had two experts filling out the questionnaire from local and national level. Because every response was valued equally, we could only include one response per country to have a balanced panel. Therefore, we used the response of the respondent with national expertise and excluded the other responses. The represented countries are described in Table 2. The years of experience with preparedness planning ranged from 2 to 34 years. The experts worked at the national public health institute or at the Ministry of Health.

Table 2 Expert panels

The expert group directly accepted 143 recommendations, one recommendation was rejected and three recommendations needed further consultation (see Additional File 3). The analysis of the open text boxes responses indicated that seven recommendations required textual adjustment. One new recommendation was suggested by the experts.

Step 3: Face-to-face meeting core set of recommendations

Twelve country experts and two experts not representing a country attended the meeting, see Table 2. The two experts not representing a country attended the meeting because of their professional interest. Experts were able to discuss in plenary and vote for the inclusion and exclusion of recommendations. These included recommendations needing further consultation (resulting from the statistical analysis), recommendations with suggested textual adjustments, as well as proposals of new recommendations, based on the comments provided in the open text boxes.

In total, eleven recommendations were discussed in the face-to-face meeting (three needing further consultation, one new added recommendation and seven proposed textual adjustments). Out of the eleven recommendations discussed in the face-to-face meeting, eight were textually adjusted by the experts, two were rejected and one was added. Three recommendations were added by the researchers because we identified elements were missing when evaluating the set. In total, 149 recommendations were included in the core set of recommendations, see Table 3.

Table 3 Core set of recommendations (149 recommendations)

The recommendations were grouped among seven domains: Governance (57 recommendations), Capacity building & maintenance (11 recommendations), Surveillance (19 recommendations), Risk assessment (16 recommendations), Risk and crisis management (35 recommendations), Post-event evaluation (6 recommendations) and Implementation of lessons learned (5 recommendations).

Step 4: Preselection prioritization of recommendations

In total 38 recommendations from all domains were preselected to be included in the prioritized set of recommendations (Additional File 4).

Step 5: Questionnaire prioritization of recommendations

Experts from 23 EU/EEA countries out of 31 completed the questionnaire, see Table 1. The experts represented the NFPs P&R. The experience of the experts ranged from < 3 years (1 expert), to > 10 years (9 experts). The experts worked at the national public health institute or at the Ministry of Health. Out of the 38 preselected recommendations, 36 were directly accepted by the expert panel, zero were rejected, and two needed further consultation, see Additional File 4. Three recommendations (added by the researchers in step 3) were assessed by experts of ten countries because they were added to the digital questionnaire when some expert already finished the questionnaire. Table 4 shows the prioritized set of recommendations. If all respondents scored the recommendation with a 7, 8 or 9 the recommendation is described as subject to ‘universal’ acceptance. If one or more respondents scored 6 or lower, the recommendation is described as subject to ‘majority’ acceptance. Given this method, there were 7 recommendations subject to universal acceptance and 36 to majority acceptance. All recommendations in Table 4 met the selection criteria.

Table 4 Prioritized set of recommendations

Step 6: Face-to-face meeting prioritization

The step 6 face-to-face meeting was a part of a regular NFP&PR meeting. Nineteen countries participated in the face-to-face meeting. The experts present were the NFP&PR members or their alternates. There were several parallel sessions and experts could choose to attend the session of their interest. In the meeting, five selected recommendations were altered textually and two recommendations were added to the prioritized set. In total, 42 recommendations were prioritized (Table 4).

Discussion

In this consensus procedure, EU/EEA country preparedness experts reached consensus on core priority principles of public health emergency preparedness. Experts selected 149 core recommendations and prioritized 42. The recommendations selected support EU/EEA countries in preparing for public health emergencies by providing guidance across the full spectrum of preparedness, describing governance, capacity building & maintenance, surveillance, risk assessment, risk and crisis management, post-event evaluation and implementation of lessons learned.

In the selection of the core set of recommendations (step 2 and 3), practically all recommendations were accepted by the panel and only one recommendation was added. In the prioritization, experts accepted most preselected recommendations (36 out of 38). Experts could add recommendations from the core set to the prioritized set if they thought this was needed. However, the aim of this consensus procedure was to select recommendations that were accepted and applicable for the majority of countries. Differences between countries regarding current level of preparedness, available resources and healthcare organizations can influence the score a country gives for a recommendation. In the questionnaire and consensus meeting, experts had the opportunity to provide comments or suggestions to add, delete or modify the recommendations. The recommendations in our consensus procedure are formulated in a way that they can be used by all countries.

The selected recommendations in this consensus procedure are in line with public health emergency indicators developed by other organisations. Khan et al. [11] performed a Rand Modified Delphi with Canadian experts to develop indicators for public health emergency preparedness. The selected recommendations in the paper of Khan are similar to the recommendations selected in our consensus procedure. This implies that the selected recommendations in our consensus procedure are not only suitable for EU/EEA countries but may be valuable for countries worldwide. WHO developed a strategic framework including elements of preparedness on different levels [12]. Our recommendations are in line with the elements of preparedness in this framework, but provide more detail on the practical application.

In the face-to-face meetings most discussion concentrated on terminology. Terminology in public health preparedness is not always uniform and countries interpret terms differently. When aiming to achieve consensus on a set of recommendations that is applicable to multiple countries, it is very important that everyone interprets terms and concepts in the same way. Hence, discussion on terminology was not unexpected. During the face-to-face discussion the recommendations were reformulated by the participating experts in a way that the recommendation was clear to all experts and possible double interpretation unlikely. Countries can have various reporting systems, coordination structures and a different organization of healthcare. Therefore, one term can mean different things for different countries. In such a multinational context, achieving consensus requires a face-to-face meeting, to clarify recommendations and concepts behind them.

Another pattern is the content of the recommendations. The majority of the recommendations describe the domains ‘governance’, ‘surveillance’ and ‘risk assessment’. While the domains ‘post –event evaluation’ and ‘implementation of lessons learned’ contain only small amount of recommendations. In the included literature, there is a strong focus on the content based recommendations. This could be explained by the difficulty of defining recommendations on when and how to implement lessons learned. However, we believe that post-event evaluation and implementing lessons learnt are highly important for quality improvement. Training, exercises, threats and events should be used to learn valuable lessons for future outbreaks. Moreover, preparedness plans and future trainings should be updated regularly based on lessons learnt.

In this consensus procedure solely grey literature was included since the scientific evidence available on outbreak preparedness has a low evidence base, is mostly disease specific and often does not describe preparedness from a national perspective [11, 13].

In a previous literature review [14] and modified Delphi procedure [13] we systematically reviewed scientific literature and selected recommendations for generic preparedness from the perspective of first responders (among others hospitals, general practitioners, municipal health services, ambulance services). These recommendations describe a wide range of preparedness activities from developing a preparedness plan, preparing for control measures, coordination and collaboration, and evaluation. The previous study and the process described in this paper are complementary. A country that is well prepared for public health emergencies is prepared on all levels, from the local level to the national level. When assessing the level of preparedness within a country, both the set targeted at the national level and the set targeted at the first responders should be used.

Our consensus procedure has several strengths and limitations. In this consensus procedure we selected recommendations using a systematic approach. One of the strengths is that all EU/EEA countries were invited for the panels and their responses were valued equally. The value of a face-to-face meeting can be found in building a common understanding of preparedness needs and priorities among countries. The bottom-up approach of this consensus procedure contributes to a widely accepted set of recommendations and a common understanding of preparedness. In both our panels, a very high number of countries were represented (27 and 23). Although in both face-to-face meetings the number of countries represented was lower (12 and 19), both groups contained a sufficient number of participants, as compared to the recommended 7–15 participants for a Delphi panel [10].

One of the limitations is that the basis for the recommendations selected in this consensus procedure lies within grey literature because these documents were used to extract the recommendations from, and that no scientific literature was used. We did not include scientific literature because the evidence base in scientific literature regarding preparedness planning is low [11, 13]. We therefore included literature from international organizations that have a leading role in preparedness planning and are developed not by one single author but by a group of international experts.

The recommendations extracted from literature formed the basis of the first questionnaire. This could potentially have pushed the experts in a certain direction because the experts to assess a list of recommendations and not to come up with a set of recommendations from a blank page. However, experts had the opportunity in the online questionnaires and during the face-to-face consultation meetings to add, remove or adjust recommendations. After our Delphi procedure, new relevant documents were published that could not be included because the data collection was already finished. An example of these relevant documents are a report describing competencies for individuals who work in emergency preparedness [15], and the WHO framework for emergency preparedness [12]. The recommendations selected in our consensus procedure provide a detailed description of the elements of preparedness as described in the WHO strategic framework [12].

Conclusions

The recommendations identified in our consensus procedure can be a useful guidance for preparedness planning in EU/EEA countries. This can be done in various ways, for example providing a framework for development of preparedness evaluation, and incorporating the recommendations in guidelines. A tool aiming to assess a country’s level of preparedness was developed based on the recommendations selected and prioritized in this consensus procedure [16]. The recommendations represent the criteria on which the level of preparedness is evaluated. This tool comprises the core set of recommendations and the prioritized set of recommendations. Countries can use this tool to assess their level of preparedness and identify gaps and priorities in preparedness planning. The prioritized set provides direction for the most urgent recommendations to implement.