Skip to main content
Download PDF

Constructing an emergency preparedness evaluation index system for public use during major emerging infectious disease outbreaks: a Delphi study

BMC Public Health volume 23, Article number: 1109 (2023) Cite this article

Abstract

Background

The major emerging infectious diseases (MEIDs) have occurred frequently and become increasingly serious in the world. Sufficient personal emergency preparedness is critical for the general people in efficiently responding to and recovering from MEIDs. Nevertheless, few specific indicators are available for assessing the individual emergency preparedness of the general public during these periods. Therefore, the aim of this study was to construct an index system for comprehensively evaluating the personal emergency preparedness of the public regarding MEIDs.

Methods

Based on the global national-level emergency preparedness index framework and a literature review, a preliminary index system was constructed. From June 2022 to September 2022, a panel of 20 experts from nine provinces and municipalities across multiple research areas participated in this Delphi study. They rated the importance of pre-defined indicators using a five-point Likert scale and provided their qualitative comments. According to the feedback of each round of experts, the indicators of the evaluation index system were revised.

Results

After two rounds of expert consultation the evaluation index system reached a consensus, containing five first-level indicators, cooperating with prevention and control work, improving emergency response capacity, securing supplies and equipment, preparing economic resources, maintaining physical and mental health with affiliated 20 s-level indicators and 53 third-level indicators. The expert authority coefficient of consultation was 0.88 and 0.90. The Kendall’s coefficient of concordance of expert consultations was 0.294 and 0.322, respectively. The differences were statistically significant (P < 0.05).

Conclusion

A valid, reliable and scientific evaluation index system was established. This personal emergency preparedness index system, as a precursor form, will further lay the foundation for the formation of an assessment instrument. At the same time, it could provide a reference for future education and training of emergency preparedness for the general public.

Peer Review reports

Background

Since the 21st century, major emerging infectious diseases (MEIDs) have occurred frequently and become increasingly serious, against a background of globalization and economic and cultural exchange [1]. MEIDs such as Severe Acute Respiratory Syndrome (SARS), human avian influenza, influenza A (H1N1), avian influenza A (H7N9), Ebola hemorrhagic fever, the Middle East Respiratory Syndrome (MERS), and Corona Virus Disease 2019 (COVID-19) have appeared in recent years [2]. However, MEIDs have the characteristics of suddenness, high infectivity and uncertain prognosis, which seriously affect public health, social stability and economic development [3]. Countries across the world have realized that preventing and controlling MEIDs has become a significant public health issue [4, 5].

Emergency management cannot completely rely on the power of the government [6]. It requires the joint participation and close cooperation of multiple sectors of society. It is necessary for the public to have the facility to identify risks early, respond scientifically to disasters and rebuild after disasters. Although MEIDs usually appear suddenly, appropriate preparedness of the public can mitigate the harm caused by MEIDs [7]. The COVID-19 has provided a stark example of the need for emergency preparedness in outbreak preparedness [8]. Emergency preparedness of the general public for MEIDs is defined in this study as the capacity of individuals effectively to prevent, respond to, and recover from MEIDs, including the preparedness of emergency skills, legal compliance, avoiding secondary disasters, economic estimation, and somatopsychic health [9].

Although studies have been developed in recent years to evaluate the emergency preparedness of the national government, public health departments, and healthcare workers during MEIDs [10,11,12], few studies have focused in depth on emergency preparedness at the public level. Ou Yanling et al [13] developed an evaluation index system of residents health emergency literacy for major epidemics. The evaluation content of this index system was not sufficiently comprehensive because it mainly screened out indicators from the six areas of basic cognition, basic attitude, scientific knowledge, behavioral knowledge, cognitive skills and operational skills of major epidemic health emergency, and did not include comprehensive preparation for legal compliance, economic estimation, material reserves, physical and mental health. Heagele TN et al [14] constructed Household Emergency Preparedness Instrument, which had four dimensions: preparedness actions; communication plans; evacuation plans; and disaster supplies. The tool was highly reliable, but the results obtained were only on the competencies required in routine situations and did not address the competencies required during MEIDs. Therefore, this study intends to construct a standardized, comprehensive and practical emergency preparedness evaluation index system for public use during MEIDs.

Methods

This study was approved by the Ethics Committee of the Affiliated Hospital of Southwest Medical University (NO. KY2022345). A Delphi study was conducted to explore experts’ attitude towards the evaluation index system. To explore the rationality of it, each expert was invited to rate the importance of a series of pre-defined indexes independently [15]. A modified recommendation for the Conducting and Reporting of Delphi studies (CREDES) was used to guide the study [16].

Study design

The Delphi method is considered to be an established and adaptable research method for querying experts and ultimately transforming expert opinions into group consensus [17]. Meanwhile, Delphi study is regarded as a flexible research methodology to set goals, items, etc. Researchers can send questionnaires and communicate with experts online or offline, so they do not need to consider the geographical location of experts. The process of gathering experts’ advice is independent and anonymous, ensuring that the experts will not discuss or exchange views [18]. Through iterative multistage process, accurate and reliable data will eventually be obtained.

Expert selection

Purposive sampling augmented by snowball recruitment was employed to deliberately select Delphi panel members from different regions and organizations in China [19], with the following inclusion and exclusion criteria: 1.inclusion criteria: (a) they engaged in public health management, infectious diseases prevention and control, disease control management, or psychology (those who have post-epidemic psychological counseling experience ); (b) their work experience over eight years; (c) they obtained the professional title qualification of intermediate or above; (d) they had bachelor’s degree or above; and (e) they voluntarily participate in this study and actively give feedback. 2. exclusion criteria: (a) they were reluctant to participate in this study due to personal reasons; (b) they had no practical experience of working with MEIDs. (c) they withdrew from the study while the study was ongoing. Finally, we recruited 20 experts according to the published recommendations [20].

Construction an initial evaluation index system

We constructed an educational content framework for emergency preparedness of the public during MEIDs, which has been published [9]. Based on our previous studies, supplemented by literature review, a preliminary draft of the evaluation index system was constructed. The initial draft hinged on the global national-level emergency preparedness index framework [10], the emergency plan for public health emergencies [21] and the citizen health emergency literacy in China [22]. Furthermore, the initial evaluation index system included five first-level indicators, 21 s-level indicators and 55 third-level indicators (Appendix 1). The first-level indicators are the main dimension of the public emergency preparedness, and the core indicators. The second-level indicators are classified and defined according to the emergency preparedness contents contained under the first-level indicator. The third-level indicators are the specific evaluation indicators under the second-level indicator.

Questionnaire design

The expert consultation questionnaire was composed of four parts. (1) Preface: this section briefly explained the research purpose, the content, the requirements for completing the questionnaire, etc. (2) General information of the experts: age, educational background, professional title, working years, research fields, etc. (3) Expert consultation form of the evaluation index system: this part was the main body of assessment in the consultation questionnaire, all the indicators were shown in this section. And the importance of each indicator was assessed by experts using the Likert five-level scoring method. Points were scored from five to one in order: very important = 5, important = 4, neutral = 3, unimportant = 2, and completely unimportant = 1. In addition, the experts could freely express their personal ideas and suggestions in this part to enrich the content for the evaluation index system [23]. (4) Expert familiarity with the content of the study and index judgment.

Delphi consulting and feedback cycle

The researchers contacted the experts personally via emails, WeChat® or other means between June and September 2022, experts did not communicate with each other. The researchers sorted out the opinions, suggestions and feedback of each expert, and readjusted the consultation questionnaire for the second round for further confirm accuracy of the evaluation index system [24]. Additionally, the index inclusion criterion: the average importance score of each index evaluated by the expert panel > 3.5, the coefficient of variation < 2.5, and the full mark rate > 20% [25]. When some important indicators were not in the inclusion criterion, the research team needed to discuss whether the indicator was retained or deleted [26]. After two rounds of expert consultation, the evaluation index system reached a consensus.

Data analysis

Excel 2010 and SPSS 25.0 software were used for statistical analysis. Frequency and percentage are used to describe the personal information of experts. The enthusiasm of experts was expressed by the response rate of the questionnaire. The authority coefficient (Cr) of the expert was the mean value of the expert’s judgment basis (Ca) and degree of familiarity (Cs) with the research content, based on the formula Cr = (Ca + Cs)/2. The degree of expert’s opinion dispersion was represented by the Kendall coefficient of concordance (Kendall’s W) and coefficient of variation (CV), also the importance of each index was described by mean ± standard deviation. P < 0.05 was considered to indicate statistical significance for the differences. The weight coefficient of each index was determined by establishing the judgement matrix using yaahp12.9 software (a software developed by Shanxi Yuanshi Software Technology Co., Ltd) with the Analytic Hierarchy Process (AHP).

Quality control

To ensure the scientific credibility and accuracy of the research results, the criteria for selecting experts were strictly formulated. Garnett et al [27]claimed that choosing experts from the same background may contribute to a certain degree of bias in the results of a study. Accordingly, in this study, the experts we selected were devoted to multiple research areas. If the returned questionnaires were found to be incorrect or incomplete, we would contact the experts in time for verification. If more than 10% of the questionnaire was not completed, it would be considered as an invalid questionnaire and eventually deleted. All data were entered by two researchers using Excel 2010 software.

Results

Expert sociodemographic information

The present research enrolled a panel of 20 experts from Sichuan, Chongqing, Guangxi, Shanghai, Yunnan, Hubei, Guizhou, Hunan and Guangdong in China. The working years of experts are between 8 and 38 years. All the experts had experience working with an epidemic. The sociodemographic details of the experts were presented as Table 1.

Table 1 The socio-demographic information of the experts
Full size table

Enthusiasm and authority coefficient of experts

The enthusiasm of the experts can be expressed by the response rate of the questionnaire. It is generally considered that experts have fairly high enthusiasm when the response rate is greater than 70% [28]. In the first round, 20 questionnaires were distributed and all of them were returned, with a response rate of 100%. In the second round, 20 questionnaires were distributed, and 18 effective questionnaires were returned, with a response rate of 90% (Table 2). It is claimed by researchers that the expert consultation authority coefficient > 0.7 is considered to be reliable [29].Through rigorous calculation, the authority coefficient (Cr) of the two rounds of expert consultation were 0.88 and 0.90, suggesting that the experts were highly authoritative.

Table 2 Positive coefficient of experts
Full size table

Degree of concentration and coordination of experts’ opinions

In the first round of consultation, the importance scores of all items ranged from 3.55 to 4.95, and the full mark rate was greater than 20%. The coefficient of variation (CV) was less than 0.25, except for one index whose coefficient of variation (CV) was 0.26. The Kendall’s W was 0.294 (χ2 = 470.614, P < 0.001). In the second round of consultation, the importance scores of all items ranged from 3.83 to 5.00. The full mark rate was greater than 20%, and the coefficient of variation (CV) was less than 0.25. The Kendall’s W was 0.322 (χ2 = 445.703, P < 0.001). (Table 3)

Table 3 The results of the degree of coordination among experts
Full size table

The formation process of the evaluation index system

In this study, we conducted two rounds of consultation. According to the screening criteria of indicators, experts’ opinions, and the research group discussion, the indicator system was adjusted until a consensus was reached (Table 4).

Table 4 The process of items revision
Full size table

In the first round of consultation, the experts did not make any comments on the first-level indicators, so they were all retained. However, the second-level indicators, three experts pointed out that ' cooperate with epidemic prevention and control of each department ' had logical problems with the same level indicators, they recommend revising it to ' cooperate with epidemic prevention and control of responsible departments '. The traffic health quarantine station is also a responsible department for epidemic prevention and control [21], experts suggested that it should be a third-level indicator, we deleted the ' cooperate with quarantine of department of transportation and health ' and adjusted it to the third-level indicators. While in the third-level indicators, several experts claimed that the public is not yet fully equipped to analyze the secondary disasters caused by MEIDs, and accordingly 'analysis of possible secondary disasters caused by epidemic' was deleted. Three experts pointed out that 'strictly comply with centralized isolation requirements ' was not clearly defined, and it should be revised to 'strictly comply with various control requirements related to isolation' which could make it more comprehensive. According to the experts’ suggestions, in addition to revising the above items, duplicate indicators were removed, and combined with the special requirements during MEIDs, the content of personal emergency preparedness was supplemented, as presented in Appendix2. Then, a new consultation questionnaire was developed and a total of 78 indicators were incorporated into the round 2 survey.

In the second round of expert consultation, experts had relatively unified opinions on each indicator. Only two experts proposed amendments, including adjusting the order of indicators and modifying the expression of indicators. In the reserve of emergency supplies during MEIDs, it was not recommended people to purchase unusually large amounts of products for which would result in a frenzied purchasing of supplies [30]. Thus, experts suggested changing ' purchase sufficient amount of emergency living goods, such as grain, oil and rice ' to ' purchase appropriate amount of emergency living goods, such as grain, oil and rice '. The other three similar indicators were also modified like this. Through the feedback of two rounds of Delphi study, the details of the content of the evaluation index system have been enriched and improved. All the experts agreed on the index system, the evaluation index system was finally formed, consisting of five first-level indicators, cooperating with prevention and control work, improving emergency response capacity, securing supplies and equipment, preparing economic resources, maintaining physical and mental health with affiliated 20 second-level indicators and 53 third-level indicators (Table 5).

The index weight

AHP could analyze experts’ subjective judgment with mathematical form and conduct multi-objective decision-making analysis of scientific treatment to ensure the best results [26]. The weights of the indicators were calculated by the AHP according to the experts’ opinion, and the consistency test of all levels of indicators were CR < 0.10, demonstrating the judgment matrix was within the accepted range. The first-level indicators, cooperation with prevention and control work (0.3892) showed the highest value, followed by fully guaranteed supplies and equipment (0.2474), while improving emergency response capacity and maintaining physical and mental health showed the same value (0.1386). Finally, preparing economic resources (0.0862) was the smallest. The index weight of all levels of indicators are shown in Table 5.

Table 5 The emergency preparedness evaluation index system for public use during MEIDs
Full size table

Discussion

Content analysis of the evaluation index system

Emergency preparedness, while not an arbitrary concept, is dependent on many factors [31]. This study has compiled a set of relatively comprehensive rating index system, including 53 items of knowledge, skills, emotion and behavior tendency about personal emergency response in five aspects.

Cooperate with prevention and control work

From the experts’ preference, cooperate with prevention and control work (0.3892) is particularly important and indispensable in the public emergency preparedness during MEIDs. It clearly divides the public emergency preparedness dimension from the most basic cooperation with the individual level of epidemic prevention and control, then to cooperate with the mass prevention and control, and finally rises to cooperate with the laws and regulations. This part aims to assess the public basic attitude and awareness of infectious disease prevention and control. Previous studies have identified attitudes as a key determinant of emergency preparedness [32, 33]. After the outbreaks, the impact is generally global. However, it cannot be ignored that the form of prevention and control is complex and changeable, the difficulty of prevention and control is huge, and the task of prevention and control is also very arduous [34]. Upholding the concept of self-discipline, the public cooperate with the government, relevant departments and agencies to carry out epidemic prevention and control work, which are both self-help and altruism [35, 36]. Such as complying with government or community arrangements and truthfully reporting infectious disease exposure history.

Fully guaranteed supplies and equipment

Fully guarantee supplies and equipment (0.2474) aims to assess the public could or could not reserve living materials during the epidemic, including a series of food, protective tools, etc., which are indispensable for human beings depend during the MEIDs [37, 38]. When a major emerging infectious disease is underway, we should wear protective equipment which can reduce the chance of being infected, and can also prevent patients or asymptomatic infected persons from spreading the virus in society [39]. During this period, if the public lacks medicine to deal with common diseases, they have to go out to see a doctor, thus increasing the probability of infection. Experts also considered reserving emergency supplies to be fairly important, but we should reserve them properly. Thus, they suggested changing the ' sufficient ' to ' appropriate ' in this part. It was reported that people have opted for conformity consumption to obtain a sense of belonging and security from the group during the infectious disease, thereby alleviating inner fear [40]. However, people often scramble to emergency supplies, which may lead to panic behavior and have a serious negative impact on public health management [30]. Therefore, it should be emphasized to guide the public behavior of purchasing emergency supplies.

Improve emergency response capacity

Improving emergency response capacity(0.1386) includes personal cognition of infectious diseases, judgment of epidemic risk, self-protection and seeking help. The purpose of this part is mainly to evaluate whether the public has a clear understanding of infectious diseases, and whether they can protect their own safety through themselves or others during the outbreaks. When the public own the knowledge of infectious diseases, they may not have a strong sense of panic and can make a more accurate judgment on their current situation based on their own experience. When encountering difficulties, they should know about how to seek help from institutions [41]. In China, some green channels for special groups are generally set up.

Maintain physical and mental health

Maintaining physical and mental health (0.1386) just to evaluate whether the public has a healthy lifestyle, which can also reflect whether they could positively and steadily live through the outbreaks. Along with the development of positive psychology, the guidance of positive emotions in public should be strengthened in MEIDs, and the development of positive psychological qualities should be emphasized to promote physical and mental health [42]. Maintaining good health could increase individual resistance, which in turn could reduce infections during MEIDs. Also, the public may experience psychological problems such as anxiety, depression, insomnia, post-traumatic stress disorder, and suicidal thoughts due to excessive stimulation and inability to cope with the epidemic [42].Then there would be secondary disasters in the mental, especially those on the epidemic prevalence areas, and the mental trauma is more serious [43, 44].Therefore, it is particularly necessary to maintain mental health before and after the outbreak of MEIDs.

Prepare economic resources

Preparing economic resources (0.0862) seems no obvious significance. However, during the COVID-19 pandemic, a highly representative viral disaster, some public suffered financial difficulties indeed. Thus, it is necessary for the public to have the facility to use their own resources reasonably. Judging from the scores of the expert panel, the overall score is indeed the lowest, which may be because experts consider that China’s medical security department provides positive treatment guarantee for MEIDs, which can largely relieve the worries of patients [45]. However, the daily expenses of the public are still a problem. When they are confined at home or cannot go out to earn money due to special epidemic conditions, how to control their own deposits also needs to be carefully considered.

The profound meaning of the evaluation index system

The COVID-19 is responsible for millions of deaths globally [46, 47], and continues to demonstrate the risks and profound health impacts that result from infectious disease emergencies. The lockdown measures introduced led to the collapse of medical systems, the outbreak of economic crises, and serious social disorder in many countries [48]. MEIDs and concomitant pandemic measures are highly destructive, sudden, complex and uncertain, which seriously threaten the health and safety of the public. The preparedness of the public to cope with emergencies is growing importance [49]. Efficient emergency preparedness can not only help the public respond to MEIDs, but also alleviate the negative emotions towards MEIDs [50]. Brown KL has pointed out that the public preparedness research mostly focuses on general emergency preparedness rather than preparedness on specific hazards [31]. In the context of the increasing number of new infectious diseases [51], it is particularly necessary to construct a specific evaluation index system and to evaluate the emergency preparedness of public, which not only allows the public to know the aspects of their weaknesses in preparedness, but also facilitates relevant educators to provide targeted guidance to them.

Reliability and scientific credibility of the evaluation index system

The emergency preparedness evaluation index system for public use during MEIDs constructed in this study is scientific, comprehensive and diversified with the following characteristics. Firstly, it is scientific and authoritative to some extent, since it was built based on the the mature and widely used global national-level emergency preparedness index framework. This index system was also built based on massive literature support [21, 22, 52,53,54,55]. Secondly, it is reliable due to qualified experts. In this study, 20 experts are influential and outstanding in the prevention and control of MEIDs. Moreover, they are from different regions and different departments, reducing area distribution bias to some extent. At the same time, the experts involved in this study specialized in multiple research areas, they could give targeted suggestions to this study, thus making the evaluation index system more scientific and reliable. Thirdly, the evaluation index system is systematic and comprehensive, since it does not only include the basic behavior and skills to deal with MEIDs, but also consider the comprehensive preparation for legal compliance, economic estimation, material reserves, physical and mental health. The evaluation content runs through the whole epidemic response process. Fourthly, experts all held intermediate and above titles, of which senior professional titles and above account for 60%. In addition, three-quarters of the experts were postgraduate students, indicating that the suggestions and comments made by them were based on rich theoretical knowledge and practical experience. The authority coefficients of this research were 0.88 and 0.90, which proved that the authority of the study was assured. The Kendall’s concordance coefficient of the two rounds of consultation were statistically significant, so it suggested again that the results of the evaluation index system were scientific and reliable.

Conclusion

A valid, reliable and scientific evaluation index system was established through two rounds of expert consultation. This emergency preparedness index system, as a precursor form, will further lay the foundation for the formation of an assessment instrument and provide reference for future education and training of the public. However, the initially constructed index system was still limited to the theoretical framework, and a further study will implement empirical research to analyze the reliability and validity of the index to verify its practicality, applicability and feasibility.

Data Availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Abbreviations

AHP:

Analytic Hierarchy Process

COVID-19:

Corona virus disease 2019

CV:

Coefficient of variation

Kendall’s W:

Kendall coefficient of concordance

MEIDs:

Major emerging infectious diseases

SD:

Standard deviation

References

  1. Wu C, Wu P, Li P, Cheng F, Du Y, He S et al. Construction of an index system of core competence assessment for infectious disease specialist nurse in China: a Delphi study. BMC Infect Dis. 2021 Aug 10;21(1):791. https://doi.org/10.1186/s12879-021-06402-2

  2. Fang XE, Chen DP, Cheng KY, Mao YJ, Lu M, Tang LL, et al. Cross-cultural adaptation, validity, and reliability of the chinese version of the Nurses’ perceptions of Disaster Core Competencies Scale (NPDCC). Ann Palliat Med. 2020;9(5):3304–33.

    Article  PubMed  Google Scholar 

  3. Lunt H, Heenan H. Mitigating the impact of disasters and emergencies on clinical tr-ials site conduct: a site perspective following major and minor unforeseen events. Contemp Clin Trials Commun 2019 Nov 12;16:100487.

  4. Wang Y, Wang X, Liu X, Ren R, Zhou L, Li C, et al. Epidemiology of Imported I-nfectious Diseases, China, 2005–2016. Emerg Infect Dis. 2018 Jan;25(1):33–41.

  5. Jones KE, Patel NG, Levy MA, Storeygard A, Balk D, Gittleman JL et al. Global trends in emerging infectious diseases. Nature. 2008 Feb 21;451(7181):990-U4.

  6. Yu Z, Ma J. Review of government information disclosure mechanism in emergency management [ J ] Leadership Science Forum 2022(04):35–43. https://doi.org/10.19299/j.cnki.42-1837/C.2022.04.006

  7. Basolo V, Steinberg LJ, Burby RJ, LevineJ, Cruz AM, Huang C. The effects of con-fidence in government and information on perceived and actual preparedness for disasters.Environ Behav. (2009) 41:338–64. https://doi.org/10.1177/00139165083

  8. Skelton P, Salio F, Emiroglu N. Emergency preparedness and readiness; anticipating the need for rehabilitation. Bull World Health Organ. 2022 Nov 1;100(11):744–746. https://doi.org/10.2471/BLT.22.289085

  9. Luo Y, Li M, Tang J, Ren J, Zheng Y, Yu X, et al. Design of a Virtual Reality In-teractive Training System for Public Health Emergency Preparedness for Major Emerging Infectious Diseases: Theory and Framework. JMIR Serious Games. 2021 Dec 14;9(4):e29956.

  10. Oppenheim B, Gallivan M, Madhav NK, Brown N, Serhiyenko V, Wolfe ND, et al. Assessing global preparedness for the next pandemic: development and application of an epidemic preparedness index. Bmj Glob Health. 2019 Feb;4(1):e001157.

  11. Stoto MA, Savoia E, Nelson C, Piltch-Loeb R, Guicciardi S, Takacs J et al. Develo-pment of a public health emergency preparedness competency model for European Union countries. Euro Surveill Bull Eur Sur Mal Transm Eur Commun Dis Bull. 2018 Dec;23(49).

  12. Huiling Wang. Construction and test of ICU nurses’ emergency capability evaluation index system for major infectious disease emergencies [D]. Qingdao University; 2021.

  13. Yanling Ou. Research on influencing factors of residents’ health[D]. Nanjing University of Chinese Medicine, 2021.

  14. Heagele TN, McNeill CC, Adams LM, Alfred DM. Household Emergency preparedness Instrument Development: a Delphi Study. Disaster Med Public Health Prep. 2022 Apr;16(2):570–82.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Jones J, Hunter D. Consensus methods for medical and health services research. BMJ. 1995;311:376–80. https://doi.org/10.1136/bmj.311.7001.376

  16. Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conductingand REporting DElphi Studies (CREDES) in palliative care: recommendations based on a methodological systematic review. Palliat Med. 2017 Sep;31(8):684–706.

  17. McPherson S, Reese C, Wendler MC. Methodology update: Delphi Studies. Nurs Res. 2018 Oct;67(5):404–10.

    Google Scholar 

  18. Black M, Matthews LR, Millington MJ. Using an adapted Delphi process to develop a survey evaluating employability assessment in total and permanent disability insurance claims. Work Read Mass. 2018;60(4):539–48.

    Google Scholar 

  19. Quinn PM. Chapter 5. Designing qualitative studies In: Anonymous qualitative research and evaluation methods. Thousand Oaks, CA: SAGE Publications; 2015. pp. 243.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Shi C, Zhang Y, Li C, Li P, Zhu H. Using the Delphi Method to Identify Risk Fa-ctors Contributing to Adverse Events in Residential Aged Care Facilities. Risk Manag Healthc Policy. 2020;13:523–37.

  21. National Emergency Plan for Public Health Events. http://www.gov.cn/gzdt/2006-02/28/content_213129.htm. Accessed 20 Nov 2022.

  22. Citizen health emergency literacy items. http://www.nhc.gov.cn/yjb/s2908/201804/b2a724c794914d19b92b96e0882b9fbf.shtml. Accessed 20 Nov 2022.

  23. Toussaint L, Vincent L, Whipple AO, McAllister S MJ, Finnie M, Hathaway DC. Forgiveness education in fibromyalgia: a qualitative inquiry. Pain StudTreat. 2014;02(01):11–6.

    Google Scholar 

  24. Keeney S, Hasson F, McKenna H. The Delphi technique in nursing andhealth res-earch.West Sussex, UK:Wiley-Blackwell.

  25. Cao PY, Zhao QH, Xiao MZ, Gan P. A study on long-term care demand assessment indicators for the disabled elderly in nursing home. Chin J Nurs. 2017;52(08):980–5.

    Google Scholar 

  26. Shen L, Yang J, Jin X, Hou L, Shang S, Zhang Y. Based on Delphi method and Analytic Hierarchy process to construct the evaluation index system of nursing simulation teaching quality. Nurse Educ Today. 2019 Aug;79:67–73.

  27. Garnett C, Crane D, West R, Brown J, Michie S. Identification of Behavior Change Techniques and Engagement Strategies to Design a Smartphone App to Reduce Alcohol Consumption using a formal Consensus Method. JMIR MHealth UHealth. J-un. 2015;29(2):e73.

    Google Scholar 

  28. Dai F, Wei K, Chen Y, Ju M. Construction of an index system for qualitative evaluation of undergraduate nursing students innovative ability: a Delphi study. J Clin Nurs. 2019 Dec;28(23–24):4379–88.

  29. Long Y, Zhou X, Deng P, Liao X, Wu L, Zhou J, et al. Construction of a physiological aging scale for healthy people based on a modified Delphi method. Zhong Nan Da Xue Xue Bao Yi Xue Ban. 2016 Apr;41(4):422–6.

  30. Jin X, Li J, Song W, Zhao T. The impact of COVID-19 and Public Health Emergencies on Consumer Purchase of Scarce Products in China. Front Public Health 2020 Dec 2;8:617166. https://doi.org/10.3389/fpubh.2020.617166

  31. Brown KL. Emergency preparedness during a global pandemic: Individual preparedness for COVID-19. J Emerg Manag. 2021 Special Issue on COVID-19;18(7):99–113. https://doi.org/10.5055/jem.0533. PMID: 34723351.

  32. Fung OW, Loke AY. Disaster preparedness of families with young children in Hong Kong. Scand J Public Health. 2010 Dec;38(8):880-8. https://doi.org/10.1177/1403494810382477. Epub 2010 Sep 3. PMID: 20817655.

  33. Loke AY, Lai CK, Fung OW. At-home disaster preparedness of elderly people in Hong Kong. Geriatr Gerontol Int. 2012 Jul;12(3):524–31. https://doi.org/10.1111/j.1447-0594.2011.00778.x. Epub 2011 Nov 28. PMID: 22122609.

  34. Guo LS. Roots and outlets: reflections on the Novel Coronavirus Pneumonia and the Global Crisis - An interview with russian Scholars. Marxism and Reality. 2021;(01):152–8.

  35. Dang J. From ' selfish ' to ' extraordinary altruism ' - reflections on the role of education in the pandemic. Educational Rev. 2020;(04):33–6.

  36. Peng BL. Ethical ideas to be advocated in the Prevention and Control of Major Public Health events - - taking the Prevention and Control of Novel Coronavirus Pneumonia as an Example. Jiangh-an Forum. 2020;(03):5–9.

  37. Centers for Disease Control and Prevention. Emergency Preparedness. Available at https://blogs.cdc.gov/yourhealthyourenvironment/2021/09/22/emergency-preparedness/.Accessed March 7, 2023.

  38. Levac J, Toal-Sullivan D, O’Sullivan TL. Household emergency preparedness: a literature review. J Community Health. 2012;37:725–33.

    Article  PubMed  Google Scholar 

  39. Chughtai AA, Seale H, Macintyre CR. Effectiveness of Cloth Masks for Protection against severe Acute Respiratory Syndrome Coronavirus 2. Emerg Infect Dis. 2020 Oct;26(10):e200948. https://doi.org/10.3201/eid2610.200948

  40. Song W, Jin X, Gao J, Zhao T. Will buying follow others ease their threat of death? An analysis of consumer data during the period of COVID-19 in China. Int J Environ Res Public Health. 2020;17:3215. https://doi.org/10.3390/ijerph17093215

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. Wang F, Xie Z, Pei Z, Liu D. Emergency Relief Chain for Natural Disaster Response Based on Government-Enterprise Coordination. Int J Environ Res Public Health. 2022 Sep 7;19(18):11255. https://doi.org/10.3390/ijerph191811255. PMID: 36141522; PMCID: PMC9517505.

  42. Makivić I, Švab V, Selak Å. Mental Health needs Assessment during the COVID-19 pandemic: Consensus based on Delphi Study. Front Public Health. 2021;9(2):732539.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Li L, Sun N, Fei S, Yu L, Chen S, Yang S, et al. Current status of and factors influencing anxiety and depression in front-line medical staff supporting Wuhan in containing the novel coronavirus pneumonia epidemic. Jpn J Nurs Sci JJNS. 2021 Apr;18(2):e12398.

  44. Kang L, Li Y, Hu S, Chen M, Yang C, Yang BX, et al. The mental health of medical workers in Wuhan, China dealing with the 2019 novel coronavirus. Lancet Psychiatry. 2020 Mar;7(3):E14–4.

  45. Ma T, Guo B, Xu J. The persistence and change in China’s Healthcare Insurance Reform: clues from fiscal subsidy policies made for settling COVID-19 patients’ medical costs. Risk Manag Healthc Policy. 2022 May;24:15:1129–44.

  46. Stoddard M, Sarkar S, Yuan L, Nolan RP, White DE, White LF, et al. Beyond the new normal: assessing the feasibility of vaccine-based suppression of SARS-CoV-2. PLoS ONE. 2021;16(7):e0254734. https://doi.org/10.1371/journal.pone.0254734

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. WHO Coronavirus (COVID-19). Dashboard World Health Organization.https://covid19.who.int,Accessed 20 Nov 2022.

  48. Mofijur M, Fattah IMR, Alam MA, Islam ABMS, Ong HC, et al. Impact of COVID-19 on the social, economic, environmental and energy domains: Lessons learnt froma global pandemic. Sustain Prod Consum. 2021 Apr;26:343–59. https://doi.org/10.1016/j.spc.2020.10.016

  49. Kubás J, Bugánová K, Polorecká M, Petrlová K, Stolínová A. Citizens’ Preparedness to Deal with Emergencies as an Important Component of Civil Protection. Int J Environ Res Public Health. 2022 Jan 12;19(2):830. https://doi.org/10.3390/ijerph19020830

  50. Bodas M, Peleg K, Stolero N, Adini B. Understanding societal resilience-cross-sectional study in eight countries. Front Public Health. 2022;10:883281.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Xue L, Cao X, Wan H. Releasing Wolbachia-infected mosquitos to mitigate the transmission of Zika virus. J Math Anal Appl. 2021;496(1):124804. https://doi.org/10.1016/j.jmaa.2020.124804

    Article  Google Scholar 

  52. Ablah E, Konda K, Kelley CL. Factors predicting individual emergency preparedness: a multi-state analysis of 2006 BRFSS data. Biosecur Bioterror. 2009 Sep;7(3):317 – 30. https://doi.org/10.1089/bsp.2009.0022. PMID: 19821751.

  53. Bethel JW, Foreman AN, Burke SC et al. Disaster prepared-ness among medically vulnerable populations. Am J Prev Med. 2011;(2): 139–43.

  54. Murphy ST, Cody M, Frank LB et al. Predictors of Emergency Preparedness and Compliance. Disaster Med Public Health Prep. 2009 Jul 8. https://doi.org/10.1097/DMP.0b013e3181a9c6c5. Epub ahead of print. PMID: 19590429.

  55. Diekman ST, Kearney SP, O’Neill ME, et al. Qualitative study of homeowners’ emergency preparedness: experiences, perceptions, and practices. Prehosp Disaster Med. 2007;22(6):494–501.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We sincerely thank all the experts who participated in this study.

Funding

This research was supported by the Sichuan Science and Technology Program (Grant No.2021JDKP0102), the Sichuan Social Science Association Project (Grant No.SC21JD016), the Project of Southwest Medical University (Grant No.2022YB014), and the Zigong Health and Humanities Research Center Project (Grant No.JKRWY22-24).

Author information

Author notes

  1. Wei Wei, Yubei Liu and Na Zhou contributed equally.

Authors and Affiliations

  1. School of Nursing, Southwest Medical University, 1 Xianglin Road, Luzhou, China

    Wei Wei, Yubei Liu, Na Zhou, Yanhua Chen & Weili Hu

  2. Department of Orthopedics, The Affiliated Hospital of Southwest Medical University, 25 Taiping Street, Luzhou, China

    Min Tian

  3. Department of Nephrology, The Affiliated Hospital of Southwest Medical University, 25 Taiping Street, Luzhou, China

    Longsheng Xie

  4. Health and Social Care Faculty, University of Hull, Cottingham Road, Hull, HU6 7RX, USA

    Roger Watson

  5. Department of Science and Technology, Southwest Medical University, 1 Xianglin Road, Luzhou, China

    Fengling Dai

  6. Department of Nursing, The Affiliated Hospital of Southwest Medical University, 25 Taiping Street, Luzhou, China

    Yanhua Chen

  7. College of Humanities and Management, Southwest Medical University, 1 Xianglin Road, Luzhou, China

    Weili Hu

Authors
  1. Wei Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yubei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Na Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Min Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Longsheng Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Roger Watson
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Fengling Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yanhua Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Weili Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Study conception and design: Wei Wei, Weili Hu, Yanhua Chen. Data collection:Min Tian, Yubei Liu. Data analysis: Longsheng Xie, Na Zhou, Fengling Dai. Study supervision: Weili Hu. Manuscript writing: Wei Wei. Critical revision of the article: Yanhua Chen, Roger Watson, Weili Hu. The author(s) read and approved the final manuscript.

Corresponding authors

Correspondence to Yanhua Chen or Weili Hu.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the Ethics Committee of the Affiliated Hospital of Southwest Medical University, China (NO. KY2022345 ). The study was performed in accordance with the Declaration of Helsinki. Informed consent was obtained from all participants included in the study.

Consent for publication

Not Applicable.

Competing interests

Authors report no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary Material 2

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wei, W., Liu, Y., Zhou, N. et al. Constructing an emergency preparedness evaluation index system for public use during major emerging infectious disease outbreaks: a Delphi study. BMC Public Health 23, 1109 (2023). https://doi.org/10.1186/s12889-023-15980-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12889-023-15980-6

Keywords

BMC Public Health

ISSN: 1471-2458

Contact us