Where is the evidence for emergency planning: a scoping review

Background Recent terrorist attacks and natural disasters have led to an increased awareness of the importance of emergency planning. However, the extent to which emergency planners can access or use evidence remains unclear. The aim of this study was to identify, analyse and assess the location, source and quality of emergency planning publications in the academic and UK grey literature. Methods We conducted a scoping review, using as data sources for academic literature Embase, Medline, Medline in Process, Psychinfo, Biosis, Science Citation Index, Cinahl, Cochrane library and Clinicaltrials.gov. For grey literature identification we used databases at the Health Protection Agency, NHS Evidence, British Association of Immediate Care Schemes, Emergency Planning College and the Health and Safety Executive, and the websites of UK Department of Health Emergency Planning Division and UK Resilience. Aggregative synthesis was used to analyse papers and documents against a framework based on a modified FEMA Emergency Planning cycle. Results Of 2736 titles identified from the academic literature, 1603 were relevant. 45% were from North America, 27% were commentaries or editorials and 22% were event reports. Of 192 documents from the grey literature, 97 were relevant. 76% of these were event reports. The majority of documents addressed emergency planning and response. Very few documents related to hazard analysis, mitigation or capability assessment. Conclusions Although a large body of literature exists, its validity and generalisability is unclear There is little evidence that this potential evidence base has been exploited through synthesis to inform policy and practice. The type and structure of evidence that would be of most value of emergency planners and policymakers has yet to be identified.


Background
An effective and efficient emergency response can reduce avoidable mortality and morbidity after a mass casualty incident. Numberous guidance documents have been issued since the Civil Contingencies Act placed a legal responsibility for emergency planning with NHS organisations [1][2][3]. Following the 7 th July 2005 London bombings Lady Justice Hallett's Rule 43 report raised concerns about the capacity of the London Ambulance service to plan communications for a major incident, the ability to establish a Gold Control Room and the process of triage [4]. Similar comments about the mismatch between plans and events followed the 9/11 World Trade Center attacks [5].
The process for developing policy and guidance documents is variable with the strength and quality of the underlying evidence base often proving unclear. For example the UK Emergency Preparedness document gives no information on the provenance of its evidence base [6]. The review team was commissioned by the UK National Institute of Health Research to conduct a scoping review of the academic and UK grey literature to identify the location, source and quality of emergency planning publications. This would allow us to characterise the evidence that exists to inform emergency planning.

Academic literature identification
Following compilation by the authors of themes and topics considered relevant to the field, pilot searching was carried out in the subarea of health services business continuity. This is an accepted stage for all scoping reviews as it seeks to establish an optimal balance between sensitivity and specificity. This is important conceptually, to finalise the scope of the review, and pragmatically, to ensure that the review is feasible within the available time and resources [7]. A final search strategy was then developed to retrieve evidence relevant to the whole subject area, subdivided into Business continuity, Hazard analysis, Capability assessment and maintenance, Recovery, Communications/informatics and Organisational behaviour/Human Resources. This final search aimed to identify slices of the evidence, but was designed to be more focussed, producing a higher yield of relevant papers and was therefore more time effective to review. The information specialist searched the electronic databases Embase, Medline, Medline in Process and Psychinfo (via Ovid SP), Biosis and Science Citation Index (via Web of Science), Cinahl (via EBSCO), the Cochrane library (via Wiley) and Clinicaltrials.gov (Table 1). Searches were conducted in November 2010 with searches covering the period January 1990 to October 2010. We did not limit to "human" as we wished to identify literature on bioterrorism and zoonoses that might be relevant.
Extracted titles and abstracts were screened by KC, AL, PG and AB. Each title or abstract was reviewed and deemed to be relevant, equivocal (subject matter suggestive of relevance to emergency planning and/or management), not relevant or containing inadequate information for coding (limited PRISMA chart figure 1). To operationalise the brief from the commissioners of the study to prioritise research relevant to UK health emergency planning we sought to include literature relating to comparable health services. We therefore excluded articles relating to non-health emergency planning, nonemergency planning, and non-UK legislative issues, and those from low-and middle-income countries unless they were likely to be generalisable to the UK. Two subsets of two hundred references were coded by a pair of researchers (either KC/PG or AL/AB) and kappa values for agreement calculated to examine consistency.

Grey literature identification
We aimed to identify grey literature that included primary data or analysis that could be used to inform decisionmaking. Potential sources of grey literature were identified in advance by our expert advisory group and supplemented by the academic literature review and concurrent stakeholder interviews (reported elsewhere). Based on this,

Literature analysis and synthesis
Where the title and abstract were considered to be relevant or equivocal we extracted further information relating to country of origin, type of publication and type of event discussed. Publications were coded on an Microsoft Excel spreadsheet using aggregative synthesis, a technique appropriate for exploring qualitative data where the concepts are secure, predefined and not contested [8]. We used a thematic framework developed a priori from the FEMA Emergency Management Cycle ( Figure 2) [9]. This framework is widely known and used within the emergency planning community, and covers the range of issues in the field using mutually exclusive concepts. Framework synthesis can be used in conjunction with either aggregative or interpretive reviews, has been shown to facilitate more rapid coding of the literature [10] and is, therefore, particularly suited to the objectives of a scoping review.

Academic literature review
We reviewed 2940 titles and abstracts. Of these, 625 were coded by a pair of reviewers and with kappa values of 0.578 (AL/AB) and 0.740 (KC/PG). 1545 publications were felt to be relevant or equivocal. The country, type of publication and type of event are shown in Tables 2   Table 1 Search strategies (Continued) Organisational behaviour 1. Disasters/pc 2. (emergency response or emergency preparedness or emergency plan$ or emergency operation plan$ or disaster or major incident$ or incident plan$).ti,ab.
3. 1 or 2 4. (community engagement or community involvement or participatory involvement or participatory engagement or consumer participation or organi?ational behavio?r or health personnel or human resources).ti,ab.   [11]. EM-DAT is a multi-source validated database of disasters where 10 or more people are killed, 100 or more affected, a state of emergency is declared or a call for international assistance is made. As we deliberately excluded much of the literature from low-and middle-income countries, percentages shown relate to EM-DAT records for high-income countries. This shows that the US and Canada are relatively over-represented in the literature, while continental Europe, Australasia and Japan are relatively under-represented. Themes addressed in each publication are shown in Table 4. Table 5 shows the thematic analysis of publications in the event report, narrative review, systematic review, modelling, survey and RCT categories. These demonstrate a predominance of publications (over half ) relating to emergency planning and emergency response, with relatively few addressing mitigation, with 219 of 362 event reports describing the emergency response.

Grey literature review
192 documents were initially identified, of which 97 were relevant. Of these, 52 were published by the Health Protection Agency. Table 3 shows the type of publication and type of event. The preponderance of publications relating to CBRN probably reflects the accessibility of Health Protection Agency reports. As in the academic literature, the majority of documents addressed emergency response, although there was significant discussion of organisational issues and communications.

Discussion
This scoping review demonstrates that the published literature relating to health emergency planning is disproportionately centred on North America; Australasia and Europe have produced surprisingly little academic literature given the number of reported incidents in these areas, although non-English publications from Europe were excluded from the scope of this review. The proliferation of emergency planning research in the US may represent the effect of widespread federal funding following the 9/11 attacks. To our knowledge this represents the first attempt to scope the emergency planning literature relevant to a UK setting. This scoping review used a deliberately  inclusive strategy and therefore our findings may overestimate the extent of useable evidence. Equally, we did not assess the methodological quality of individual publications beyond their stated design. It is unclear whether the traditional hierarchy of evidence applied to clinical research (privileging meta-analyses and randomised controlled trials) [12] is appropriate in this field. Except in very specific areas (for example brief psychological interventions for survivors) randomised controlled trial designs are not practical. Narrative synthesis of observational studies may be possible [13] but is currently hampered by the unevenness of collection. Repeated calls have been made to standardise major incident reporting and suggestions exist for the format this should take [14,15]. Even though multiple event reports exist there is little indication that this potential evidence base has been fully exploited through synthesis. We found it difficult to identify a repository of emergency planning literature that was easily and openly accessible. The two main repositories were the HPA's Chemical Incidents and Poisons Report database and the Emergency Planning College library. Access to the Emergency Planning College library at Easingwold is controlled and their collection is not open to the public. We have identified a substantial amount of literature relating to emergency response. However, the validity and generalisability of the data is unclear, as is its amenability to synthesis and how this should inform policy and practice. We have also exposed the limited evidence base available to assist emergency planners and policymakers in the areas of mitigation and recovery. The development of an evidence base for emergency planning needs to take into account practical considerations regarding what type of evidence is most likely to be influential, as well as theoretical considerations of what constitutes valid evidence. The type and structure of evidence that would be of most value of emergency planners and policymakers has yet to be identified. Further research into stakeholder perceptions will be required to determine this.

Conclusions
A significant body of emergency planning literature exists. Nevertheless its usefulness to UK planners is constrained because it is dominated geographically by the US, thematically by emergency response topics and methodologically by event reports.