This study addresses the function of Yemen’s electronic diseases early warning system (eDEWS), as a national disease surveillance system, from 2013 to 2017, during an on-going conflict. Disease surveillance is an important component of public health for tracking potential epidemics, monitoring interventions, and informing health policy [14]. This study is one of the few performed in Yemen and the Eastern Mediterranean Region to assess an eDEWS, based on CDC standard indicators and identify the system’s usefulness during Yemen’s ongoing complex emergency.
Key findings showed that eDEWS is a resilient and useful system, and despite the conflict, the system is still functioning. Data quality and response timeliness were somewhat problematic, since only 21% of all eDEWS alerts were verified in the first 24 h of detection in 2016. However, these gaps did not affect the system’s ability to identify outbreaks in the current fragile situation. This study’s findings show that eDEWS data is representative.
Data completeness remains a significant challenge for many national surveillance programs [15], however, the high level of completeness in eDEWS is ensured due to a mobile software electronic data collection process, and the system’s validation function that ensures submission of complete reports by health facilities. Global evidence shows that the use of electronic reporting systems contributes to good data quality in terms of availability, timeliness, reliability, and completeness [16]. The high reporting rate in eDEWS reflects the data completeness and system acceptability for all health staff and partners involved in the system, this findings are in agreement with the findings of similar evaluations conducted in Sana’a Governorate [17].
Despite the high rate of report completeness in eDEWS, manual data management and analysis poses significant risk to data accuracy in generated epidemiological reports, which may affect data usage in decision making [18]. Manually managing and analysing data is time intensive, increases workload, and poses significant risk of human error in data compilation and analysis. The shortage in laboratory confirmation and dependence on case definitions for diagnosis is one of the challenges affecting the quality of data, particularly in an emergency as seen in other countries with similar circumstances [19], which may compromise data quality and the accuracy of disease surveillance data used by decision-makers. We recommend switching from manual to automated data analysis processes within the existing online database system. This approach can drastically decrease dependency on manual methods and help to avoid errors and delays [20].
Immediate public health action is always required in public health surveillance following the effective reporting of health facility information. Mayad et al. (2019) argue for the perfect timeliness of eDEWS at 100% [17]. However, our study shows that only 21% of alert responses for diseases requiring immediate action occurred within the requisite 24-h window (out of a total of 3721 such alerts), thus highlighting a gap in the response timeliness. A response delay during outbreaks increases the burden of morbidity and mortality [20]. For example, in 2016, only 31% of the cholera cases received a response within the first 24 h of the eDEWS alert notification [21]. The delays in verification of data has a substantial effect on the detection process. As another example, during the cholera outbreak in 2016, there were many alerts of acute watery diarrhoea in Al Baidha governorate several weeks before declaring the outbreak in Sana’a, however, these alerts were not verified in a timely manner, and thus early warning of the possible spread of the disease was not delivered [21]. Response timeliness remains a problem even in many higher-income countries, e.g., in the USA, a study found a significant difference in response delay times compared to the standard immediate response time for Category II vaccine-preventable diseases in West Virginia [22].
In Yemen, where public health efforts are often implemented by non-governmental partners, delays in dissemination of weekly information may be one reason for delayed partner intervention (especially in water, sanitation and hygiene [WASH] interventions), thus reducing surveillance usefulness due to a missing link between data collection and public health action [23]. This study revealed that dissemination delays increased over time from 2.8 days in 2014 to 9.0 days in 2016 and 2017, the eDEWS was expanded quickly from 100 health facilities in 2013 to 1982 in 2017, thus data processing was affected by the overwhelming amount of data received each week. All key informants interviewed in this study confirmed the delay in the dissemination of the weekly eDEWS bulletin. In Syria, the average delay for publishing information was 24 days for the Early Warning and Response System (EWARS) based in Damascus, while in Turkey, the average delay was 11 days for Early Warning and Response Networks (EWARNs) [24].
Identifying the barriers and challenges facing a surveillance system is a critical step for improving performance; our study revealed timeliness as a particularly chronic issue with eDEWS in Yemen [25]. Rapid staff turnover, the security situation, limited resources for alert response, health staff motivation, refresher training needs, limited technical capacities, logistics, issues of internet connectivity and a lack of financial resources all likely contribute to poor timeliness. Despite all these challenges, eDEWS nevertheless demonstrated its appropriateness even during the conflict and its on-going utility for detecting new emerging outbreaks. Cordes KM et. al recommends at-risk countries to invest in such systems, as early warning, alert, and response networks (referred to by WHO as EWARN) have been useful sources of information where no other data were available during many emergencies [26].
A Positive Predictive Value (PPV) reflects the ability of the system to detect true outbreak. Having low false positive alerts, especially in 2015 and 2017 in Yemen, reflected the program’s effectiveness in detecting outbreaks. Therefore, detected outbreaks were generally true with an average PPV of more than 95% (range 95–100%). This in line with a systematic review study comparing an electronic surveillance system with a paper surveillance method that showed that electronic surveillance has moderate to excellent utility compared with conventional surveillance methods [27]. A low PPV for a surveillance system leads to wasted resources and time due to an unnecessary investigation of every reported case [28]. The eDEWS is the only system capturing data on epidemic prone diseases during the humanitarian emergency of Yemen with lack of laboratories, therefore, sensitivity is difficult to assess [29].
Data obtained from the system shows that eDEWS is able to detect changes over time since data can be supported by field investigation and laboratory testing. eDEWS is very useful in monitoring disease trends in Yemen’s current situation. This study shows that Yemen’s eDEWS is a reliable surveillance system with the possibility of contributing to the timely detection and monitoring of diseases. For example, in 2016, eDEWS monitored the trends of dengue fever cases in the system on a weekly basis, and there were a total of five confirmed dengue outbreaks (Aden, Lahj, Mareb, Hajjah and Al-Hodeida). The eDEWS was useful in locating outbreaks in unusual geographic locations, for example, cholera and dengue fever were reported for first time in Sanaa in 2016 [13]. Even when a public health surveillance system has low sensitivity, it can still be useful in trend monitoring as long as the sensitivity remains reasonably constant and change is notable [28].
Population representation in any surveillance system is influenced by access to the health facilities as well as sex and age groups [30]. In Yemen, eDEWS data are regularly used to provide national estimates of the incidence and prevalence of infectious diseases and guidance for required interventions. The eDEWS is used by only 37% of all health facilities in the country, however, this represents 83% of all functional health facilities [31]. All age groups are represented in eDEWS data; and one-third of the patients were between 15 and 44 years. Approximately 53% of the registered patients in the health facilities were women, a similar finding was reported in a study on the representativeness of an online nationwide surveillance system for influenza in France [32].
Measuring eDEWS’s usefulness and acceptability is the main attribute of an evaluation to demonstrate functionality and ensure the system’s sustainability [33]. Acceptability is a cross-cutting measure of surveillance usefulness. It can be measured by several indicators such as the percentage of reporting, completeness and responses by surveillance staff and relevant stakeholders. Results show that various partners are supporting eDEWS in the field, and many donors trust the system to identify new emerging outbreak in the country. The evidence showed that increasing the health staff and field health partners’ transparency and knowledge of the system’s processes will increase the surveillance system’s accessibility [34]. Many key informants did not agree that eDEWS is a flexible system since they believed that eDEWS needs more time to achieve change. However, flexibility is not a matter of time, but rather the ability to adapt to changes in risks and information input [33].
The major limitation of the study revolves around the difficulty to get full data for the period from 2013 to 2017, we used eDEWS data predominantly extracted from the epidemiological bulletins. Moreover, we found only three annual reports of eDEWS, and the annual reports of 2015 and 2017 were not produced by the program. It was difficult to extract full data on alerts from the program dashboard for all targeted years, and the only available full data was for 2016. For consistency, all incomplete data on alerts was excluded. Due to the lack of data on routine surveillance, it was impossible to compare eDEWS data with other Yemeni routine surveillance data sources for better assessment of the quality of eDEWS information.
In emergency situation as in Yemen, it is difficult to have complete data and it is difficult to do laboratory or field investigation for all alerts. Therefore, we tried to use the available data in the system to identify the PPV for the system as whole. The sensitivity of the eDEWS system could not be determined exactly on the basis of available data. Therefore, this paper focused on the application of case definitions to determine whether there have been any factual changes.