Skip to main content
Download PDF

Development and assessment of a hospital admissions-based syndromic surveillance system for COVID-19 in Ontario, Canada: ACES Pandemic Tracker

BMC Public Health volume 21, Article number: 1230 (2021) Cite this article

Abstract

Background

The COVID-19 pandemic has continued to pose a major global public health risk. The importance of public health surveillance systems to monitor the spread and impact of COVID-19 has been well demonstrated. The purpose of this study was to describe the development and effectiveness of a real-time public health syndromic surveillance system (ACES Pandemic Tracker) as an early warning system and to provide situational awareness in response to the COVID-19 pandemic in Ontario, Canada.

Methods

We used hospital admissions data from the Acute Care Enhanced Surveillance (ACES) system to collect data on pre-defined groupings of symptoms (syndromes of interest; SOI) that may be related to COVID-19 from 131 hospitals across Ontario. To evaluate which SOI for suspected COVID-19 admissions were best correlated with laboratory confirmed admissions, laboratory confirmed COVID-19 hospital admissions data were collected from the Ontario Ministry of Health. Correlations and time-series lag analysis between suspected and confirmed COVID-19 hospital admissions were calculated. Data used for analyses covered the period between March 1, 2020 and September 21, 2020.

Results

Between March 1, 2020 and September 21, 2020, ACES Pandemic Tracker identified 22,075 suspected COVID-19 hospital admissions (150 per 100,000 population) in Ontario. After correlation analysis, we found laboratory-confirmed hospital admissions for COVID-19 were strongly and significantly correlated with suspected COVID-19 hospital admissions when SOI were included (Spearman’s rho = 0.617) and suspected COVID-19 admissions when SOI were excluded (Spearman’s rho = 0.867). Weak to moderate significant correlations were found among individual SOI. Laboratory confirmed COVID-19 hospital admissions lagged in reporting by 3 days compared with suspected COVID-19 admissions when SOI were excluded.

Conclusions

Our results demonstrate the utility of a hospital admissions syndromic surveillance system to monitor and identify potential surges in severe COVID-19 infection within the community in a timely manner and provide situational awareness to inform preventive and preparatory health interventions.

Peer Review reports

Background

The COVID-19 pandemic continues to pose a major public health risk globally. As of May 2021, more than 157 million confirmed cases of COVID-19, including over 3.2 million (2.1%) deaths, have been reported to the World Health Organization since December 2019 [1]. In Ontario, Canada’s most populous province, there have been more than 486,200 confirmed COVID-19 cases, including over 23,500 (4.8%) hospitalizations and over 8200 (1.7%) deaths as of May 2021 [2].

The importance of public health surveillance systems to monitor the spread and impact of disease within the population has been well demonstrated during the COVID-19 pandemic [3,4,5,6]. Public health surveillance systems have the capability to serve as early warning systems and provide situational awareness during public health emergencies, including communicable disease outbreaks, natural disasters and bioterrorism, among others [7, 8]. Information from public health surveillance systems can also provide scientific evidence essential to public health decision-making and control measures. Additionally, public health surveillance can guide health-related policy development, including disease prevention and risk mitigation strategies, and contribute to epidemiologic understanding of various communicable and non-communicable diseases [7, 8].

Real-time syndromic surveillance (SyS) is one type of public health surveillance tool that uses pre-diagnostic health indicators discernable before diagnostic confirmation as an alert of changes in disease activity in the population [9]. For example, SyS systems can rely on signs, symptoms or preliminary diagnoses to monitor disease activity within the population and apply statistical methods to indicate a potential outbreak or other public health concern. Real-time SyS systems can facilitate rapid investigation of potential disease outbreaks or elevations in specific disease-related illnesses [8, 9]. Data sources for large scale regional SyS systems often include health or pseudo-health information from healthcare services such as emergency department (ED) [10, 11] or primary care visits [12,13,14], but could also include information from retail pharmaceutical sales [15, 16], emergency medical services dispatch data [17,18,19,20,21], telehealth phone lines [22, 23] and non-health information from social media [6, 24]. The indirect nature of health-related indicators, although nonspecific [25], have the ability to improve timeliness and sensitivity of data collection and provide flexibility to adapt to different illnesses and situations [7, 26].

The significance of real-time SyS surveillance has been demonstrated with previous threats to global health (i.e. bioterrorism, pandemics) such as SARS, H1N1 influenza and Ebola [22, 27,28,29,30,31,32,33,34,35]. Most recently, several SyS systems have been implemented globally to help monitor COVID-19 including the Centers for Disease Control and Prevention’s COVID Tracker [36] and a smartphone application to gather COVID-like illness data to identify potential clusters of COVID-19 in the United States [37], COOPERA (Covid-19: Operation for Personalized Empowerment to Render smart prevention And care seeking) which collects crowdsourced data in Japan [38, 39] and a participatory SyS tool for tracking COVID-19 using self-report syndromic data in Bangladesh [40], among others.

In response to the pandemic threat of SARS in 2003, Kingston, Frontenac and Lennox & Addington (KFL&A) Public Health developed the Acute Care Enhanced Surveillance (ACES) system in Ontario, Canada to improve the provincial public health system’s ability to monitor, prevent and respond to future communicable and non-communicable disease outbreaks [41]. ACES is a province-wide system that monitors hospital registration records for ED visits in real-time for more than 95% of Ontario’s acute care hospitals and nearly 80% of inpatient admissions records. ACES monitors 84 distinct syndromes; 24 are regularly validated against ICD-10 codes.

The objective of this study was to describe the development and effectiveness of a real-time public health SyS system – ACES Pandemic Tracker – as an early warning system and to provide situational awareness in response to the COVID-19 pandemic in Ontario, Canada using hospital admissions data.

Methods

Development of ACES pandemic tracker

Data sources

ACES Pandemic Tracker [42] uses hospital admissions data from the ACES system [41]. ACES collects patient data in real-time including date, time, age, sex, first 5 digits of postal code, reason for visit or admission and Canadian Triage Acuity Score; no direct personal identifiers are collected, such as name or health card number.

ACES applies natural language processing (NLP) to categorize words or phrases from the “reason for admission” or “chief complaint” into predefined syndromes. The use of chief-complaints, syndromes or symptoms collected as text have been cited as an advantage for public health surveillance since they are available more closely to real-time than diagnostic indicators such as ICD codes [43,44,45]. The NLP algorithms were developed by a team of content experts (acute care physicians and epidemiologists) that manually classified a large dataset of patient triage records into syndromes based on their chief complaint. The algorithms do not rely on keyword searches, but rather probabilistic decisions based on attaching learned weighting values to each word, part of a word, or phrase in the chief complaint. The performance of each syndrome was previously validated against diagnostic hospital records.

Syndromes of interest (SOI)

Syndromes are pre-defined groupings of symptoms or health indicators that may indicate a clinical diagnosis or health outcome. The following syndromes were initially used to detect possible cases of COVID-19 at time of hospital admission: asthma (AST), congestive heart failure (CHF), chronic obstructive pulmonary disorder (COPD), influenza-like illness (ILI), general infection (INF), pneumonia (PN), respiratory illness (RESP) and sepsis (SEP). These syndromes were chosen as they encompassed many of the symptoms described in patients that had contracted COVID-19 at the start of the pandemic. Aggregate numbers of admissions for all SOI and overall admission trends were examined to assess the specificity of syndromes and remove those experiencing significant decreases in hospital admissions related to the COVID-19 pandemic. After correlation analysis with laboratory confirmed cases, several aforementioned syndromes were removed as they were complicating the interpretation of COVID-19 surveillance data [46]. Thus, the most recent version of ACES Pandemic Tracker, as discussed in this paper, includes three SOI: pneumonia, influenza-like illness and general infection (see Additional file 1 for descriptions of symptoms captured in each SOI including ICD-10 codes used for validation).

Possible cases of COVID-19 were flagged as “suspect COVID-19 counts” identified by COVID-19 keywords (“covid*” OR “coronav*” OR “ncov”) and included “susp” or SOI. Suspected COVID-19 cases identified by both COVID-19 keywords and SOI are hereinafter referred to as suspected COVID-19 (including SOI), whereas suspected COVID-19 cases identified using only COVID-19 keywords are referred to as suspected COVID-19 (excluding SOI). Each patient was only counted once, even if they had more than one related syndrome (see Additional file 2 for full COVID-19 flagging criteria). All free-text admissions were reviewed bi-weekly to determine how COVID-related admissions were being captured in order to adapt to changing free-text patterns over time. Data presented in this manuscript are based on the final SOI inclusion criteria. The change in case numbers flagged due to bi-weekly review updates was minimal and likely to have a negligible impact on case numbers flagged in real-time as opposed to post-event analysis. The total number of admissions for all SOI were calculated daily.

Baseline data

We used 2 years of historical hospital admissions data (2018 and 2019) to compare recent trends of hospital admissions at the regional-level to what would be expected under normal conditions. Since there is no historical data for COVID-19 flagged admissions, these admissions were grouped with all admissions for SOI in 2020 and compared to 2018–2019 SOI. For each pandemic day, the average number of admissions for the same pre-pandemic day in 2018 and 2019 were used for comparison. In order to reduce the risk of increased baseline data counts due to chance, 7- and 30-day moving averages were used [47]. We calculated historical 7- and 30-day moving averages by adding together syndromes related to COVID-19 during the 2018–2019 period to minimize short-term fluctuations and remove day-of-the-week/month variation. To account for random variation in comparisons with historical data, we calculated + 1 and + 2 standard deviation(s) from the historical average. Hospitals without 2 years of complete baseline admissions data for 2018 and 2019 were excluded from ACES Pandemic Tracker.

SOI related to COVID-19 are monitored for abnormal numbers of patients in comparison to historical baselines. A signal for possible anomalous regional hospital activity occurs when the pandemic moving average of hospital admissions counts is greater than + 1 standard deviation above the historical average, but less than or equal to + 2 standard deviations (Level 2). Highly anomalous activity is indicated when the pandemic moving average is greater than + 2 standard deviations above the historical pre-pandemic average (Level 3).

Evaluation of ACES pandemic tracker

Descriptive analyses

We performed descriptive analyses using data from ACES Pandemic Tracker to calculate the median, interquartile range and incidence rate of daily suspected COVID-19 hospital admissions. We compared these data to confirmed COVID-19 admissions using data available from the Ontario Ministry of Health [48]. The date recorded for a confirmed COVID-19 admission was the date of reporting to public health following a confirmed laboratory test.

We used Spearman’s rho to examine correlations between confirmed and suspected COVID-19 admissions in Ontario between March 1, 2020 and September 21, 2020. We also compared each SOI with confirmed COVID-19 admissions to determine which SOIs demonstrated more favorable correlations with confirmed COVID-19 admissions.

In an effort to detect the potential delay between SyS flags for suspected COVID-19 admissions and confirmed COVID-19 admissions in Ontario, a cross-correlation function (CCF) with differencing was calculated between daily suspected COVID-19 admissions and daily confirmed COVID-19 admissions. On March 19, 2020, the Ontario Ministry of Health disseminated a province-wide directive for the use of suspected COVID-19 flags to improve monitoring of ED visits and hospital admissions for symptoms related to COVID-19. Therefore, we used data between March 19, 2020 and May 31, 2020 (end of first wave/peak) to calculate the CCF. Statistical analyses were conducted using R (R Foundation for Statistical Computing, Vienna, Austria). All analyses were conducted with aggregate data in accordance with the ethical and legal limitations defined by the data sharing agreements between ACES, public health agencies and the participating hospitals in Ontario and is exempt from Research Ethics Board review.

Results

Key characteristics

Between March 1, 2020 and September 21, 2020, ACES Pandemic Tracker identified 22,075 suspected COVID-19 hospital admissions, or 150 suspected COVID-19 admissions per 100,000 population among 131 hospitals in Ontario (Table 1). Of these suspected COVID-19 admissions, 48% were flagged as pneumonia, 21% as influenza-like illness, 20% as suspected COVID-19 (excluding SOI) and 10% as general infection. During the same time period, the province registered 47,268 confirmed cases of COVID-19, including 5047 COVID-19-related admissions, or 34 confirmed COVID-19 admissions per 100,000 population. A comparison of 7-day rolling averages of SOI admissions and confirmed COVID-19 cases is illustrated in Fig. 1.

Table 1 Counts for confirmed and suspected COVID-19 hospital admissions between March 1, 2020 and September 21, 2020
Full size table
Fig. 1
figure 1

Trends in 7-day averages among suspected COVID-19 hospital admissions and confirmed COVID-19 cases between March 1, 2020 and September 21, 2020 in Ontario, Canada. ILI Influenza-like illness, INF General infection, PN Pneumonia, SOI Syndromes of interest. Note: Total confirmed cases (dotted-line) refers to the y-axis on the right

Full size image

ACES Pandemic Tracker identified a median number of 102 (IQR 91–124) daily suspected COVID-19-related admissions (including SOI) in Ontario, including 49 daily suspected admissions for pneumonia (IQR 44–66), 22 for influenza-like illness (IQR 19–28), 16 for suspected COVID-19 (excluding SOI; IQR 10–31) and 11 for general infection (IQR 8–14; Table 1). The median number of confirmed COVID-19 admissions in Ontario during the same time period was 9 per day (IQR 5–37).

Correlation analyses

We sought to determine whether suspected COVID-19 hospital admissions from ACES Pandemic Tracker were correlated with confirmed COVID-19 admissions from provincial data. We also calculated correlation coefficients for each SOI and suspected COVID-19 flag to determine which were more highly correlated with confirmed COVID-19 admissions.

Table 2 displays correlations between suspected and confirmed COVID-19 hospital admissions, including and excluding SOIs. We found suspected COVID-19 (including SOI) admissions to have a strong positive correlation with confirmed COVID-19 admissions (Spearman’s rho = 0.617). When SOI without COVID keywords were excluded, a stronger positive correlation was observed between suspected and confirmed COVID-19 admissions (Spearman’s rho = 0.867).

Table 2 Correlations between suspected COVID-19 hospital admissions/SOIs and confirmed COVID-19 admissions between March 1, 2020 and September 21, 2020
Full size table

Since suspected COVID-19 (excluding SOI) hospital admissions were most strongly correlated with confirmed COVID-19 admissions, we conducted a cross-correlation analysis to determine whether suspected COVID-19 (excluding SOI) admissions preceded confirmed COVID-19 admissions in Ontario, Canada. Our findings suggest confirmed COVID-19 admissions lagged suspected COVID-19 admissions by 3 days (CCF = 0.320). The time-series lag analysis is illustrated in Fig. 2.

Fig. 2
figure 2

Time series lag analysis of suspected COVID-19 admissions compared with confirmed COVID-19 admissions in Ontario. The blue dotted lines represent 95% confidence intervals (CI) for lack of trend (0.229). Values that exceed these lines represent time-lagged trends. A CCF greater than the CI indicates a probable connection between the timing of suspected COVID-19 (excluding SOI) admissions and confirmed COVID-19 admissions. A lag of − 3 indicates that confirmed COVID-19 admissions lagged suspected COVID-19 admissions by 3 days. CCF values slightly greater than the CI at − 7 and 7 days are likely due to regular weekly fluctuations in hospital usage and acute/care patterns

Full size image

Discussion

This study described and evaluated the utility of ACES Pandemic Tracker, a province-wide real-time public health SyS tool. ACES Pandemic Tracker was developed to detect possible surges in severe COVID-19 infection requiring hospitalization and to provide situational awareness in Ontario, Canada. ACES Pandemic tracker uses hospital admissions data from 131 hospitals across the province to monitor syndromes related to COVID-19 with built-in spatial and temporal capabilities. Ontario has an estimated population of 14.7 million people; 39% of Canada’s population [49].

We found a strong correlation between suspected COVID-19 (including SOI) hospital admissions and confirmed COVID-19 admissions and a stronger correlation between suspected COVID-19 (excluding SOI) admissions and confirmed COVID-19 admissions. Although SOI can be valuable for identification of an unexpected disease outbreak, our findings suggest that suspected COVID-19 flags (excluding SOI) may be a more appropriate proxy for identifying COVID-19 infection when it is confirmed to be circulating within the population. These findings demonstrate that care should be taken when deciding whether to use pre-existing syndromes versus developing new flagging criteria for a novel disease outbreak. When SOI were assessed individually against confirmed COVID-19 admissions, we found a moderate positive correlation with suspected general infection admissions, a moderate negative correlation with suspected influenza-like illness admissions and a weak correlation with suspected pneumonia admissions. One possible explanation for the negative correlation with suspected influenza-like illness admissions is the overall decline in hospital patient volumes since COVID-19 restrictions were implemented in March 2020 [11, 50].

The negative effect of the COVID-19 pandemic on ED attendances has been demonstrated by Hughes et al. using Public Health England’s Emergency Department Syndromic Surveillance System (EDSSS) [11]. Compared to 2019 data, the authors determined ED attendances during March and April 2020 were significantly lower than ED attendances during the same time period in the previous year (non-respiratory indicators fell by 44–67% and acute respiratory infection fell by 4.4%) [11]. In relation to ACES Pandemic Tracker, syndromes often reported as suspected pneumonia and influenza-like illness admissions prior to the COVID-19 pandemic were instead being flagged by clinicians as suspected COVID-19 during the pandemic. This could be a result of recommendations circulated by the Ontario Ministry of Health to use suspected COVID-19 flags to improve surveillance and monitoring of COVID-19.

The cross-correlation analysis in the present study demonstrated a possible lag of 3 days between suspected COVID-19 (excluding SOI) admissions and confirmed COVID-19 admissions. Although this is likely related to the length of time required to report on laboratory results early in the pandemic, it illustrates the potential utility of using hospital admissions surveillance as an earlier indicator of COVID-19 severity when it is known to be circulating within the population. In terms of emergency preparedness, early warning of a potential surge in COVID-19 hospital admissions of 3 days can provide adequate time to re-allocate hospital resources including hospital staffing and availability of hospital beds in anticipation of severe COVID-19 infections that may require hospitalization and intensive care.

The SOI displayed in ACES Pandemic Tracker include symptoms that may be related to COVID-19 but are not clinical diagnoses. Correlation analyses between SOI and diagnosed COVID-19 cases need to be assessed on a regular basis and validated when diagnostic records are available, which are often offset by several days. The early warning capabilities built into ACES were exploited in the Pandemic Tracker based on aggregate counts of SOI and COVID-19 flagged hospital admissions to identify a potential surge in novel COVID-19 cases. SyS in this manner can provide situational awareness for epidemiologists, public health officials and hospitals to monitor for local analysis, improve identification and investigation and inform responses to possible surges in severe COVID-19 infection and other potential population health threats. It can also inform risk communication to the media, public and policy decision-makers.

Our results suggest one of the greatest strengths of ACES Pandemic Tracker is its ability to be easily updated with new keyword-based syndromes and adapted to new and emerging viruses and diseases which makes it a highly relevant and effective public health tool [33, 51]. ACES Pandemic Tracker monitors hospital admissions for syndromes related to COVID-19 since these presentations are expected to increase when widespread community infection of COVID-19 occurs. Improving the monitoring of syndromes related to COVID-19 admissions will assist in identification of potential cases and enable earlier physical distancing and self-isolation to limit the risk of person-to-person transmission and community spread of the virus. The incorporation of spatial information also allows public health professionals to identify specific regions that may have an increase in suspected COVID-19 infection rates.

Another strength of our surveillance approach lends itself to the monitoring of hospital admissions rather than ED visits. We believe surveillance of hospital admissions was more appropriate in this context since ED visits steadily declined from mid-March 2020. A possible explanation for the decline in ED visits in Ontario is that individuals who exhibited symptoms related to COVID-19 were directed to local assessment centers, which do not share data with ACES. Another explanation may simply be compliance with physical distancing measures. In addition, ED visit data was obscured by individuals who sought COVID-19 testing and other health-seeking behaviors during the initial wave of the COVID-19 pandemic which could lead to false assumptions regarding COVID-19 visits. Therefore, we believe hospital admissions data is a reliable source for surveillance of indications related to community spread of COVID-19 and resulting impacts on the healthcare system. Other international surveillance groups could consider adopting similar methods to monitor hospital admissions for future novel disease outbreaks. Furthermore, patients requiring hospital admission due to COVID-19 are generally more severe cases of infection that threaten to overwhelm hospital resources.

Overall, this study enumerates the statistical validity of ACES Pandemic Tracker and its usefulness as a disease surveillance system. The syndromic data provides a good estimation of real data counts that can inform public health action to reduce impact of a disease outbreak and community spread. Early in the pandemic, a limited number of public resources were available that could provide real-time information on COVID-19 cases and hospital admissions. The already established ACES system allowed for timely implementation of ACES Pandemic Tracker while more formalized provincial surveillance systems were under development. ACES Pandemic Tracker supported local public health agencies and other public health practitioners across the province as a standardized method to report hospital admissions data. The methodology, lessons learned and evidence from the post-hoc analyses demonstrate that a similar tool could be beneficial and adopted by other public health authorities in future pandemic scenarios.

Our findings are relevant for other national, regional and local public health agencies globally that may wish to establish and implement disease surveillance systems at a population level. Although our SyS was implemented in a Canadian setting, the methods used to monitor hospital admissions for possible increases in severe COVID-19 disease activity are not specific to one setting or disease. SyS of hospital admissions can be applied to other global contexts and to other novel disease outbreaks. Based on our results, we wish to emphasize that development of a new flagging criteria for a novel disease outbreak may be necessary under certain circumstances which may include a decline in baseline hospital volumes/syndromes or when disease presentation does not accurately reflect symptoms captured in existing syndromes. The new flagging criteria should be continuously updated to reflect possible changes or variation in disease presentation due to genetic drift over time. It should be noted that our approach required effective communication and collaboration among government, healthcare, public health agencies and the public. Globally, this may pose a barrier to implementation depending on institutional arrangements, cooperation, communication and public health resources [52].

There are limitations of ACES Pandemic Tracker that should be considered. First, we were unable to calculate sensitivity and specificity of the Pandemic Tracker since we did not have access to laboratory data. However, based on previous evaluations of syndromic surveillance systems, due to the non-differential nature of selected key syndromes and since laboratory confirmation is not required for suspected COVID-19 flags, we would expect Pandemic Tracker to be highly sensitive with low specificity, when compared to confirmed COVID-19 admissions [25]. Second, the value of ACES Pandemic Tracker was perhaps best demonstrated as an early warning system to help guide system planners while laboratory testing for COVID-19 was delayed and there were significant restrictions on testing prior to establishment of COVID-19 assessment centres. However, it maintains the ability to estimate the potential burden of severe COVID-19 cases on hospital resources.

Conclusion

The ability to predict potential surges of severe COVID-19 and other communicable diseases within the population is of particular importance for public health surveillance, intervention and prevention. The evaluation of ACES Pandemic Tracker suggests SyS of hospital admissions could be used to provide situational awareness, monitor real-time patient volumes and identify potential surges in severe COVID-19 infection levels in a timely manner. Such SyS systems can be utilized by public health agencies, healthcare professionals, community service partners and testing laboratories to take informed measures to plan and allocate resources appropriately and aid public health in implementing measures to prevent further spread of infection.

Availability of data and materials

ACES is bound by data sharing agreements with each participating hospital and/or hospital corporation and their regional local public health agencies. ACES Pandemic Tracker is a publicly accessible tool and available without a user agreement enabling free and open information for academics, health professionals, emergency services, the media and the public on the community spread of COVID-19 and its impact on hospitals at https://www.kflaphi.ca/aces-pandemic-tracker/. Data for confirmed COVID-19 cases is publicly available and openly accessible from the Ontario Ministry of Health at https://covid-19.ontario.ca/data.

Abbreviations

ACES:

Acute Care Enhanced Surveillance

AST:

Asthma

CCF:

Cross-correlation function

CHF:

Congestive heart failure

CI:

Confidence interval

COPD:

Chronic obstructive pulmonary disorder

COVID-19:

Coronavirus disease 2019

ED:

Emergency department

ICD:

International classification of diseases

ILI:

Influenza-like illness

KFL&A:

Kingston, Frontenac and Lennox & Addington

NLP:

Natural language processing

PN:

Pneumonia

RESP:

Respiratory

SARS:

Severe acute respiratory syndrome

SEP:

Sepsis

SOI:

Syndrome of interest

SyS:

Syndromic surveillance

References

  1. World Health Organization. WHO Coronavirus Disease (COVID-19) Dashboard. 2020. https://covid19.who.int. Accessed 8 Mar 2021.

    Google Scholar 

  2. Public Health Ontario. Ontario COVID-19 Data Tool. 2020. https://www.publichealthontario.ca/en/data-and-analysis/infectious-disease/covid-19-data-surveillance/covid-19-data-tool?tab=summary. Accessed 8 Mar 2021.

    Google Scholar 

  3. Melidou A, Pereyaslov D, Hungnes O, Prosenc K, Alm E, Adlhoch C, et al. Virological surveillance of influenza viruses in the WHO European region in 2019/2020 - impact of the COVID-19 pandemic. Euro Surveill. 2020;25(46). https://doi.org/10.2807/1560-7917.ES.2020.25.46.2001822.

  4. Ibrahim N. Epidemiologic surveillance for controlling COVID-19 pandemic: types, challenges and implications. J Infect Public Health. 2020;13(11):1630–8. https://doi.org/10.1016/j.jiph.2020.07.019.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Post L, Issa T, Boctor M, Moss C, Murphy R, Ison M, et al. Dynamic public health surveillance to track and mitigate the US COVID-19 epidemic: longitudinal trend analysis study. J Med Internet Res. 2020;22(12):e24286. https://doi.org/10.2196/24286.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Murray C, Alamro N, Hwang H, Lee U. Digital public health and COVID-19. Lancet Public Heal. 2020;5(9):E469–70. https://doi.org/10.1016/S2468-2667(20)30187-0.

    Article  Google Scholar 

  7. Hughes H, Edeghere O, O’Brien S, Vivancos R, Elliot A. Emergency department syndromic surveillance systems: a systematic review. BMC Public Health. 2020;20(1):1891. https://doi.org/10.1186/s12889-020-09949-y.

    Article  PubMed  PubMed Central  Google Scholar 

  8. World Health Organization. Public health surveillance. 2020. https://www.who.int/immunization/monitoring_surveillance/burden/vpd/en/. Accessed 8 Mar 2021.

    Google Scholar 

  9. Nsubuga P, White M, Thacker S, Anderson M, Blount S, Broome C, et al. Public health surveillance: a tool for targeting and monitoring interventions. In: Disease control priorities in developing countries. 2nd ed. Washington (DC): Oxford University Press; 2006. p. 997–1015. https://www.ncbi.nlm.nih.gov/books/NBK11770/.

    Google Scholar 

  10. Hiller K, Stoneking L, Min A, Rhodes S. Syndromic surveillance for influenza in the emergency department-a systematic review. Plos One. 2013;8(9):e73832. https://doi.org/10.1371/journal.pone.0073832.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Hughes H, Hughes T, Morbey R, Challen K, Oliver I, Smith G, et al. Emergency department use during COVID-19 as described by syndromic surveillance. Emerg Med J. 2020;37(10):600–4. https://doi.org/10.1136/emermed-2020-209980.

    Article  PubMed  Google Scholar 

  12. Smith G, Hippisley-Cox J, Harcourt S, Heaps M, Painter M, Porter A. Developing a national primary care-based early warning system for health protection--a surveillance tool for the future? Analysis of routinely collected data. J Public Heal. 2007;29(1):75–82. https://doi.org/10.1093/pubmed/fdl078.

    Article  Google Scholar 

  13. Gault G, Larrieu S, Durand C, Josseran L, Jouves B, Filleul L. Performance of a syndromic system for influenza based on the activity of general practioners. Fr J Public Heal. 2009;31(2):286–92. https://doi.org/10.1093/pubmed/fdp020.

    Article  CAS  Google Scholar 

  14. Harcourt S, Morbey R, Smith G, Loveridge P, Green H, Pebody R, et al. Developing influenza and respiratory syncytial virus activity thresholds for syndromic surveillance in England. Epidemiol Infect. 2019;147:e163. https://doi.org/10.1017/S0950268819000542.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Vergu E, Grais R, Sarter H, Fagot J, Lambert B, Valleron A, et al. Medication sales and syndromic surveillance. France Emerg Infect Dis. 2006;12(3):416–21. https://doi.org/10.3201/eid1203.050573.

    Article  PubMed  Google Scholar 

  16. Todd S, Diggle P, White P, Fearne A, Read J. The spatiotemporal association of non-prescription retail sales with cases during the 2009 influenza pandemic in Great Britain. BMJ Open. 2014;4(4):e004869. https://doi.org/10.1136/bmjopen-2014-004869.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Mostashari F, Fine A, Das D, Adams J, Layton M. Use of ambulance dispatch data as an early warning system for communitywide influenzalike illness, New York City. J Urban Heal. 2003;80:i43–9.

    Google Scholar 

  18. Coory M, Kelly H, Tippett V. Assessment of ambulance dispatch data for surveillance of influenza-like illness in Melbourne, Australia. Public Health. 2009;123(2):163–8. https://doi.org/10.1016/j.puhe.2008.10.027.

    Article  CAS  PubMed  Google Scholar 

  19. Greenko J, Mostashari F, Fine A, Layton M. Clinical evaluation of the emergency medical services (EMS) ambulance dispatch-based syndromic surveillance system, New York City. J Urban Heal. 2003;80:i50–6.

    Article  Google Scholar 

  20. Todkill D, Loveridge P, Elliot A, Morbey R, Edeghere O, Rayment-Bishop T, et al. Utility of ambulance data for real-time syndromic surveillance: a pilot in the West Midlands Region, United Kingdom. Prehosp Disaster Med. 2017;32(6):667–72. https://doi.org/10.1017/S1049023X17006690.

    Article  PubMed  Google Scholar 

  21. Public Health England. Syndromic surveillance: systems and analyses. 2021.

    Google Scholar 

  22. Ma T, Englund H, Bjelkmar P, Wallensten A, Hulth A. Syndromic surveillance of influenza activity in Sweden: an evaluation of three tools. Epidemiol Infect. 2015;143(11):2390–8. https://doi.org/10.1017/S0950268814003240.

    Article  CAS  PubMed  Google Scholar 

  23. Cook E, Randhawa G, Guppy A, Large S. A study of urgent and emergency referrals from NHS direct within England. BMJ Open. 2015;5(5):e007533. https://doi.org/10.1136/bmjopen-2014-007533.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Charles-Smith L, Reynolds T, Cameron M, Conway M, Lau E, Olsen J, et al. Using social media for actionable disease surveillance and outbreak management: a systematic literature review. Plos One. 2015;10(10):e0139701. https://doi.org/10.1371/journal.pone.0139701.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Provincial Infectious Diseases Advisory Committee (PIDAC). Syndromic surveillance discussion paper. 2012. https://www.publichealthontario.ca/-/media/documents/S/2012/syndromic-surveillance-discussion.pdf?la=en.

    Google Scholar 

  26. Ward M, Stanley A, Deeth L, Deardon R, Feng Z, Trotz-Williams L. Methods for detecting seasonal influenza epidemics using a school absenteeism surveillance system. BMC Public Health. 2019;19(1):1232. https://doi.org/10.1186/s12889-019-7521-7.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Hughes H, Morbey R, Hughes T, Locker T, Pebody R, Green H, et al. Emergency department syndromic surveillance providing early warning of seasonal respiratory activity in England. Epidemiol Infect. 2016;144(5):1052–64. https://doi.org/10.1017/S0950268815002125.

    Article  CAS  PubMed  Google Scholar 

  28. Buda S, Tolksdorf K, Schuler E, Kuhlen R, Haas W. Establishing an ICD-10 code based SARI-surveillance in Germany - description of the system and first results from five recent influenza systems. BMC Public Health. 2017;17(1):612. https://doi.org/10.1186/s12889-017-4515-1.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Loonsk J. BioSense--a national initiative for early detection and quantification of public health emergencies. MMWR Suppl. 2004;53:53–5.

    PubMed  Google Scholar 

  30. van Dijk A, Aramini J, Edge G, Moore K. Real-time surveillance for respiratory disease outbreaks, Ontario. Canada Emerg Infect Dis. 2009;15.

  31. Hall G, Krahn T, Majury A, van Dijk A, Evans G, Moore K, et al. Emergency department surveillance as a proxy for the prediction of circulating respiratory viral disease in eastern Ontario. Can J Infect Dis Med Microbiol. 2013;24(3):150–4. https://doi.org/10.1155/2013/386018.

    Article  PubMed  PubMed Central  Google Scholar 

  32. van Dijk A, Dawson E, Moore K, Belanger P. Risk assessment during the Pan American and Parapan American games, Toronto, 2015. Public Health Rep. 2017;132(1_suppl):106S–10S. https://doi.org/10.1177/0033354917708356.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Hope K, Durrheim D, D’Espaignet E, Dalton C. Syndromic surveillance: is it a useful tool for local outbreak detection? J Epidemiol Community Heal. 2006;60(5):374–5. https://doi.org/10.1136/jech.2005.035337.

    Article  Google Scholar 

  34. Jia K, Mohamed K. Evaluating the use of cell phone messaging for community Ebola syndromic surveillance in high risked settings in southern Sierra Leone. Afr Health Sci. 2015;15(3):797–802. https://doi.org/10.4314/ahs.v15i3.13.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Houlihan C, Youkee D, Brown C. Novel surveillance methods for the control of Ebola virus disease. Int Health. 2017;9(3):139–41. https://doi.org/10.1093/inthealth/ihx010.

    Article  CAS  PubMed  Google Scholar 

  36. Centers for Disease Control and Prevention. COVID Data Tracker. 2021. https://covid.cdc.gov/covid-data-tracker/#cases_casesper100klast7days. Accessed 10 May 2021.

    Google Scholar 

  37. Guemes A, Ray S, Aboumerhi K, Desjardins M, Kvit A, Corrigan A, et al. A syndromic surveillance tool to detect anomalous clusters of COVID-19 symptoms in the United States. Sci Rep. 2021;11(1):4660. https://doi.org/10.1038/s41598-021-84145-5.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Yoneoka D, Tanoue Y, Kawashima T, Nomura S, Shi S, Eguchi A, et al. Large-scale epidemiological monitoring of the COVID-19 epidemic in Tokyo. Lancet Reg Heal - West Pacific. 2020;4.

  39. Nomura S, Yoneoka D, Shi S, Tanoue Y, Kawashima T, Eguchi A, et al. An assessment of self-reported COVID-19 related symptoms of 227,898 users of a social networking service in Japan: has the regional risk changed after the declaration of the state of emergency? Lancet Reg Heal - West Pacific. 2020;1.

  40. Mahmud A, Chowdhury S, Sojib K, Chowdhury A, Quader M, Paul S, et al. Participatory syndromic surveillance as a tool for tracking COVID-19 in Bangladesh. Epidemics. 2021;35:100462. https://doi.org/10.1016/j.epidem.2021.100462.

    Article  PubMed  PubMed Central  Google Scholar 

  41. KFL&A Public Health. Acute Care Enhanced Surveillance. 2020. https://www.kflaphi.ca/acute-care-enhanced-surveillance/. Accessed 8 Mar 2021.

    Google Scholar 

  42. KFL&A Public Health. ACES Pandemic Tracker. 2020. https://www.kflaphi.ca/aces-pandemic-tracker/. Accessed 8 Mar 2021.

    Google Scholar 

  43. Ising A, Proescholdbell S, Harmon K, Sachdeva N, Marshall S, Waller A. Use of syndromic surveillance data to monitor poisonings and drug overdoses in state and local public health agencies. Inj Prev. 2016;22(Suppl 1):i43–9. https://doi.org/10.1136/injuryprev-2015-041821.

    Article  PubMed  Google Scholar 

  44. Ising A, Travers D, MacFarquhar J, Kipp A, Waller A. Triage note in emergency department-based syndromic surveillance. Adv Dis Surveill. 2006;1:34.

    Google Scholar 

  45. Travers D, Barnett C, Ising A, Waller A. Timeliness of emergency department diagnoses for syndromic surveillance. AMIA Annu Symp Proc. 2006:769–73.

  46. Maharaj A, Parker J, Hopkins J, Gournis E, Bogoch I, Rader B, et al. The effect of seasonal respiratory virus transmission on syndromic surveillance for COVID-19 in Ontario, Canada. Lancet Infect Dis. 2021;21(5):593–4. https://doi.org/10.1016/S1473-3099(21)00151-1.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Wong W-K, Moore A. Classical time-series methods for biosurveillance. In: Wagner M, Moore A, RM A, editors. Handbook of biosurveillance. MA: Elsevier Academic Press; 2006, doi: https://doi.org/10.1016/B978-012369378-5/50016-8.

  48. Ontario Ministry of Health. All Ontario: case numbers and spread. 2021. https://covid-19.ontario.ca/data. Accessed 4 Mar 2021.

    Google Scholar 

  49. Statistics Canada. Population estimates, quarterly. 2020. https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1710000901. Accessed 15 Dec 2020.

    Google Scholar 

  50. Elliot A, Harcourt S, Hughes H, Loveridge P, Morbey R, Smith S, et al. The COVID-19 pandemic: a new challenge for syndromic surveillance. Epidemiol Infect. 2020;148:e22.

    Article  Google Scholar 

  51. Paterson B, Durrheim D. The remarkable adaptability of syndromic surveillance to meet public health needs. J Epidemiol Glob Health. 2013;3(1):41–7. https://doi.org/10.1016/j.jegh.2012.12.005.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Hyland-Wood B, Gardner J, Leask J, Ecker U. Toward effective government communication strategies in the era of COVID-19. Humanit Soc Sci Commun. 2021;8(1). https://doi.org/10.1057/s41599-020-00701-w.

Download references

Acknowledgements

ACES is maintained by KFL&A Public Health and funded by the Ontario Ministry of Health.

Funding

The authors received no financial support for the research, authorship or publication of this article.

Author information

Authors and Affiliations

  1. Knowledge Management Division, Kingston, Frontenac and Lennox & Addington Public Health, 221 Portsmouth Avenue, Kingston, Ontario, K7M 1V5, Canada

    Nicholas Papadomanolakis-Pakis, Allison Maier, Adam van Dijk & Nancy VanStone

  2. Office of the Medical Officer of Health, Kingston, Frontenac and Lennox & Addington Public Health, 221 Portsmouth Avenue, Kingston, Ontario, K7M 1V5, Canada

    Kieran Michael Moore

Authors
  1. Nicholas Papadomanolakis-Pakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Allison Maier
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adam van Dijk
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nancy VanStone
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kieran Michael Moore
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AM, AVD, NV and KMM contributed to the development of ACES Pandemic Tracker. KMM conceived the idea of the study. NPP conducted data analyses and prepared the manuscript. All authors reviewed, revised and approved the final manuscript.

Corresponding author

Correspondence to Nicholas Papadomanolakis-Pakis.

Ethics declarations

Ethics approval and consent to participate

The data analyses in this research article has been conducted with aggregate data in accordance with the ethical and legal limitations defined by the data sharing agreements between ACES, public health agencies and the participating hospitals in Ontario and is exempt from Research Ethics Board review.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Syndrome descriptions and ICD-10 codes for validation.

Additional file 2.

COVID-19 flagging criteria.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Papadomanolakis-Pakis, N., Maier, A., van Dijk, A. et al. Development and assessment of a hospital admissions-based syndromic surveillance system for COVID-19 in Ontario, Canada: ACES Pandemic Tracker. BMC Public Health 21, 1230 (2021). https://doi.org/10.1186/s12889-021-11303-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12889-021-11303-9

Keywords

BMC Public Health

ISSN: 1471-2458

Contact us