Skip to main content

Performance of existing and novel surveillance case definitions for COVID-19 in household contacts of PCR-confirmed COVID-19

Abstract

Background

Optimized symptom-based COVID-19 case definitions that guide public health surveillance and individual patient management in the community may assist pandemic control.

Methods

We assessed diagnostic performance of existing cases definitions (e.g. influenza-like illness, COVID-like illness) using symptoms reported from 185 household contacts to a PCR-confirmed case of COVID-19 in Wisconsin and Utah, United States. We stratified analyses between adults and children. We also constructed novel case definitions for comparison.

Results

Existing COVID-19 case definitions generally showed high sensitivity (86–96%) but low positive predictive value (PPV) (36–49%; F-1 score 52–63) in this community cohort. Top performing novel symptom combinations included taste or smell dysfunction and improved the balance of sensitivity and PPV (F-1 score 78–80). Performance indicators were generally lower for children (< 18 years of age).

Conclusions

Existing COVID-19 case definitions appropriately screened in household contacts with COVID-19. Novel symptom combinations incorporating taste or smell dysfunction as a primary component improved accuracy. Case definitions tailored for children versus adults should be further explored.

Peer Review reports

Background

Coronavirus disease 2019 (COVID-19) is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). The virus was first identified in a cluster of patients with atypical pneumonia in Wuhan, China, in December 2019 [1]. Since its emergence, the virus has spread globally, causing widespread infection and death. Following evidence of person-to-person transmission and a broader clinical spectrum of infections, case definitions for COVID-19 have been revised [2]. In the initial weeks of the pandemic, COVID-19 was labeled a pneumonia of unknown etiology, and many who presented to medical care had classic pneumonia-like symptoms such as fever, cough, and dyspnea. An exceptional variety of symptoms has since been reported for COVID-19, ranging from none or mild indistinct symptoms to invasive neurological disease and fulminant respiratory failure [3,4,5,6,7]. As is common in the early response phases to novel emerging pathogens, there is ongoing need to reassess and refine surveillance case definitions for COVID-19 based on new information. Changes to case definitions affect interpretation of surveillance data, as was demonstrated by substantially different prevalence estimates when China broadened the COVID-19 case definition early in its epidemic response [2].

A few studies have demonstrated the predictive value of symptom profiles in healthcare workers [4, 8, 9] and other populations potentially not necessarily representative of the general public [5, 10]. These studies are subject to other limitations, too. Some applied predictive models that included serum biomarkers and imaging [11, 12]. Obtaining this information may limit real-world capture of people with mild-to-moderate SARS-CoV-2 infection and may delay public health intervention. Further, few studies to date have examined symptom combinations exclusively. Respiratory pathogens routinely behave differently in children and adults, and this appears to be true for COVID-19 as well [13]. For example, an assessment of ambulatory case surveillance definitions for influenza demonstrated lower sensitivity among children less than 5 years of age [14]. Similar analyses across age strata are lacking for COVID-19. Reliable, age-stratified syndromic surveillance definitions would likely aid public health officials to scale up community contact tracing and develop protocols to safely operate various congregate venues, such as schools and workplaces, should unlimited, timely diagnostic testing be unavailable.

Dedicated symptom-based surveillance systems have been developed to track COVID-19 cases. These include the U.S. Council of State and Territorial Epidemiologists (CSTE) original (CSTE combination 1; released April 5, 2020) and revised (CSTE combination 2; released August 7, 2020) clinical criteria for reporting SARS-CoV-2 infection, and the original CDC COVID-19–like illness (CLI) definition (Table 1). Similarly, the Centers for Disease Control and Prevention (CDC) maintains a list of symptoms for priority SARS-CoV-2 testing. These COVID-19 case definitions and the priority testing symptom list are intended to capture as many persons with COVID-19 as possible with confirmatory testing. Finally, longstanding respiratory virus surveillance networks established to monitor influenza-like illnesses (ILI) and acute respiratory infection (ARI), which is used for community-based syndromic surveillance of respiratory syncytial virus by the World Health Organization (WHO), may be plausibly adaptable platforms for monitoring COVID-19. The performance characteristics and utility of these syndromic surveillance platforms for COVID-19 have not been well defined [5].

Table 1 Existing COVID-19 case definitions, respiratory illness surveillance case definitions, and derived compound symptom combinations assessed for diagnostic performance in a community cohort of 185 individuals with household COVID-19 exposure in Utah and Wisconsin, United States, March–May 2020

We aimed to describe the diagnostic performance of two existing case definitions for COVID-19, the CDC COVID-19 symptom list, and two longstanding viral respiratory disease surveillance definitions among persons with confirmed SARS-CoV-2 exposure, stratified between adults and children. We also aimed to derive novel, practical symptom combinations in the same population. We interpreted the results primarily within the framework of two core public health surveillance functions: 1) symptom-based screening of individuals to guide SARS-CoV-2 diagnostic testing, contact tracing, and community-based isolation and quarantine, and 2) estimating disease frequency in persons with documented SARS-CoV-2 exposure. For symptom screening, we considered the merits of novel combinations when unlimited, timely diagnostic testing is unavailable.

Methods

Study design and data collection

CDC collaborated with state and local health departments in the Milwaukee, Wisconsin and Salt Lake City, Utah metropolitan areas in the United States to identify and enroll a convenience sample of people with laboratory-confirmed SARS-CoV-2 infection and their household contacts from March 22 to April 22, 2020. Ours are secondary analyses of this household transmission investigation whose questionnaire and methods were previously published in detail [15, 16]. This activity was reviewed by CDC and was conducted consistent with applicable federal law and CDC policy. See e.g., 45 Code of Federal Regulations (CFR) part 46, 21 CFR part 56; 42 United States Code (USC) §241(d); 5 USC §552a; 44 USC §3501 et seq.

We administered questionnaires to household contacts to assess the presence of 15 symptoms during the 14 days prior to or at enrollment (day 0). Additionally, participants completed a daily symptom diary during days 1–14 after enrollment. We collected serum and upper respiratory specimens (i.e., both nasopharyngeal [NP] and anterior nares swabs) on day 0 and day 14. We additionally collected NP swabs at any interim date if any household contact newly developed or had worsening of any one of 15 symptoms consistent with COVID-19: nasal congestion or runny nose, sore throat, cough, chest pain, shortness of breath, discomfort while breathing, wheezing, headache, new loss of taste or smell, fever/chills, fatigue, muscle aches, diarrhea (≥3 loose stools per day), abdominal pain, or nausea/vomiting. The Milwaukee Health Department and Utah Public Health Laboratories tested the swabs using the CDC real-time Reverse Transcriptase Polymerase Chain Reaction (RT-PCR) assay for SARS-CoV-2 [17], and CDC tested sera using a CDC-developed SARS-CoV-2 enzyme-linked immunosorbent assay (ELISA) [18].

Definitions

We defined a household contact to be a COVID-19 case if they had at least one specimen test positive for SARS-CoV-2 by RT-PCR. We classified persons < 18 years of age as children, and persons ≥18 years of age as adults. We combined all symptoms recorded at any time prior to enrollment (and after the index case’s symptom onset date) through the end of the 14-day observation period. We assessed individual symptoms, existing symptom combinations, and newly constructed symptom combinations for their association with SARS-CoV-2 test result by RT-PCR (Table 1). We asked enrollees to state whether they experienced any loss of taste and, separately, smell during the specified time period. For enrollees who responded yes to this question, we then asked whether the loss was partial or complete. We defined loss and/or dysfunction of taste or smell to include any level of loss, whether partial or complete. For ARI, we interpreted coryza as runny nose or nasal congestion.

Analytic methods

We excluded household contacts from the main analysis if not present at enrollment or not completing the study procedures. Our analysis of combinations predictive of COVID-19 included all 15 symptoms surveyed. We formally described the diagnostic performance of each individual symptom, existing COVID-19 case definitions, respiratory illness case definitions, and newly constructed symptom combinations (Table 1). The goal of assessing symptom combinations was to accurately divide the population into two groups: those who tested positive for SARS-CoV-2 and those who tested negative. For a given combination, we calculated the association of the combination with respect to laboratory-confirmed SARS-CoV-2 infection, yielding the number of contacts who were true positive (TP) (i.e., positive symptom profile and positive test), false negative (FN) (i.e., negative symptom profile and positive test), false positive (FP) (i.e., positive symptom profile and negative test), or true negative (TN) (i.e., negative symptom profile and negative test). From these values, we calculated the symptom combination’s sensitivity (i.e., TP / [TP + FN]), specificity (i.e., TN / [TN + FP]), positive predictive value (PPV) (i.e., TP / [TP + FP]), negative predictive value (NPV) (i.e., TN / [TN + FN]), F-1 score (the harmonic mean of sensitivity and PPV), and Youden’s index ([sensitivity + specificity] – 100). To determine how well each definition would estimate prevalence in a syndromic surveillance system, we also calculated the difference in the number of positive symptom screens (i.e., TP + FP) from the actual number of contacts who tested positive by RT-PCR. We assessed combinations across all ages and in children and adults separately, and we reported all measures on the percentage scale. To assess variability in each performance measure, we constructed bias-corrected and accelerated bootstrap confidence intervals [19] over 10,000 pseudosamples constructed by resampling households with replacement. We reported 95% confidence intervals with two exceptions. For measures estimated at 100% in the observed sample, we omitted confidence intervals, because the pseudosamples could not exhibit any variability. For the difference in specificity and sensitivity between adults and children, we reported 97.5% confidence intervals (a Bonferroni correction) to allow for a 95% joint confidence level regarding the differences in each pair.

We adapted innovative methods previously applied in the low-resource context to derive a parsimonious symptom combination to prioritize diagnostic testing for tuberculosis [20]. We chose this approach to be as comprehensive as practical for COVID-19 in that it systematically assessed nearly every conceivable combination of symptoms. First, we searched over 245,000 combinations of between one and 15 symptoms (i.e., simple combinations of the form “at least m symptoms present out of n symptoms considered”). We gave greater weight to combinations with high F-1 score or high Youden’s index. We then conducted an exhaustive search using pairs of these “m–of–n” combinations (i.e., compound combinations) to allow for more nuanced combinations. We limited this second search to single combinations of no more than five symptoms, such that the number of total symptoms evaluated for a compound combination was never more than ten. We allowed each pair of combinations to be joined by the logical operators [AND] and [OR], yielding approximately 73 million unique combination pairs. After the search, we selected four combination pairs to include in the primary analysis based on diagnostic performance and parsimony. We measured diagnostic performance by F-1 score (higher being better). We measured parsimony by the total number of symptoms evaluated (fewer being better) (Table 1).

We performed all calculations in R 4.0.0 (R Core Team), Python 3.7 (Python Software Foundation), or both. To allow for parallel processing, the exhaustive combinatorial search and bootstrap confidence intervals were implemented on a scientific workstation with 24 logical cores and 64 GB of RAM. De-identified data and analytic scripts in R and Python are publicly available through a GitHub repository: https://github.com/scotthlee/covid-casedefs.

Results

Study population

We enrolled 199 contacts of index patients with laboratory-confirmed SARS-CoV-2 infections within 62 households. We excluded one contact who was not living in the home on the day of enrollment, one who was hospitalized at enrollment, and two who did not consent to have specimens collected. Ten contacts had negative RT-PCR and positive serology test results; they were also excluded from the primary analyses. Therefore, our analyses included the remaining 185 household contacts. The median time interval from index patient’s symptom onset date to enrollment was 10 days (interquartile range [IQR]: 7–13). About half (95; 51%) were female. 108 (58%) were Caucasian/white, 32 (17%) Latinx/Hispanic, 23 (12%) African American/black, 14 (8%) Asian, four (2%) Native American, and four (2%) multiracial. The median age was 22 years ([IQR]: 14–47), with 122 (66%) adults and 63 (34%) children. Thirty-three enrollees (18%) were over age 50 years and 6 (3%) were over age 65 years. Despite the relatively young age distribution, nearly one-third (55; 30%) had at least one underlying health condition. Among children, nine (14%) were < 5 years, 19 (30%) were 5–9 years, and 35 (56%) were 10–17 years of age. SARS-CoV-2 infection was detected by RT-PCR in 49 (27%) household contacts. Separated by age group, 35/122 (29%) adults and 14/63 (22%) children had laboratory-confirmed SARS-CoV-2 infection by RT-PCR. Among the 49 RT-PCR-positive contacts, most (45; 92%) also had a positive serology result, three had a negative serology result, and one was not tested by serology.

Performance characteristics for individual symptoms (all ages pooled)

Individual symptoms with the highest sensitivity were nasal congestion or rhinorrhea, headache, and cough (Table 2, Fig. 1). Many of the individual symptoms reported were highly specific, although generally resulting in lower sensitivity. The exception was loss or dysfunction of taste or smell (categorized as a single symptom), which had a moderate sensitivity of 63% (95% confidence interval [CI] 47–77%), high specificity (96%; 95% CI 90–99%), high NPV (88%; 95% CI 80–93%), and the highest PPV (84%; 95% CI 64–94%), Youden’s index (59%; 95% CI 42–73%), and F-1 score (72%; 95% CI 57–83%).

Table 2 Diagnostic performance indicators for individual COVID-19 symptoms, existing case definitions, and derived compound symptom combinations for a community cohort of 185 people with household COVID-19 exposure in Utah and Wisconsin, United States, March–May 2020
Fig. 1
figure 1

Sensitivity and 100%-specificity for individual COVID-19 symptoms, existing case definitions, and derived compound symptom combinations for a community cohort of 185 individuals with household exposure to COVID-19 in Utah and Wisconsin, United States, March–May 2020. Specificity is the probability of testing negative when disease is absent. Sensitivity is the probability of testing positive when disease is present. CDC symptom list=U.S. Centers for Disease Control and Prevention (CDC) list of symptoms that may indicate COVID-19 infection (https://www.cdc.gov/coronavirus/2019-ncov/symptoms-testing/symptoms.html). This symptom list was last updated on 13 May 2020. ARI=World Health Organization (WHO) acute respiratory infection (ARI) definition for community-based respiratory syncytial virus (RSV) surveillance (https://www.who.int/influenza/rsv/rsv_case_definition/en/). Last updated 04 February 2020. CSTE combination 1=U.S. Council of State and Territorial Epidemiologists (CSTE) original clinical criteria for COVID-19 reporting (https://cdn.ymaws.com/www.cste.org/resource/resmgr/2020ps/interim-20-id-01_covid-19.pdf). This interim position statement (Interim-20-ID-01) was approved on 05 April 2020 and was replaced by Interim-20-ID-02 on 07 August 2020. CSTE combination 2=U.S. Council of State and Territorial Epidemiologists (CSTE) revised clinical criteria for COVID-19 reporting (https://cdn.ymaws.com/www.cste.org/resource/resmgr/ps/positionstatement2020/interim-20-id-02_COVID-19.pdf). This interim position statement (Interim-20-ID-02) was approved on 07 August 2020 and replaced Interim-20-ID-01. CLI=U.S. Centers for Disease Control and Prevention (CDC) COVID-19-like illness (CLI) definition was used to guide early diagnostic testing strategies from 17 January 2020–08 March 2020 (https://emergency.cdc.gov/han/han00426.asp). ††ILI=Influenza-like illness (ILI) outpatient visit information collected through the U.S. Outpatient Influenza-like Illness Surveillance Network (ILINet) (https://www.cdc.gov/flu/weekly/overview.htm#anchor_1539281266932). This collaborative effort between CDC, state and local health departments, and healthcare providers has been tracking patients with ILI since the 1997–1998 influenza season. Derived compound combination 1: Taste and/or smell dysfunction, OR one of the following: shortness of breath, myalgia, or fever or chills. Derived compound combination 2: Taste and/or smell dysfunction or discomfort breathing, OR at least two of the following: shortness of breath, wheezing, or fever/chills. Derived compound combination 3: Taste and/or smell dysfunction, OR at least two of the following: shortness of breath, wheezing, discomfort breathing, or fever/chills. Derived compound combination 4: Taste and/or smell dysfunction, OR shortness of breath and fever/chills. Points closest to the upper left corner represent those with the highest sensitivity and specificity values

Performance characteristics for existing COVID-19 case definitions, the CDC symptom list, and respiratory syndromic Surveillance networks (all ages pooled)

Among the existing case definitions, sensitivity was perfect (100%) for the CDC symptom list definition, and also high for ARI (96%; 95% CI 86–100%), CSTE combination 1 (original); 94%; 95% CI 82–98%), CSTE combination 2 (revised); 98%; 95% CI 88–100%), and CLI (86%; 95% CI 70–94%) (Table 2, Fig. 1). While these definitions offered high sensitivity, they were poorly specific. Conversely, ILI was highly specific (90%; 95% CI 83–94%) but insensitive (51%; 95% CI 35–65%). All existing definitions demonstrated low PPV. Youden’s indices and F-1 scores were highest for CSTE combination 1 and CLI, though still suboptimal. None of the existing definitions predicted prevalence well; the difference from true prevalence ranged from − 20 for ILI to 196 for the CDC symptom list. Compared to CSTE combination 1, CSTE combination 2 had slightly higher sensitivity and NPV, but performed more poorly on all other diagnostic performance indicators. Although not included in the primary analyses, existing case definitions capture ≥90% of COVID-19 detected by ELISA only (Supplemental Table 1).

Performance characteristics for derived compound symptom combinations (all ages pooled)

The four highest performing novel symptom combinations, based on F-1 score and parsimony, were compound symptom combinations that included dysfunction of taste or smell. These four combinations performed similarly well on all performance measures (Table 2, Fig. 1). We determined compound symptom combination 3 (i.e., loss or dysfunction of taste or smell, or at least two of the following: shortness of breath, wheezing, discomfort breathing, or fever/chills), to be simple to implement, have higher specificity (93%; 95% CI 85–97%), NPV (92%; 95% CI 86–96%), PPV (79%; 95% CI 60–91%), Youden’s index (70%; 95% CI 54–82%), and F-1 score (78%; 95% CI 66–87%) than existing case definitions. The compound symptom combination 3 showed near-perfect prevalence prediction (− 2; 95% CI − 20–24), and sensitivity was moderately high (78%; 95% CI 63–88%).

Adult-child differences in discriminatory performance

The accuracy of symptom profiles for defining RT-PCR confirmed COVID-19 differed by age (Table 3, Fig. 2). Overall, existing case definitions were less sensitive in children compared to adults. One exception, the CDC symptom list for priority testing (Table 1), captured all COVID-19 cases regardless of age. The existing case definitions were more specific in children, but the greater specificity was statistically significant for CSTE combination 1 only. Individual symptoms showed a similar pattern of lower sensitivity among children, notably taste/smell dysfunction. Sore throat was more sensitive in children, and fever/chills and nausea were similar regardless of age group. We observed a similar pattern of increased specificity for most derived symptom combinations in children (Table 3, Fig. 2). Cough was the sole symptom where the difference in both sensitivity and specificity was statistically significant. For both children and adults, the CLI case definition provided the greatest balance between both sensitivity and specificity (Youden’s Index 53%; 95% CI 8–80% vs. 52%; 95% CI 36–66%, respectively) and harmonization of sensitivity with PPV (F-1 61%; 95% CI 26–83% vs. 63%; 95% CI 49–76%, respectively) (Table 2). CLI also most accurately predicted overall prevalence amongst children (percent difference from true prevalence 36%; 95% CI –17–157) (Table 3).

Table 3 Diagnostic performance indicators for individual COVID-19 symptoms, existing case definitions, and derived compound symptom combinations by age group for 122 adults and 63 children with household exposure to COVID-19 in Utah and Wisconsin, United States, March–May 2020
Fig. 2
figure 2

Sensitivity and 100%-specificity for individual COVID-19 symptoms, existing case definitions, and derived compound symptom combinations for a community cohort of 122 adults (upper case letters) and 63 children (lower case letters) with household exposure to COVID-19 in Utah and Wisconsin, United States, March–May 2020. Specificity is the probability of testing negative when disease is absent. Sensitivity is the probability of testing positive when disease is present. CDC symptom list=U.S. Centers for Disease Control and Prevention (CDC) list of symptoms that may indicate COVID-19 infection (https://www.cdc.gov/coronavirus/2019-ncov/symptoms-testing/symptoms.html). This symptom list was last updated on 13 May 2020. ARI=World Health Organization (WHO) acute respiratory infection (ARI) definition for community-based respiratory syncytial virus (RSV) surveillance (https://www.who.int/influenza/rsv/rsv_case_definition/en/). Last updated 04 February 2020. CSTE combination 1=U.S. Council of State and Territorial Epidemiologists (CSTE) original clinical criteria for COVID-19 reporting (https://cdn.ymaws.com/www.cste.org/resource/resmgr/2020ps/interim-20-id-01_covid-19.pdf). This interim position statement (Interim-20-ID-01) was approved on 05 April 2020 and was replaced by Interim-20-ID-02 on 07 August 2020. CSTE combination 2=U.S. Council of State and Territorial Epidemiologists (CSTE) revised clinical criteria for COVID-19 reporting (https://cdn.ymaws.com/www.cste.org/resource/resmgr/ps/positionstatement2020/interim-20-id-02_COVID-19.pdf). This interim position statement (Interim-20-ID-02) was approved on 07 August 2020 and replaced Interim-20-ID-01. CLI=U.S. Centers for Disease Control and Prevention (CDC) COVID-19-like illness (CLI) definition was used to guide early diagnostic testing strategies from 17 January 2020–08 March 2020 (https://emergency.cdc.gov/han/han00426.asp). ILI=Influenza-like illness (ILI) outpatient visit information collected through the U.S. Outpatient Influenza-like Illness Surveillance Network (ILINet) (https://www.cdc.gov/flu/weekly/overview.htm#anchor_1539281266932). This collaborative effort between CDC, state and local health departments, and healthcare providers has been tracking patients with ILI since the 1997–1998 influenza season. Derived compound combination 1: Taste and/or smell dysfunction, OR one of the following: shortness of breath, myalgia, or fever or chills. Derived compound combination 2: Taste and/or smell dysfunction or discomfort breathing, OR at least two of the following: shortness of breath, wheezing, or fever/chills. Derived compound combination 3: Taste and/or smell dysfunction, OR at least two of the following: shortness of breath, wheezing, discomfort breathing, or fever/chills. Derived compound combination 4: Taste and/or smell dysfunction, OR shortness of breath and fever/chills. Points closest to the upper left corner represent those with the highest sensitivity and specificity values

Discussion

Existing case surveillance definitions for COVID-19, as shown in Table 1, were generally sensitive in our study conducted among household contacts of infected persons, a population with proven SARS-CoV-2 exposure. However, they tended to have low specificity and poorly estimated disease prevalence. By systematically screening novel definitions that optimized sensitivity, specificity, and PPV, we improved community prevalence estimation and overall accuracy of individual screening, which could be useful if diagnostic testing is limited. In particular, we affirmed loss or dysfunction of taste or smell as a uniquely discerning characteristic central to constructing an effective, concise case surveillance definition when applied across all age groups (i.e., derived compound combination 3).

An appropriate discriminatory balance between sensitivity and specificity for a newly emerging pathogen depends on the objectives of the surveillance activity [21]. Highly sensitive case definitions capture a larger proportion of true COVID-19 cases, which is ideal when diagnostic testing is widely available and results are timely. Highly sensitive definitions, however, generally rule in a larger number of non-cases (i.e., FP symptom screens) [22]. In addition to testing resources, the public health system’s tolerance for false-positive screens is, of course, dependent on human resources. This is especially apparent when intensive interventions involve extensive contact tracing, isolation and quarantine. At high community COVID-19 prevalence, these intensive mitigation efforts may benefit from evidence-based prioritization. By example, CSTE combination 2 had a FP symptom screening rate (77/136; 57%) eight times that for derived compound combination 3 (10/136; 7%) in our cohort. At the population level, such differences could expose shortcomings in resources for core interventions, such as universal contact tracing. The COVID-19 response has repeatedly been strained in these requisite areas [23,24,25]. Novel symptom screening criteria that more tightly couple sensitivity and specificity (i.e., diagnostic accuracy), such as the derived compound combination 3, could help to prioritize interventions when strategically deployed. This principle may also apply when evaluating novel vaccines or therapeutics in large clinical trials involving thousands of participants, where feasibility constraints often dictate the use of symptom-prioritized testing to confirm outcomes. Still, highly sensitive symptom rules, such as CSTE combination 2, are preferred for COVID-19 when resources are unlimited.

For using syndromic surveillance systems to estimate community burden, the highly sensitive existing case definitions overestimated true burden. Conversely, highly specific case definitions, such as ILI, may detect changes in disease trends over time but underestimate true burden [21]. ILI underestimated disease prevalence by more than 80% in this study population. Current laboratory-based surveillance grossly under-ascertains incidence [26], especially where diagnostic testing is not easily accessible or widespread. Retailoring community-based syndromic surveillance systems already in place [27] (i.e., altering the symptoms included or applying a correction factor based on results such as ours) would more accurately reflect true burden.

For most symptoms and their combinations, overall performance, most notably sensitivity, differed between child and adult household contacts. These findings are consistent with prior observations whereby children generally show fewer and milder symptoms of COVID-19 compared with adults [28], and COVID-19 syndromes vary across ages [13]. The small number of children with COVID-19 in this cohort limits the conclusion of specific recommendations, but further examination into the utility of age-specific case definitions is warranted in considering policies for schools and other child congregate settings, and for deriving accurate burden estimates from syndromic surveillance.

While the number of individuals in this study is relatively small, our study population is well-characterized. We collected extensive symptom data, which yielded a comprehensive assessment of multiple symptom combinations. We also employed inclusion criteria that were not based on disease status or symptom status, and a reference category based on standardized laboratory testing. Nonetheless, we acknowledge this study’s limitations. These analyses were not intended to produce definitive symptom combinations to be applied to the general public, however they may be used to guide the development of future candidate case definitions. One key consideration for future validation efforts is that enrollment started immediately after the precipitous decline in laboratory-confirmed influenza virus infections in the United States in mid-March 2020 [29]. Thus, our estimates of diagnostic performance may differ during the viral respiratory season. In addition, COVID-19 prevalence was higher for our study population (i.e., contacts of laboratory-confirmed household members) compared to the entire community, thereby limiting the generalizability of predictive values (although sensitivity and specificity remain unaffected by disease prevalence). Our study population was younger than the general population and the screening criteria may perform differently in older adults. We did not have enough older household contacts to permit further stratification among adults. Finally, screening criteria applied to persons seeking medical care may also perform differently, as those individuals probably tend to have more severe illness.

Additionally, we showed that existing COVID-19 case definitions are highly sensitive and do well to screen in persons for testing and individual-level public health interventions like community isolation. In the first such endeavor for evaluating and deriving novel COVID-19 case surveillance definitions in a community setting among SARS-CoV-2–exposed individuals with largely mild illness, we evaluated novel symptom combinations for COVID-19 using methodology previously applied to tuberculosis in low resource settings [20]. These derived combinations and CSTE definition 2 better estimated disease burden and used taste and/or smell dysfunction as a primary component. The latter is supported by prior studies [5, 8,9,10]. Because most SARS-CoV-2 infections are mild [30] and core public health functions may need prioritization when testing and other resources are limited, case definitions that accurately determine COVID-19 status in the general public may assist continued interruption of community transmission [31]. When timely diagnostic testing is readily available, however, using less sensitive screening tools could inappropriately miss cases and lead to further community transmission.

Our study population, which includes participants enrolled independent of disease and symptom status, may better reflect the diagnostic performance in the general population than previously published research. Accurate clinical case definitions are likely to also apply to large clinical trials for candidate vaccines and therapeutics where serial confirmatory SARS-CoV-2 testing for any new symptom is impractical. It is important that our results be validated against the growing body of larger ambulatory surveillance databases in diverse communities and in other countries; in particular, our methodology should be assessed in the context of the annual influenza season, at varying community COVID-19 prevalence, and across the age spectrum. Such studies ideally can be accompanied by cost-effectiveness modeling of intervention strategies.

Conclusions

The discriminatory performance of case surveillance definitions for COVID-19 is important for implementing effective epidemic mitigation strategies. Our study illustrates the performance of case definitions in community members with household exposure to SARS-CoV-2 based solely on symptom profiles. Prior work overrepresented healthcare workers or otherwise studied non-representative populations, and they did not examine both adults and children. Our study also provides a novel framework for refining definitions. Using 15 symptoms associated with COVID-19 for all contacts regardless of disease status, we systematically evaluated the discriminatory performance of individual symptoms and previously defined case surveillance definitions in adults and children, and according to two core surveillance applications: 1) screening non-hospitalized individuals to prioritize public health interventions, and 2) estimating the number of non-hospitalized persons with COVID-19 (i.e., community-based syndromic surveillance). We also constructed novel symptom combinations that effectively performed both functions and, in this study population, improved upon widely used case surveillance definitions that may help to target interventions in the absence of unlimited laboratory diagnostic capacity. Based on our results, case surveillance definition performance may increase if developed separately for adults and children.

Availability of data and materials

De-identified data and analytic scripts in R and Python are publicly available through a GitHub repository: https://github.com/scotthlee/covid-casedefs.

Abbreviations

COVID-19:

Coronavirus disease 2019

PPV:

Positive predictive value

SARS-CoV-2:

Severe acute respiratory syndrome coronavirus 2

CSTE:

Council of State and Territorial Epidemiologists

CLI:

Coronavirus-like illness

CDC:

United States Centers for Disease Control and Prevention

ILI:

Influenza-like illness

ARI:

Acute respiratory illness

WHO:

World Health Organization

CFR:

Code of Federal Regulations

USC:

United States Code

NP:

Nasopharyngeal

RT-PCR:

Reverse transcriptase polymerase chain reaction

ELISA:

Enzyme-linked immunosorbent assay

TP:

True positive

FN:

False negative

TN:

True negative

FP:

False positive

NPV:

Negative predictive value

GB:

Gigabytes

RAM:

Random access memory

IQR:

Interquartile range

CI:

Confidence interval

References

  1. Zhu N, Zhang D, Wang W, Li X, Yang B, Song J, et al. A novel coronavirus from patients with pneumonia in China, 2019. N Engl J Med. 2020;382(8):727–33. https://doi.org/10.1056/NEJMoa2001017.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  2. Tsang TK, Wu P, Lin Y, Lau EHY, Leung GM, Cowling BJ. Effect of changing case definitions for COVID-19 on the epidemic curve and transmission parameters in mainland China: a modelling study. Lancet Public Health. 2020;5(5):e289–96. https://doi.org/10.1016/S2468-2667(20)30089-X.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Pascarella G, Strumia A, Piliego C, Bruno F, Del Buono R, Costa F, et al. COVID-19 diagnosis and management: a comprehensive review. J Intern Med. 2020;288(2):192–206. https://doi.org/10.1111/joim.13091.

    CAS  Article  PubMed  Google Scholar 

  4. Tostmann A, Bradley J, Bousema T, Yiek WK, Holwerda M, Bleeker-Rovers C, et al. Strong associations and moderate predictive value of early symptoms for SARS-CoV-2 test positivity among healthcare workers, the Netherlands, March 2020. Euro Surveill. 2020;25(16). https://doi.org/10.2807/1560-7917.ES.2020.25.16.2000508.

  5. Menni C, Valdes AM, Freidin MB, Sudre CH, Nguyen LH, Drew DA, et al. Real-time tracking of self-reported symptoms to predict potential COVID-19. Nat Med. 2020;26(7):1037–40. https://doi.org/10.1038/s41591-020-0916-2.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  6. Chen N, Zhou M, Dong X, Qu J, Gong F, Han Y, et al. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. Lancet. 2020;395(10223):507–13. https://doi.org/10.1016/S0140-6736(20)30211-7.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  7. Varatharaj A, Thomas N, Ellul MA, Davies NWS, Pollak TA, Tenorio EL, et al. Neurological and neuropsychiatric complications of COVID-19 in 153 patients: a UK-wide surveillance study. Lancet Psychiatry. 2020;7(10):875–82. https://doi.org/10.1016/S2215-0366(20)30287-X.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Lan FY, Filler R, Mathew S, Buley J, Iliaki E, Bruno-Murtha LA, et al. COVID-19 symptoms predictive of healthcare workers’ SARS-CoV-2 PCR results. PLoS One. 2020;15(6):e0235460. https://doi.org/10.1371/journal.pone.0235460.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  9. Clemency BM, Varughese R, Scheafer DK, Ludwig B, Welch JV, McCormack RF, et al. Symptom criteria for COVID-19 testing of heath care workers. Acad Emerg Med. 2020;27(6):469–74. https://doi.org/10.1111/acem.14009.

    Article  PubMed  Google Scholar 

  10. Ahmed MAM, Colebunders R, Siewe Fodjo JN. Evidence for important COVID-19 community transmission in Somalia using a clinical case definition. Int J Infect Dis. 2020;98:206–7. https://doi.org/10.1016/j.ijid.2020.06.068.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  11. Struyf T, Deeks JJ, Dinnes J, Takwoingi Y, Davenport C, Leeflang MM, et al. Signs and symptoms to determine if a patient presenting in primary care or hospital outpatient settings has COVID-19 disease. Cochrane Database Syst Rev. 2020;7:CD013665. https://doi.org/10.1002/14651858.CD013665.

    Article  PubMed  Google Scholar 

  12. Wynants L, Van Calster B, Collins GS, Riley RD, Heinze G, Schuit E, et al. Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ. 2020;369:m1328.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Ludvigsson JF. Systematic review of COVID-19 in children shows milder cases and a better prognosis than adults. Acta Paediatr. 2020;109(6):1088–95. https://doi.org/10.1111/apa.15270.

    CAS  Article  PubMed  Google Scholar 

  14. Casalegno JS, Eibach D, Valette M, Enouf V, Daviaud I, Behillil S, et al. Performance of influenza case definitions for influenza community surveillance: based on the French influenza surveillance network GROG, 2009-2014. Euro Surveill. 2017;22(14). https://doi.org/10.2807/1560-7917.ES.2017.22.14.30504.

  15. Lewis N, Chu V. Household transmission of SARS-CoV-2 in the United States. Clin Infect Dis. 2020. https://doi.org/10.1093/cid/ciaa1166.

  16. Laws RL, Chancey RJ, Rabold EM, Chu VT, Lewis NM, Fajans M, et al. Symptoms and transmission of SARS-CoV-2 among children-Utah and Wisconsin, March-May 2020. Pediatrics. 2020;147(1). https://doi.org/10.1542/peds.2020-027268.

  17. CDC. 2019-Novel Coronavirus (2019-nCoV) Real-Time RT-PCR Diagnostic Panel. 2020.

    Google Scholar 

  18. Freeman B, Lester S, Mills L, Ata Ur Rasheed M, Moye S, Abiona O, Hutchinson GB, Morales-Betoulle M, Krapinunaya I, Gibbons A, Chiang C-F, Cannon D, Klena J, Johnson JA, Owen SM, Graham BS, Corbett KS, Thornburg NJ. Validation of a SARS-CoV-2 spike protein ELISA for use in contact investigations and sero-surveillance. bioRxiv. 2020.04.24.057323. https://doi.org/10.1101/2020.04.24.057323, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7239067/.

  19. Efron B. Better bootstrap confidence intervals. J Am Stat Assoc. 1987;82(397):171–85. https://doi.org/10.1080/01621459.1987.10478410.

    Article  Google Scholar 

  20. Cain KP, McCarthy KD, Heilig CM, Monkongdee P, Tasaneeyapan T, Kanara N, et al. An algorithm for tuberculosis screening and diagnosis in people with HIV. N Engl J Med. 2010;362(8):707–16. https://doi.org/10.1056/NEJMoa0907488.

    CAS  Article  PubMed  Google Scholar 

  21. Giesecke J. Modern infectious disease epidemiology. New York: Oxford University Press; 2002.

    Google Scholar 

  22. Ngobeni H, Tempia S, Cohen AL, Walaza S, Kuonza L, Musekiwa A, et al. The performance of different case definitions for severe influenza surveillance among HIV-infected and HIV-uninfected children aged <5 years in South Africa, 2011-2015. PLoS One. 2019;14(9):e0222294. https://doi.org/10.1371/journal.pone.0222294.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  23. Keeling MJ, Hollingsworth TD, Read JM. Efficacy of contact tracing for the containment of the 2019 novel coronavirus (COVID-19). J Epidemiol Community Health. 2020:jech-2020-214051. https://doi.org/10.1136/jech-2020-214051.

  24. Watson C, Cicero A, Blumenstock J, Fraser M. A national plan to enable comprehensive COVID-19 case finding and contact tracing in the U.S. Baltimore: Johns Hospkins University; 2020. p. 15.

    Google Scholar 

  25. Hellewell J, Abbott S, Gimma A, Bosse NI, Jarvis CI, Russell TW, et al. Feasibility of controlling COVID-19 outbreaks by isolation of cases and contacts. Lancet Glob Health. 2020;8(4):e488–96. https://doi.org/10.1016/S2214-109X(20)30074-7.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Havers FP, Reed C, Lim T, Montgomery JM, Klena JD, Hall AJ, et al. Seroprevalence of antibodies to SARS-CoV-2 in 10 sites in the United States, March 23–May 12, 2020. JAMA Intern Med. 2020. https://doi.org/10.1001/jamainternmed.2020.4130:E1-E11.

  27. Chan AT, Brownstein JS. Putting the public back in public health - surveying symptoms of Covid-19. N Engl J Med. 2020;383(7):e45. https://doi.org/10.1056/NEJMp2016259.

    CAS  Article  PubMed  Google Scholar 

  28. Davies NG, Klepac P, Liu Y, Prem K, Jit M, group CC-w, et al. Age-dependent effects in the transmission and control of COVID-19 epidemics. Nat Med. 2020;26(8):1205–11. https://doi.org/10.1038/s41591-020-0962-9.

    CAS  Article  PubMed  Google Scholar 

  29. Influenza Surveillance. Available from cdc.gov/flu/weekly/index.htm. Accessed 4 Aug 2020.

  30. To KK, Tsang OT, Leung WS, Tam AR, Wu TC, Lung DC, et al. Temporal profiles of viral load in posterior oropharyngeal saliva samples and serum antibody responses during infection by SARS-CoV-2: an observational cohort study. Lancet Infect Dis. 2020;20(5):565–74. https://doi.org/10.1016/S1473-3099(20)30196-1.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  31. Li R, Pei S, Chen B, Song Y, Zhang T, Yang W, et al. Substantial undocumented infection facilitates the rapid dissemination of novel coronavirus (SARS-CoV-2). Science. 2020;368(6490):489–93. https://doi.org/10.1126/science.abb3221.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

Utah site:

Davis County (UT) Health Department: Heather Gibb, Sara Hall

Summit County (UT) Health Department: Carolyn Rose

Salt Lake County (UT) Health Department: Dagmar Vitek, Ilene Risk, Lee Cherie Booth, Jeff Sanchez, Madison Clawson, Tara Scribellito, Ha Khong, Carlene Claflin, Theresa Beesley, Victoria Castaneda

Utah Department of Health: Nathan LaCross, Robyn Atkinson-Dunn, COVID-19 response team

Wisconsin site:

North Shore Health Department: Kala Hardy, Christine Cordova, Kevin Rorabeck, Kathleen Platt

City of Milwaukee Health Department: Catherine Bowman, Nancy Burns, Barbara Coyle, Elizabeth Durkes, Carol Johnsen, Jill LeStarge, Erica Luna-Vargas, Sholonda Morris, Mary Jo Gerlach, Jill Paradowski, Bill Rice, Michele Robinson, Virginia Thomas, Keara Jones, Chelsea Watry, Richard Weidensee, Jeanette Kowalik, Heather Paradis, Julie Katrichis

Wauwatosa Health Department: Laura Conklin, Paige Bernau, Emily Tianen

Wisconsin Department of Health Services: COVID-19 response team

City of Milwaukee Laboratory SARS-CoV-2 Testing Team: Jordan Hilleshiem, Beth Pfotenhauer, Manjeet Khubbar, Jennifer Lentz, Zorangel Amezquita-Montes, Kristin Schieble, Noah Leigh, Joshua Weiner, Tenysha Guzman, Kathy Windham, and Julie Plevak

Centers for Disease Control and Prevention:

CDC COVID-19 Response Team: Claire Midgley, Melissa Rolfes, Mayer Antoine, Adebowale Ojo, Fiona Havers

Laboratory Task Force: Brandi Limbago, Michelle Owens, Wendi Kuhnert-Tallman, Jeff Johnson, Collette Leaumont, Gayle Langley

Field Investigations Team: Jenny Milucky, Emily Weston, Jessica Smith, Michelle Johnson Jones, Jennifer Huang, Erin Moritz, Aubrey Gilliland, Laura Calderwood, Jennifer Imaa, Kaytlin Renfro, Allison Miller, Katie Bantle, Margaret Williams, Stacy Thorne, Jana Manning, Micha Ghertner, Adriane Niare, Yoonjae Kang, Charlotte D. Kaboré

U.S. COVID-19 Household Investigation Team (group authorship in the byline)

Michelle Banks, MS1, Katherine A. Battey, MPH1, Alison M. Binder, MS1, Sean Buono, PhD1,5, Rebecca J. Chancey, MD1, Ann Christiansen, MPH6, Erin E. Conners, PhD1, Trivikram Dasu, PhD4, Patrick Dawson, PhD1,2, Elizabeth Dietrich, PhD1, Lindsey M. Duca, PhD1,2, Angela C. Dunn, MD3, Victoria L. Fields, DVM1,2, Garrett Fox, MPH1, Brandi D. Freeman, PhD1,5, Radhika Gharpure, DVM1,2, Christopher Gregory, MD1, Tair Kiphibane, BSN7, Rebecca L. Laws, PhD1, Sandra Lester, PhD1, Nathaniel M. Lewis, PhD1,2,3, Perrine Marcenac, PhD1,2, Almea M. Matanock, MD1, Lisa Mills, PhD1, Henry Njuguna, MD1, Michelle O’Hegarty, PhD1, Daniel Owusu, DrPH1,2, Lindsey Page, MPH4, Lucia Pawloski, PhD1, Eric Pevzner, PhD1, Mary Pomeroy, MSN1,2, Ian W. Pray, PhD1,2,8, Elizabeth M. Rabold, MD1,2, Jared R. Rispens, MD1,2, Phillip Salvatore, PhD1,2, Amy Schumacher, PhD1, Cuc H. Tran, PhD1, Jeni Vuong, MS1, Ashutosh Wadhwa, PhD1,5, Ryan P. Westergaard, MD, PhD8, Sarah Willardson, MPH9, Dongni Ye, PhD1, Sherry Yin, MPH1, Anna Yousaf, MD1,2

1 COVID-19 Response Team, Centers for Disease Control and Prevention, Atlanta, Georgia, USA (1)

2 Epidemic Intelligence Service, Centers for Disease Control and Prevention, Atlanta, Georgia, USA (2)

3 Utah Department of Health, Salt Lake City, Utah, USA (3)

4 City of Milwaukee Health Department, Milwaukee, Wisconsin, USA (4)

5 Laboratory Leadership Service, Centers for Disease Control and Prevention, Atlanta, Georgia, USA (5)

6 North Shore Health Department, Milwaukee, Wisconsin, USA (7)

7 Salt Lake County Health Department, Salt Lake City, Utah, USA

8 Wisconsin Department of Health, Madison, Wisconsin, USA (6)

9 Davis County Health Department, Salt Lake City, Utah, USA

Funding

This research was wholly supported by the U.S. Centers for Disease Control and Prevention (CDC). The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the U.S. Centers for Disease Control and Prevention/the Agency for Toxic Substances and Disease Registry.

Author information

Authors and Affiliations

Authors

Consortia

Contributions

All authors contributed to study design, data collection, scientific interpretation, and review and approval of the final manuscript. In addition, the following authors made unique contributions: SHL and CMH conducted all data analysis, provided open analytic code for public dissemination, led data interpretation, wrote data-analytic methods, and assisted with integrating analysis and interpretation throughout the manuscript. RAD, SB, TD, KC, BDF, SL, LM, LP, and NJT coordinated or performed diagnostic laboratory analyses. KAB, SB, RJC, AC, EEC, PD, ED, LMD, VLF, RG, CG, TK, RLL, NML, PM, AMM, HN, MO, DO, LP, EP, MP, IWP, MR, JRR, PS, CHT, JV, AW, SW, SY, AY implemented the study protocol, developed standards of practice, and collected data and specimens from the field. MB, ACD, AF, AJH, HLK, SAN, AS, JET, RPW, and DY provided overall project coordination and leadership. HER, MF, GF, AMB, and VTC curated the data. HER, MF, SHL, CMH, and SAN drafted the original manuscript. All authors have read and approved the manuscript.

Corresponding author

Correspondence to Scott A. Nabity.

Ethics declarations

Ethics approval and consent to participate

This protocol was reviewed by the Associate Director for Science of the 2019 Novel Coronavirus Response and the activity was deemed nonresearch as part of the COVID-19 public health response. This is the CDC process for ethics review; the authors had no role in this decision. The activity was conducted consistent with applicable federal law and CDC policy. See e.g., 45 C.F.R. part 46, 21 C.F.R. part 56; 42 U.S.C. 241(d);5 U.S.C. 552a; 44 U.S.C. 3501 et seq. Still, enrollees provided written, informed consent to participate.

Consent for publication

Not applicable.

Competing interests

The authors report no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Supplemental Table 1.

Individual COVID-19 symptoms and existing case definitions by 2019-nCoV Real-Time RT-PCR assay and a SARS-CoV-2 spike protein enzyme-linked immunosorbent assay (ELISA) results in Utah and Wisconsin, United States, March–May 2020.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Reses, H.E., Fajans, M., Lee, S.H. et al. Performance of existing and novel surveillance case definitions for COVID-19 in household contacts of PCR-confirmed COVID-19. BMC Public Health 21, 1747 (2021). https://doi.org/10.1186/s12889-021-11683-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12889-021-11683-y

Keywords

  • COVID-19
  • SARS-CoV-2
  • Surveillance
  • Symptoms
  • Syndromic
  • Sensitivity
  • Specificity
  • Predictive values
  • Diagnostic accuracy
  • Children
  • Adults