The primary limitation of this analysis is the lack of definitive knowledge about the actual number of pH1N1 cases at the two universities - a "gold standard." To address this problem we developed an approach that compares ("triangulates") multiple data systems, each with its own expected biases over time, to identify those that most likely mirror actual disease trends. Epidemiologists are typically aware of these potential biases in a qualitative sense, and present their analysis of the available data with appropriate caveats. In our approach, which benefits from hindsight, we attempt to use information about the likely direction and time patterns of these biases to understand the surveillance system and the validity and utility of different syndromic surveillance data sources. This type of analysis is necessarily qualitative and contextual; rather than serving as a recipe for doing this in other settings, this analysis should be seen as an example that illustrates the concept. This analysis also illustrates to the call in U.S. National Health Security Strategy Implementation Plan (released for public comment in 2010) for the development, refinement, and wide-spread implementation of quality improvement tools, specifically methods "to collect data ... from real incidents ... to identify gaps, [and] recommend and apply programs to mitigate those gaps ."
Another limitation of the data analysis is the uncertainty of whether the ILI cases captured by the surveillance system were pH1N1. As recommended by the CDC interim guidelines , both universities stopped routinely testing for pH1N1 in early September. Although the CDC Virologic Surveillance data for region 3 suggested that the predominant proportion of the test positive specimens were subtyped as pH1N1, the total percentage for test positive samples varied from 4.4% to 55.9% in weeks 35 to 50 . Because this proportion varies so much, trends in reported ILI cases may not reflect trends in actual H1N1 infection.
As described in more detail below, this approach suggests that the peak in cases at both universities at the beginning of the semester, a peak not seen in data for the surrounding community, is probably real and a reflection of expected disease dynamics. The lower peak, especially at University A, when pH1N1 was widespread in the community might reflect the removal of susceptible cases earlier in the semester, or simply surveillance fatigue. This analysis also suggests surveillance artifacts - surveillance fatigue and changing incentives driven by the exam schedule - that are likely to influence surveillance data in future outbreaks, and that should be taken into account in the interpretation of these data.
Unique transmission pattern in IHEs
Both universities experienced the first and the highest peak in student ILI cases immediately after Fall semester classes started in early September 2009, which corresponds to peaks found in other universities and colleges in Region 3 (Delaware, the District of Columbia, Maryland, Pennsylvania, Virginia, and West Virginia). It should be noted, however, that both of these universities contributed to the ACHA reports. The CDC ILINet data for the same region and Google Flu Trends data for Washington, DC, on the other hand, did not peak until late October. University A also experienced a second, lower peak in cases with a two-week delay in early November, according to the SHC and ED data. When comparing the SHC and ED data from University A and University B (as shown in Figure 5), University B's ED data differ from other data sets from both universities, and show a transmission pattern that resembles the CDC ILINet data. Since the ILI cases from both hospital EDs are not restricted to students, they include not only student cases but also other 17 to 24 year old adults in the community and from other parts of the city. Unlike University A which is not inaccessible by mass transit, University B is located in a part of Washington that has a large population of young adults and easy access to the public transportation, so the higher volume of young adult visits at University B ED during November, when the virus was circulating in the general population, may not have come from the college population.
In the comparison between ACHA and CDC ILINet data across all states, the tendency of an early increase in ILI cases among college students in seven out of ten regions, as shown in Additional File 3. Together with our findings, this suggests that the difference in the timing of peaks reflects the differences between college students and the general public. This is plausible given that students are in an age group at higher risk for infection. Moreover, students returning to campus for the Fall term may have carried the virus from their home states, and the sudden increase of population density in dormitories and lecture halls may also have contributed to the rapid onset of the outbreak on campus due to facilitated transmission. Thus, it seems likely that on a national level, residential IHEs tended to experience an early peak immediately after the Fall term began in 2009.
Influence of incentives and informational environment
All of the data analysed in this report are based, to some degree, on students and staff taking action based their illness. Such behaviour is driven not only by the fact of being sick, but also by the incentives to report, including perceptions of barriers to help-seeking behaviour (i.e. geographic distance, queuing, chance of exposure to other infected patients), the likely benefit to be gained (medical and non-medical) by reporting, the timeliness of the help to be delivered, as well as the informational environment the students and staff are exposed to. In particular, two factors - surveillance fatigue and reporting incentives - seem capable of explaining some of the patterns in the data.
As seen in Figure 1, the Fall semester at both universities began with a high awareness of pH1N1. At University A, a number of new ad hoc surveillance systems were developed, some of which required a substantial reporting burden by students and staff at the student health center and deans' offices, athletic trainers and RAs, and of course the students themselves. Over the course of the semester, however, it became apparent that although pH1N1 was widespread in children and young adults, it was not as virulent as feared [17, 18], and the frequency of H1N1 messages dropped. It would not be surprising, therefore, that staff who put a substantial effort into reporting ILI cases in September were less enthusiastic about it as the semester wore on, and possibly less complete in their reporting. Moreover, since most students who presented themselves for medical attention early in the semester did not receive anti-viral or other specific treatment as per the CDC guidelines, it seems likely that their friends and roommates who became ill later in the semester saw no reason to seek medical care.
Surveillance fatigue is likely to be more obvious in systems that use human resources not primarily designated for disease prevention and health promotion. For instance, the reports from the RA at University A increased to their highest level in the first week after classes resumed and dropped dramatically afterwards. Although ILI activity could still be observed from other data sources after the second peak through Spring 2010, the reports from RAs completely stopped at the end of November. The RA reporting system might have been sensitive to student ILI cases in the early stages, considering the relatively low barrier of utilizing the resources (close proximity, no queuing), and the expectation of immediate help (supply distribution, accommodation relocation). However, when the reporters and those receiving the reports are all laypersons to public health practice, fading interest can be magnified in the microenvironment between the two parties.
At University A, undergraduate students were instructed to notify their deans about their influenza-like illness as a substitute for medical proof of illness otherwise required to justify absence from class. This was published on August 28, 2009, and not emphasized afterwards. However, as noted in Figure 2, reports to undergraduate deans at University A were elevated relative to the other series in October, when mid-term exams were scheduled in many classes. Thus it seems likely that the relative number of reports to deans at University A during this period (or more precisely the failure for the number to drop as sharply as other data series), may reflect the increased need for students to have medical excuses for exams rather than for ordinary classes, where attendance is usually not taken.
Evaluation of syndromic surveillance data systems
To translate these results into recommendations for IHEs regarding the design and implementation of surveillance systems for future disease outbreaks, other factors must also be taken into account. For instance, surveillance activities conducted by trained health care workers are more likely to capture actual ILI cases based on clinical findings. Moreover, well-informed healthcare workers who conduct surveillance as part of their regular responsibilities are more likely to maintain a relatively stable and predictable report triggering threshold, in line with the CDC and WHO (World Health Organization) guidelines [19–21]. On the other hand, ad hoc reporting systems may be more sensitive to changes in the informational environment. When the reporting channels are relatively new, communication messages designed to encourage their use might have a short-term effect when they are released, but surveillance fatigue may set in quickly when the intensity of media coverage decreases and public interest fades. In addition, the expected benefits of presenting oneself to the reporting system, and how easily reports can be made, may also have an impact on the direction and scope of the bias. In a pandemic characterized by low virulence and limited treatment options for young adults, the expected benefits of seeking care decreased over time, except for the mid-term exam effect observed in Deans' reports at University A. Thus, this assessment suggests that at least for outbreaks similar to pH1N1, student health center data, though biased by surveillance fatigue, provides the most accurate and useful data.