Routinely collected medical record information has several important advantages for surveillance purposes, and particularly for the surveillance of infection syndromes. Among the most important is the ability to assess the large majority of episodes of illness for which no etiologic agents are identified, either because good medical practice does not require that clinicians perform diagnostic tests, or because an unusual agent may fail to be detected by standard laboratory tests. Syndromic surveillance should thus be useful both for tracking the activity of infectious agents that are common in communities, and also for identification of new, emerging infections or bioterrorist attack. The detection of an increased frequency of events would typically trigger more intensive assessment, including more diagnostic testing than would ordinarily be indicated. Syndromic surveillance also allows the earliest possible identification of increased disease frequency, presumably days before laboratory test results become available. This early indication of a problem may be important in detecting and responding to a bioterrorist attack, for instance the release of anthrax in a community.
Other advantages of automated diagnosis data for surveillance include uniformity and increased sensitivity of detection, since clinicians are not required to recognize a condition as being of interest. These data also circumvent the need for providers to initiate reporting, an important consideration in light of the time pressures that constrain existing reporting. For some purposes, automated methods may augment or replace resource intensive sentinel surveillance programs, for example those described by Armstrong  and de Wit , that have been created to collect information that isn't captured by standard reporting systems. Finally, data from automated medical record systems lend themselves well to incorporation in automated detection systems with little or no added cost, because the data are available in electronic form, avoiding the additional costs and errors due to data entry.
For a data source to be useful for disease surveillance, it must be timely, accurate, complete, and capable of distinguishing events of interest from background occurrences, i.e., an acceptable signal to noise ratio. The interval between a patient being seen and data being available for analysis must be short, particularly in any bioterrorist threat situation. Diagnostic and demographic accuracy are needed in order to enable reliable evaluation of geographic clustering of specific emerging infections or syndromes. Complete data or at least reasonably complete sampling is essential if events of relatively small scale are to be detected.
In terms of timeliness, this ambulatory record information can be available very quickly. In practice, it is efficient extract each day's visits of interest during the succeeding night, thus making the information available within 24 hours of the patient encounter.
In terms of accuracy, the data are probably of acceptable reliability for patient demographics and encounter dates, since this information comes from the administrative database used for reimbursement. The validity or reliability of physician diagnosis in terms of ICD9 codes is neither known nor readily amenable to measurement. In other settings, ICD9 codes have been shown to have substantial discrepancies, when they are compared to the information in the full text medical record . We expect that the diagnoses of interest here also have substantial errors that reduce both sensitivity and specificity. It is likely that these errors are relatively stable over the time periods of interest for surveillance of acute disease syndromes, and so this problem may not interfere appreciably with day to day comparisons. However, there could be important differences attributable to coding practices between groups of clinicians, or in different medical record systems, for instance if the automated systems guide clinicians to choose certain codes over others.
The lack of uniformity in the use of ICD9 codes, for instance in assigning a diagnosis of pneumonia, may be ameliorated by grouping diagnoses into broader syndromic surveillance categories, as was done for this report. It is notable that the relatively non-specific diagnosis of "cough" accounted for more than half of LRI encounters studied here. At this time, we have no simple way to measure directly the accuracy of coding or of directly assessing the effect of syndromic groupings. No standard syndromic grouping is yet in wide use for surveillance purposes. The provisional grouping used in this report was developed by the U.S. Department of Defense for its own specific needs but appears to have worked reasonably well with the ambulatory care data described here. Because a small number of relatively non-specific codes (cough, pneumonia, bronchitis) account for the large majority of episodes of lower respiratory infection, it is likely that most syndrome groupings will yield very similar results. Although the inclusion of symptoms like cough clearly reduces specificity, we believe this is outweighed by the gain in sensitivity. This tradeoff is discussed in more detail below.
In terms of completeness, an automated system like the one we describe here is typically as complete for ambulatory encounters as the records that clinicians maintain. Our belated discovery that some encounters of interest were missing from the analysis dataset adds an important cautionary note, however, about the potential problems in adapting data designed for one purpose to another one. We believe the omission of emergency room visits has a minor impact on the total number of events, since their number is small in relation to the total number of ambulatory visits. However, surveillance based in emergency rooms is also of great value. We see the system described here as being complementary to emergency room based systems. A potential advantage of assessment of office visits is the possibility that it will provide an earlier signal than will an emergency room based detection system if the condition of interest begins with symptoms that don't warrant emergency room care.
The use of data from an automated medical record system in a health care environment linked to individuals' insurance coverage provides an additional reason to believe the data are reasonably complete, since individuals have a strong financial incentive to receive their care from providers whose clinical encounter information is reported to the insurer for reimbursement purposes. It may also be possible to make similar use of diagnoses contained in automated billing data, rather than automated medical records. Although most current administrative systems include time lags that diminish their utility, the development of on-line transaction processing between clinicians and payers may reduce or eliminate that deficiency. All medical records or claims based systems depend, of course, on individuals' bringing the event to clinicians' attention. Such systems provide no information about the large number of illnesses that resolve without formal contact with the medical system.
Even large health plans typically include only a portion of individuals in a community. Although the number of individuals may be sufficient for surveillance purposes, it will be important to assess the degree to which the covered population resembles the entire population. Insured populations are likely to be adequate for many conditions of interest, especially if one adjusts for major determinants of illness.
An additional advantage of using automated data from health plans is the ability to know the exact size, composition, and residence location of the source population. Although we limited our characterization of the population to age and sex, it is also possible to use more detailed information about disease history, for instance to characterize the burden of illness among individuals with specific chronic diseases. Locating clusters of illness should have great utility for identifying and remedying localized disease sources; these might be locations, such as day care facilities, where infection is transmitted person to person, or areas in which there are environmental sources of infection, such as a contaminated water supply. The geographical information available to health plan is primarily useful, of course, for conditions whose source is near individuals' homes.
Primary medical records will include multiple encounters within a single episode of infection for some patients. The decision to define a new episode on the basis of six weeks free from any LRI coded encounter was based on a combination of clinical experience and the pattern of repeated encounters. For surveillance systems to be comparable, the classification of encounters into episodes of illness is an important methodological problem that deserves additional attention. It is possible, for instance, that the pattern of repeat visits might change during a cluster of illness. Although the distribution of LRI visit intervals supports a six week disease free interval to become eligible for a second episode, other cutoffs might also have been chosen.
It seems reasonable to assume that if similar patterns of illness in time and space are observed in other systems (such as specific disease surveillance systems, hospital discharge records, or other large ambulatory care record systems), then this provides some degree of criterion validity for the data presented here. The striking similarity in seasonal variation between our LRI episodes and the national experience with pneumonia and influenza deaths provides one measure of assurance that our system identified relevant events. We also compared the CDC's pneumonia and influenza death data for Boston to our experience, but the number of reported deaths was too small for meaningful seasonal patterns to emerge. Additional support for the utility of medical record surveillance information comes from comparison of our data to that collected by the National Ambulatory Medical Care Survey (NAMCS) , which uses multistage sampling of ambulatory care physicians, requiring participants to report a random sample of patients seen in a randomly assigned week. For the period 1990 to 1996, the estimated rate for lower respiratory infection office visits was 74.2/1000 population per year, based on about 40,000 sampled records per year nationally. This estimate is reasonably close to our observed rate of 93/1000. The difference between the two rates may be due in part to sampling variation (principally arising from the smaller NAMCS sample), differences in the age/comorbidity profiles of the populations, greater sensitivity of the HVMA sample because it included telephone encounters, and lack of specificity of the cough diagnosis. In any event, the difference, even if real, should not seriously interfere with the utility of this syndromic surveillance system to identify overall disease trends or provide early warning of illness clusters.
Similarly, the age and sex distribution of these cases is consistent with our knowledge of the epidemiology of lower respiratory tract infection. Others, using data from an office practice, have shown an early childhood peak at approximately one year, also with higher rates in males of approximately 25% . The increasing rate with age among adults has been widely recognized, along with an overall male predominance [11–13]. Some studies have reported either a smaller difference between men and women among younger adults , or an excess among younger adult women , as observed in our population. Our data do not distinguish between actual differences in disease incidence by age and sex, and differential ascertainment, either because of differences in likelihood of seeking health care or differences in the way clinicians code encounters for men and women.
In order to distinguish signals of interest from background occurrences, we believe it will be necessary to develop statistical methods to identify unusual clusters that deserve further attention. The volume of data acquired is so large that it is impractical to perform manual daily inspection of data from a large geographically dispersed population. This is especially important since there were only twelve lower respiratory infection syndrome clusters each year of more than approximately five events occurring in a single day among health plan members residing in a single census tract (authors' unpublished data). The specific number of events required to be included in the twelve most extreme clusters depended on the number of health plan members in a census tract, as well as the month of the year, and the day of the week.
The fact that relatively few events occurred on any particular day in any census tract supports our inclusion of the "cough" diagnosis in the syndrome definition, in order to improve the sensitivity of our case finding. Although this is a non-specific diagnosis, and it accounted for a majority of all events, the total number of these was not so large that it compromised the utility of this particular surveillance system. An enhancement that may be useful in automated medical record systems, but not in claims based systems, is to require fever (measured value, not ICD9 code) to be part of the definition of a lower respiratory infection. This would presumably preserve sensitivity for conditions like anthrax, and also reduce the number of false positive clinical events.
To the extent this surveillance method proves useful, it will be worthwhile to extend it to other conditions that cluster in the areas of residence of affected individuals. Within infectious diseases, these might include diseases spread by airborne dissemination in residential areas, by contaminated foods or water distributed to residents of a neighborhood, by insect or other animal vectors, or by person-to-person transmission in households (secondary spread). Specific infection syndromes of interest, in addition to lower respiratory infection, include upper respiratory infection, gastrointestinal disease, neurologic disease, and rash. It may also be useful for other conditions that may be clustered in time or space, such as injuries.
We conclude that as automated ambulatory care record systems become more widely available, they can assume an important, currently unfilled, role in disease surveillance. Such systems are less prone to undercounting than traditional public health reporting systems, and they are less resource intensive than traditional sentinel surveillance systems. These data can serve several different purposes, including informing clinicians of conditions that are prevalent in their communities, providing detailed and timely information to health plans that need to allocate scarce resources, and to public health programs to allow early recognition and response to changing disease patterns. Suitably de-identified electronic data could be provided to public health systems in a format consistent with the emerging National Electronic Disease Surveillance System (NEDSS) standards . Inclusion of such reporting capability, under clinicians' and health systems' control, in commercial medical record systems is likely to be an inexpensive way to provide the required data in the most usable form. The timely use of automated diagnosis information, especially with cluster detection algorithms, may be a valuable resource for supplementing current infectious disease surveillance systems.