The sub-criteria for DSM-IV dementia, derived from a computerised algorithm were plausibly distributed in dementia cases (both those identified by clinician diagnosis, and by the 10/66 dementia algorithm) and non-cases. Over 80% of cases met the key A1, A2, B1 and B2 sub-criteria, the sole exception being that of A1, met by 71% of 10/66 algorithm cases. The requirement that each of the A1, A2, B1 and B2 sub-criteria are met accounts for the relatively low proportion, 58%, of clinically diagnosed dementia cases meeting all elements of the computerised DSM-IV criterion. On the one hand the stringency of this approach ensures high specificity; on the other, misclassification error will tend to accumulate reducing the sensitivity of the overall DSM-IV algorithm. We acknowledge a potential weakness of our approach in that the clinical diagnosis, the computerised DSM-IV diagnosis and the 10/66 diagnosis were all derived from elements of the same structured clinician assessment. This may have led to an overestimate of their concordance. Formal validation would have required independent clinical assessments of survey participants by expert clinicians, applying DSM-IV criteria but without relying upon our survey assessments to do so; however, this approach would still not address the problem of unreliability in clinician application of the DSM-IV criteria.
We have intentionally avoided describing any of the diagnostic outcomes in this study as a 'gold standard', preferring instead to explore the underlying dementia construct and its measurement through the relationships observed between the three outcomes; the 10/66 dementia and DSM-IV dementia survey diagnoses and the clinician diagnosis. In reality, each has their strengths and their weaknesses. The DSM-IV criteria are the dominant paradigm in current research and clinical practice, but have been critiqued both for the primacy accorded to memory impairment (a necessary component, which is not, however, a prominent early feature of many dementias, for example vascular dementia and frontotemporal dementia) and for the lack of specificity of the secondary cognitive criteria (aphasia, apraxia, agnosia, disturbance in executive functioning) which are arguably of more direct relevance to stroke than to dementia . We are optimistic that our 10/66 dementia diagnosis provides a robust alternative given its careful cross-cultural development and calibration ; however, it has not previously been validated in a community setting. Local clinician diagnoses have strong ecological validity, reflecting as they do local clinical practice. Nevertheless, lack of standardisation may lead to problems with reliability and validity. In the absence of an unimpeachable gold standard, our approach has therefore been closer to that of construct validation as described by Cronbach and Meehl  than to criterion validation.
What then have we learnt from this exercise regarding the comparative utility of the 10/66 and DSM-IV diagnoses? Comparison of the characteristics of 10/66 Dementia Algorithm cases confirmed and not confirmed by our computerised DSM-IV algorithm does raise some questions regarding the sensitivity of the DSM-IV criterion, in that many unconfirmed 10/66 cases were still grossly impaired compared with the controls who met neither set of criteria. Furthermore, in Cuba, clinician diagnoses matched more closely to our 10/66 dementia category than to the more restrictive DSM-IV computerised criterion. In principle, the clinician diagnoses were made using the DSM-IV criterion. Clinicians may be less stringent in the thresholds they set for clinical significance, and less rigorous than a computerised system in the application of the algorithm. In either event, our Cuban data tends to suggest that clinically relevant dementia may be prevalent beyond the confines of the narrowly defined DSM-IV criterion when, as in our study, it is strictly applied using a fully-operationalised computerised algorithm. One of the few population-based studies to examine this issue directly, the Canadian Study of Health and Aging  reported a prevalence of 20.9% for those aged 65 and over according to clinical consensus compared with 13.7% according to DSM-IV criterion. Mild cases were selectively excluded by the DSM-IV criterion. We found the same. This may have important implications for previous assessments of the global prevalence of dementia [19, 20], relying as they do, to a large extent, upon studies that have used the DSM-IV criterion. The more inclusive 10/66 dementia diagnosis may help to establish the true population burden of the dementia syndrome. Conversely, our 10/66 Dementia algorithm may be overdiagnosing dementia. Our earlier pilot study to develop and test the 10/66 algorithm against a DSM-IV criterion, suggests that this is possible; we achieved very high sensitivity (94%), but with a 6% false positive rate in low education controls and 3% FPR in high education controls. We will examine these issues carefully in each of the 10/66 population-based study centres. The relative validity of the two computerised diagnoses used in the 10/66 survey can probably best be settled when their predictive validity is addressed in the forthcoming incidence phase of the 10/66 investigations. Those with dementia would be expected to have declined further, or to have died. Stability or improvement in cognitive function and functional status would argue against the validity of the case definitions.