Clinical validity of a population database definition of remission in patients with major depression
- Antoni Sicras-Mainar1Email author,
- Milagrosa Blanca-Tamayo†2,
- Laura Gutiérrez-Nicuesa†3,
- Jordi Salvatella-Pasant†3 and
- Ruth Navarro-Artieda†4
© Sicras-Mainar et al; licensee BioMed Central Ltd. 2010
Received: 7 July 2009
Accepted: 11 February 2010
Published: 11 February 2010
Major depression (MD) is one of the most frequent diagnoses in Primary Care. It is a disabling illness that increases the use of health resources. Aim: To describe the concordance between remission according to clinical assessment and remission obtained from the computerized prescription databases of patients with MD in a Spanish population.
Design: multicenter cross-sectional. The population under study was comprised of people from six primary care facilities, who had a MD episode between January 2003 and March 2007. A specialist in psychiatry assessed a random sample of patient histories and determined whether a certain patient was in remission according to clinical criteria (ICPC-2). Regarding the databases, patients were considered in remission when they did not need further prescriptions of AD for at least 6 months after completing treatment for a new episode. Validity indicators (sensitivity [S], specificity [Sp]) and clinical utility (positive and negative probability ratio [PPR] and [NPR]) were calculated. The concordance index was established using Cohen's kappa coefficient. Significance level was p < 0.05.
133 patient histories were reviewed. The kappa coefficient was 82.8% (confidence intervals [CI] were 95%: 73.1 - 92.6), PPR 9.8% and NPR 0.1%. Allocation discrepancies between both criteria were found in 11 patients. S was 92.5% (CI was 95%: 88.0 - 96.9%) and Sp was 90.6% (CI was 95%: 85.6 - 95.6%), p < 0.001. Reliability analysis: Cronbach's alpha: 90.6% (CI was 95%: 85.6 - 95.6%).
Results show an acceptable level of concordance between remission obtained from the computerized databases and clinical criteria. The major discrepancies were found in diagnostic accuracy.
Major depression (MD) is one of the most frequent diagnoses in Primary Care (PC) and in the general population . It is a disabling illness that alters patients' quality of life and causes an increase in the use of health resources [2–6]. It is important to define the therapeutic objectives for MD , and more experts are claiming that reaching remission should be the main objective since residual symptoms prolong psychosocial dysfunction and can cause higher rates of recurrence [8–10]. In this respect, antidepressants (AD) constitute the pillar of pharmacological treatment in order to achieve the sustained remission of symptoms [11, 12].
On the other hand, some prospective naturalistic studies are carried out in order to explore health problems using computerized databases; because key data in these databases are prescriptions or pharmacy claims, and because there is no specific clinical data recorded in these databases it is usually not possible to obtain quality data to measure remission, thus necessitating "approximate definitions". No consensus currently exists regarding a definition of an approximation for remission. However, definitions exist for depression episode: usually, a new episode for depression is defined as having a new antidepressant prescription with no antidepressant prescription in the previous 6 months . From this, it can be derived that if a patient stops treatment and has no new prescription during the following 6 months, he/she has completed a depression episode. If he/she starts an antidepressant after that, it will be a new episode. This can be used in a model for remission.
Nevertheless, this "approximate definition" can cause several problems. On one hand, there may be other motives for discontinuing the prescription other than the patient being in remission (failure to observe treatment, adverse side effects, etc.). On the other hand, persistence of treatment with AD should not always be interpreted as absence of remission, as it is indicated in all patients for 6-9 months after achieving remission, and for a longer period in selected patients at risk of relapse .
Thus, it is pertinent to prove the validity of this construct, so as not to fall into classification bias , especially when there is no evidence at present in the revised literature. This study is an attempt to complement a previously started line of investigation on remission in patients with MD (results pending publication). The purpose of this study is to describe the validity (reliability, concordance) and the clinical utility of the remission measurement obtained from the direct review of patients' histories (reference criteria) and the one taken from the database registry (by approximation) in MD patients in a Spanish population.
Design and population under study
The design was cross-sectional and multi-centric. The population under study was comprised of people from six primary care facilities under the management of Badalona Serveis Assistencials SA, which provide health care for some 107,208 people, out of whom 15.9% are over 64. The diagnosis of MD was made according to International Classification of Primary Care (ICPC-2; code P76) . The following scales were used for screening/diagnostics: Goldberg (co-existence of psychiatric morbidity), Hamilton Depression (severity of depression symptoms) and Yesavage Geriatric Depression . Patients were selected according to the following criteria: a) over 17 years old, b) who have initiated a first MD episode with a treatment prescribed between January 2003 and March 2007 (determined for diagnosis by clinical interview of treating clinicians according to the ICPC-2), c) and demonstrate a period of at least six months without depression prior to the MD episode, d) prescription should follow the minimum required treatment criteria  (> 60 days of AD treatment after the first prescription) and e) a follow-up of at least 18 months (12-month study period and 6-month follow-up to assign study sub-group). Two sub-groups were considered: patients in remission and patients not in remission.
Definition of remission by approximation and according to reference criteria
Patients were considered to be in remission when they no longer required new prescriptions of AD for at least six months after completing treatment of the first episode; patients who could only interrupt treatment for a period less than six months were not considered in sustained remission . As reference criteria, remission was determined based on the assessment of the database clinical course text by a psychiatry specialist. The sample was selected through simple random sampling stratified for age and gender. Sample size was calculated assuming an expected remission prevalence of 54%, a random error of 5% (bilateral) and an accuracy of 8.5%.
Validity and utility measure indicators
Validity indicators were Sensitivity [S], specificity [Sp], positive and negative predictive value [PPV] and [NPV]) and clinical utility (positive and negative probability ratio [PPR] and [NPR]. S is the probability of obtaining a positive result when the subject meets the approximation definition. Sp indicates the probability of obtaining a negative result when the subject does not meet the approximation definition. PPV is the probability of meeting the reference criteria when the approximate remission variable is positive. NPV is the probability of not meeting the reference criteria when remission by approximation is negative. The probability ratios of positive or negative likelihood-ratios indicate up to what point a certain result in a diagnostic test will increase or diminish pre-test probability .
Confidentiality of the information and statistical analysis
Data confidentiality was respected at all times, according to the Personal Data Protection Act (Ley de protección de Datos de Carácter Personal [LOPD]); this study was approved by the Ethics Committee of Gol i Gurina Foundation. Concordance between both remission measurements was evaluated, itemized by systematic error and random error. Concordance index was calculated using Cohen's kappa coefficient, correlation coefficient was calculated for random error and McNemar test for systematic error . Reliability analysis was established using Cronbach's alpha. Confidence intervals [CI] of 95% were calculated. SPSSWIN version 12 was used, establishing a statistic significance of p < 0.05.
Validity of remission by approximation in major depression
Remission by Approximation
N = 133
Validity of the measurement
88.0 - 96.9
85.6 - 95.6
Positive predictive value
89.6 - 97.8
Negative predictive value
83.6 - 94.2
3.0 - 11.9
4.4 - 14.4
85.6 - 95.6
Area under the curve
86.5 - 96.1
49.7 - 66.5
73.1 - 92.5
Weighted kappa (Cohen)
73.1 - 92.6
Positive Probability Ratio
Negative Probability Ratio
Our study deals with the clinical validity of the measurement of remission in patients with major depression, as obtained from population databases. It is worth pointing out that the organization of care, through territorial-based assignment of teams and the increasing computerization of its centres provides a suitable environment to carry out these types of studies in a clinical practice environment [18, 19]. Thus, the quality of the measurements is a basic aspect in achieving an efficient health system, and we must be sure in any kind of biomedical research that measurement error is reasonably small. In regulated environments, such as clinical testing in the development of pharmaceuticals, data quality and especially measurement procedures require attention for reasons related to both ethics and efficiency .
The results of our study reveal an acceptable degree of concordance between the remission by approximation and the variable obtained from patient history reviews (reference criteria). The main cause of discrepancy between them is diagnostic accuracy by the PC team. In this respect, several authors have affirmed the existing variability between the PC diagnosis and the one by mental health specialists [20–22]. There seems to be certain difficulty in identifying affective and adaptive disorders, which has an impact on remission measurement. There are several possible motives affecting MD diagnosis. On one hand, it may be affected by shortcomings in the training of the professionals, health care pressure in their consults and the brief amount of time they can devote to each patient.
Furthermore, it can also affect the classification system used (ICPC-2), which can condition the diagnosis of these patients. In this regard, it must be taken into consideration that these differences between both disorders make differential diagnoses difficult, as PC doctors usually tend to view disorders from a dimensional rather than a categorical perspective, probably because it best fits the needs of their consultation. Nevertheless, there are studies that reveal a certain tendency of over-diagnosis of depression disorders, while others point out an under-diagnosis of MD and anxiety in PC [23, 24]. The interpretation of the measurement of remission in our study may be in agreement with the interpretation of probability coefficients (PPR and NPR) as the measurement of clinical utility. Thus, the measurement of remission by approximation can generate moderate changes from pre-test to post-test probability (reference criteria).
Possible limitations in the study demand caution in generalization of results. The study presented is framed within the studies known as technique validation in clinical effectiveness conditions; therefore, the bias assumed in research is typical of a study with a cross-sectional model, oriented towards validation of diagnostic criteria. Moreover, sample size can have a high random component for precision in measurement, and the study was carried out based on a specialist's diagnostic criteria; therefore, the results obtained may be influenced by the professional's characteristics.
Results show acceptable concordance in the remission obtained from the computerized databases. The largest discrepancies are in diagnostic accuracy.
International Classification of Primary Care
Predictive Positive Value
Negative Predictive Value
Positive Probability Ratio
Negative Probability Ratio.
Thanks to the different professionals of the centres for their constant input of data on a daily basis. Without their contribution, this study could not have been conducted
- ESEMeD, Haro JM, Palacin C, Vilagut G, Martinez M, Bernal M, Luque I, Codony M, Dolz M, Alonso J y el Grupo ESEMeD-España: Prevalencia de los trastornos mentales y factores asociados: resultados del estudio ESEMeD-España. Med Clin (Barc). 2006, 126: 445-51. 10.1157/13086324.View ArticleGoogle Scholar
- Sobocki P, Lekander I, Borgstrom F, Strom O, Runeson B: The economic burden of depression in Sweden from 1997 to 2005. Eur Psychiatry. 2007, 22: 146-52. 10.1016/j.eurpsy.2006.10.006.View ArticlePubMedGoogle Scholar
- Shvartzman P, Weiner Z, Vardy D, Friger M, Sherf M, Biderman A: Health services utilization by depressive patients identified by the MINI questionnaire in a primary care setting. Scand J Prim Health Care. 2005, 23: 18-25. 10.1080/02813430510018383.View ArticlePubMedGoogle Scholar
- Sobocki P, Ekman M, Agren H, Krakau I, Runeson B, Martensson B, Jonsson B: Health Related Quality of Life measured with EQ-5D in patients treated for depression in primary care. Value Heath. 2007, 10: 154-60.Google Scholar
- Goetzel RZ, Hawkins K, Ozminkowski RJ, Wang S: The health and productivity cost burden of the "top 10" physical and mental health conditions affecting six large U.S. employers in 1999. J Occup Environ Med. 2003, 45: 5-14. 10.1097/00043764-200301000-00007.View ArticlePubMedGoogle Scholar
- Sicras Mainar A, Navarro Artieda R, Rejas Gutiérrez J, Blanca Tamayo M, Serrat Tarrés J, Llopart López JR: Comorbilidad y coste ambulatorio asociado a los pacientes con trastornos depresivos en un ámbito poblacional. Farm Hosp. 2007, 31: 101-5. 10.1016/S1130-6343(07)75720-3.View ArticlePubMedGoogle Scholar
- Keller MB: Past, present and future directions for defining optimal treatment outcome in depression: remission and beyond. JAMA. 2003, 289: 3152-60. 10.1001/jama.289.23.3152.View ArticlePubMedGoogle Scholar
- Ansseau M, Demyttenaere K, Heyrman J, Migeotte A, Leyman S, Mignon A: Objective: remission of depression in primary care The Oreon Study. Eur Neuropsychopharmacol. 2009, 19: 169-76. 10.1016/j.euroneuro.2008.10.003.View ArticlePubMedGoogle Scholar
- Sobocki P, Ekman M, Agren H, Runeson B, Jönsson B: The mission is remission: health economic consequences of achieving full remission with antidepressant treatment for depression. Int J Clin Pract. 2006, 60: 791-8. 10.1111/j.1742-1241.2006.00997.x.View ArticlePubMedGoogle Scholar
- Zimmernan M, McGlinchey JB, Posternak MA, Friedman M, Attiullah N, Boerescu D: How should remission from depression be defined?. The depressed patient's perspective. Am J Psychiatry. 2006, 163: 148-50. 10.1176/appi.ajp.163.1.148.View ArticleGoogle Scholar
- National Institute for Clinical Excellence: Depression: management of depression in primary and secondary care. Clinical Guideline 23. 2004. Amended. 2006, London: NICEGoogle Scholar
- Wade AG, Despiégel N, Danchenko N, Saragoussi D, Verpillat P: How does remission in major depressive disorder impact resource use in UK does: A General Practitioner Research Database (GPRD) based study. 23rd International Conference on Pharmacoepidemiology & Therapeutic Risk Management (ISPE), Québec City Canada, 19-22. 2007, AugustGoogle Scholar
- Kupfer DJ: Long-term treatment of depression. Clin Psychiatry. 1991, 52 (Suppl): 28-34.Google Scholar
- White E: Design and interpretation of studies of differential exposure measurement error. Am J Epidemiol. 2003, 157: 380-7. 10.1093/aje/kwf203.View ArticlePubMedGoogle Scholar
- Lamberts H, Wood M, Hofmans-Okkes IM, eds: The International Classification of Primary Care in the European Community. With a multi-language layer. 1993, Oxford: Oxford University Press
- MacMahon B, Trichopoulos D: Epidemiology: Principles and Methods. 1996, Boston: Lippincott Williams & Wilkins, 2Google Scholar
- Carrasco JL, Jover L: Métodos estadísticos para evaluar la concordancia. Med Clin (Barc). 2004, 122 (Supl 1): 28-34. 10.1157/13057543.View ArticleGoogle Scholar
- Campillo-Artero C: Fracturas en la informatización de los servicios de salud. Gac Sanit. 2008, 22: 371-7. 10.1157/13125361.View ArticlePubMedGoogle Scholar
- Ludwick DA, Doucette J: Adopting electronic medical records in primary care: lessons learned from health information systems implementation experience in seven countries. Int J Med Inform. 2009, 78: 22-31. 10.1016/j.ijmedinf.2008.06.005.View ArticlePubMedGoogle Scholar
- Landa-González N, Goñi-Sarriés A, García-de-Jalón-Aramayo E, Lizasoain-Urra E: Concordancia en el diagnóstico entre atención primaria y salud mental. Aten Primaria. 2008, 40: 285-9. 10.1157/13123680.View ArticlePubMedGoogle Scholar
- Herrán A, López-Lanza JR, Ganzo H, Cadiñanos A, Díez-Manrique JE, Vázquez-Barquero JL: Derivación de los pacientes con enfermedad mental desde Atención Primaria a Salud Mental. Actas Esp Psiquiatr. 2000, 28: 13-21.PubMedGoogle Scholar
- Klinkman MS, Coyne JC, Gallo S, Schwenk TL: False positives, falses negative, and the validity of the diagnosis of major depression in primary care. Arch Fam Med. 1998, 7: 451-61. 10.1001/archfami.7.5.451.View ArticlePubMedGoogle Scholar
- Thompson C, Ostler K, Baker N, Kinmonth A: Dimensional perspective on the recognition of depressive symptoms in primary care. Br J Psychiatry. 2001, 179: 317-23. 10.1192/bjp.179.4.317.View ArticlePubMedGoogle Scholar
- Gilbody S, Richards D, Barkham M: Diagnosing depression in primary care using self-completed instruments: UK validation of PHQ-9 and CORE-OM. Br J Gen Pract. 2007, 57: 650-2.PubMedPubMed CentralGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/10/64/prepub