- Research article
- Open Access
- Open Peer Review
Use of the Pediatric Symptom Checklist for the detection of psychosocial problems in preventive child healthcare
© Reijneveld et al; licensee BioMed Central Ltd. 2006
- Received: 13 February 2006
- Accepted: 27 July 2006
- Published: 27 July 2006
Early detection and treatment of psychosocial problems by preventive child healthcare may lead to considerable health benefits, and a short questionnaire could support this aim. The aim of this study was to assess whether the Dutch version of the US Pediatric Symptom checklist (PSC) is valid and suitable for the early detection of psychosocial problems among children.
We included 687 children (response 84.3%) aged 7–12 undergoing routine health assessments in nine Preventive Child Health Services across the Netherlands. Child health professionals interviewed and examined children and parents. Before the interview, parents completed an authorised Dutch translation of the PSC and the Child Behavior Checklist (CBCL). The CBCL and data on the child's current treatment status were used as criteria for the validity of the PSC.
The consistency of the Dutch PSC was good (Cronbach alpha 0.89). The area under the ROC curve using the CBCL as a criterion was 0.94 (95% confidence interval 0.92 to 0.96). At the US cut-off (28 and above), the prevalence rate of an increased score and sensitivity were lower than in the USA. At a lower cut-off (22 and above), sensitivity and specificity were similar to that of the US version (71.7% and 93.0% respectively). Information on the PSC also helped in the identification of children with elevated CBCL Total Problems Scores, above solely clinical judgment.
The PSC is also useful for the early detection of psychosocial problems in preventive child healthcare outside the USA, especially with an adjusted cut-off.
- Psychosocial Problem
- Dutch Version
- Elevated Score
- Dutch Child
- Pediatric Symptom Checklist
Early detection and treatment of psychosocial problems may lead to considerable health benefits. Psychosocial problems have a high prevalence rate and lead to high costs of disease . They also cause substantial restrictions in daily functioning in later life and are the major cause of long-term work disability in young adults . Only a minority of children with psychological or psychosocial problems are under treatment [2–4]. If untreated, problems are likely to persist in later life and can lead to serious limitations in daily functioning [2, 5]. Research has shown that early detection and treatment improves these children's prognosis substantially [6, 7], but a complete analysis of its cost effectiveness has yet to be carried out.
The community child health service is an ideal setting for the early detection of psychosocial problems among children as routine health examinations are provided through it for the entire population, as a standardised part of preventive child healthcare (PCH). In the Netherlands, municipalities are obliged by law to guarantee proper access to this type of care, free of charge.
However, the predictive value of early detection of psychosocial problems by PCH is still too low [3, 4]. For instance, Brugman et al. show that even though Dutch PCH identifies psychosocial problems in 25% of all children of school age, they miss 43% of the children with a clinical score on the Child Behavior Checklist (CBCL) . Similarly, Murphy et al. reported that paediatricians had identified psychosocial problems in less than half of the children with elevated scores on the Pediatric Symptom Checklist (PSC) or the Child Global Assessment Scale .
The PSC is a 35-item parent-completed questionnaire that supports the identification of psychosocial problems by paediatricians [8–14]. It takes less than 5 minutes to complete and score, and reflects the parent's impression of his or her child's psychosocial functioning. Its validity has been demonstrated in various paediatric settings in the USA, nationally  in inner-city children , in Hispanic children [11, 12] and in children of substance-abusing parents . Moreover, the PSC has recently been used as an outcome measure in the assessment of interventions to reduce the impact of trauma . Given its good validity and applicability in US community child health services, the PSC is a likely candidate for use in other countries with similar systems of preventive child healthcare, such as the Netherlands.
The aim of this study was to assess the test properties of the Dutch version of the PSC and determine whether it would be suitable for and contribute to the early detection of psychosocial problems in children aged 7–12 by PCH.
This study is based on a community sample of children for whom PSC and CBCL data are available, and data on the identification and management of psychosocial problems by CHPs.
The sample was obtained using a two-stage selection procedure. In the first stage, a national sample from 9 of the 41 Dutch Preventive Child Health Services was taken. In the second stage, each Service provided a sample of children aged 7–12 who were invited for routine well-child examinations. We aimed at a sample size of 700 respondents for evaluation, as earlier studies [15, 16] demonstrated that short questionnaires used in PCH settings allow for an area under the ROC curves (AUC) of about 0.90 with a clinical CBCL Total Problems score as criterion. A sample of 700 suffices to estimate this AUC with a 95% confidence interval of +/- 0.02.
Of the total sample of 815 eligible children, 687 participated and 674 provided complete data on both questionnaires (84.3% and 82.7% of the original sample, respectively). Both groups were representative of the total sample regarding age and gender, but non-response was higher for children of immigrant/minority origin (27.4% vs. 12.2%). Analyses were restricted to children with complete data for both questionnaires to make interpretation easier.
The data were collected according to a standardised procedure during routine well-child examinations, from September 2004 to July 2005. The study was performed in compliance with the Helsinki Declaration . The design of the study was approved by the local TNO Medical-Ethical Committee and includes verbal informed consent by parents.
The PSC  and the CBCL [18, 19] were mailed to children, along with the standard invitation for the preventive health assessment. Before attending the assessments, parents completed the questionnaires, placed them in sealed envelopes and gave them to the CHPs, who in turn passed them on to the researchers without opening them (in contrast with routine use, where the CHP would partially base the interview on the information from the PSC). The CHP interviewed each child and its parents regarding mental health and background, and examined each child. After each assessment, the CHP answered the following question: 'Does the child have a psychosocial problem, at this moment?' (yes, no) and scored its severity (mild, moderate or severe) and the type of problems identified using a pre-coded list. Children who only had risk indicators for the development of psychosocial problems, such as having parents with psychiatric problems or other family problems, had to be coded as having no problems.
The PSC was translated following the procedure proposed by Guillemin et al. . Firstly, the original US English version of the questionnaire was translated into Dutch by three certified translators working independently of each other. Secondly, three further certified translators each translated one Dutch translation back into US English. The resulting US English versions were compared to the originals and all discrepancies were discussed by three researchers (SAR, MRC and AGCV) who spoke both Dutch and English. Discrepancies were also discussed with the developers of the PSC, Dr J.M. Murphy and Dr M.S. Jellinek, especially where items raised questions as to their intended meaning. The PSC consists of 35 items that are rated as never, sometimes or often present (0, 1 and 2, respectively). Item scores are summed; we dichotomised at 0–27 vs. 28–70, following the US cut-off .
The CBCL was used to assess parents' reports of the behavioural and emotional problems of their children over the preceding six months. Its (good) reliability and validity has been established [18, 19]. We used only the 120 problem items from the CBCL and computed scores for two broad-band groups of syndromes designated as Internalising and Externalising, and a Total Problems score. Children were also allocated to a normal range or a clinical range, using the 90th percentile of the Dutch normative sample as the cut-off .
In the analysis we assessed the psychometric properties of the PSC and its added value in identifying psychosocial problems. Regarding psychometric properties, we first computed its internal consistency and examined the fit between the scale structure and the observed data using confirmatory factor analysis (CFA) with structural equation modelling. Next, we assessed the validity of the PSC using dichotomised CBCL scores (Total Problems score and Internalising/Externalising scales) and referral by the CHP due to psychosocial problems as criteria. Finally, we assessed whether mean PSC scores differed with the children's background.
Regarding the added value of the PSC in identifying psychosocial problems, we assessed the odds of identification of mental health problems (i.e. a clinical CBCL Total Problems score) using an elevated score on the PSC. This was repeated with adjustment for social and demographic risk indicators known to the CHP that might have helped in the identification of psychosocial problems [3, 4]. Regarding social and demographic risk indicators, we retained children with missing data in the logistic regression models by creating separate dummies for the missing category of each variable.
All analyses were done with SPSS 12.0 for Windows , except the CFA, which was done with Amos 5 . All analyses were repeated for boys and girls separately. Results for these subgroups are provided only if they differed in a statistically significant way (p < 0.05).
Demographic characteristics of the participating children, and mean Pediatric Symptom Checklist scores for selected socio-demographic groups (n = 674)*.
No. (%) *
Highest parental education
Only primary school (=8 years)
Lower vocational (=max. 12 years)
Higher vocational (=max. 16 years)
University/higher professional (17 years and over)
Parental employment status
No paid employment
One parent with paid employment
Two parents with paid employment
Scores on PSC and CBCL
Scores on the PSC and CBCL Total Problems, Internalising and Externalising scales, for all children and by gender (mean, standard deviation, range).
Total (n = 674)
Boys (n = 327)
Girls (n = 347)
Numbers and percentages of children with elevated scores on the PSC and CBCL Total Problems, Internalising and Externalising scales, and of children currently under treatment for psychosocial problems, for all children and by gender.
Total (n = 674)
Boys (n = 327)
Girls (n = 347)
Currently under treatment for psychosocial problem
Sensitivity (Sens.), specificity (spec.), AUC and positive and negative predictive value (PPV, NPV) for the PSC at cut-off 27, using CBCL Total, Internalising and Externalising Problems, and under treatment for psychosocial problems as criteria (n = 674).
Cut-off 0–27 vs. 28+
Cut-off 0–21 vs. 22+
Currently under treatment
Differences in scores by background characteristics
Mean PSC scores were higher for boys, for children from minority backgrounds, single-parent families and unemployed families (Table 1, final columns).
Finally, we examined the degree to which information from the PSC contributed to the diagnosis of psychosocial problems as measured by the CBCL over and above the clinical opinion of the child health physician without knowledge of the PSC. This yielded an odds ratio of 21.3 (95% confidence interval 8.7 to 52.2), with the only predictive background characteristic being family composition. Using the alternative PSC cut-off of 22+ yielded slightly higher odds ratios.
Parent opinion of the PSC
A large majority of parents completed the PSC fully (91.1%) and no parent missed more than 3 items. However, 20% of parents made critical remarks about the PSC, mainly concerning lack of fit between questions and answer categories (7%) and unclear questions (5%).
This study assessed the psychometric qualities of the Dutch version of the PSC and whether it is suitable for and contributes to the early detection of psychosocial problems among Dutch children aged 7–12 by PCH. Results reveal a good internal consistency and validity using the CBCL as gold standard. However, lower cut-offs have to be used for Dutch children than for children from the USA because of the Dutch children's on-average lower scores.
Methodological factors are unlikely to have affected these results. In general, the response rate was high (84%). Moreover, we used the CBCL as a criterion, which has been proven to be a valid measure for psychosocial problems. Because of complexity and high costs, structured clinical interviews such as the Diagnostic Interview Schedule for Children were not used as criteria . Doing so may have provided additional information but differences with questionnaire-based information have been shown to be small .
Fit with previous research on PSC and on other questionnaires used in PCH
This first study of the Dutch version of the PSC yielded results on reliability and on validity regarding the CBCL that are very similar to those found in comparable US samples. Jellinek et al. reported a sensitivity of 51.5% and a specificity of 95.4% at a cut-off of 0–27/28+, using the CBCL as a criterion in a sample of 206 children from the USA [9, 25]. These values are very similar to those for a cut-off of 0–24/25+ for the Dutch version, i.e. 53.3% and 97.3% respectively (compare figure). We found the internal consistency of the PSC similar to that found by Jellinek et al.  but the results of our confirmatory factor analyses cast some doubt as to whether it measures a single latent, as did a previous study of Gardner et al. .
We found much lower mean scores on the PSC than have been found for comparable US samples. Mean scores on other symptom checklists such as the CBCL are also lower for Dutch children than for children from the USA . Therefore, the Dutch children's lower mean PSC scores probably reflect real differences between these countries in the levels of symptoms reported by parents. This also implies that the cut-off for an elevated score on the PSC should be set lower for Dutch children than for children from the US. At this lower cut-off, the sensitivity and specificity of the Dutch version is similar to that of the US version. Moreover, the test characteristics of the PSC are comparable with or slightly better than those of most questionnaires currently used in Dutch PCH [15, 16, 28].
Finally, we found higher mean PSC scores for boys and for children from single-parent families, similar to those found by Jellinek et al. for children in the USA . We did not find differences in terms of parental education level, in contrast to the findings of Jellinek et al. , but we did find elevated scores among children with unemployed parents, an indicator of familial socioeconomic status that was not studied by Jellinek et al. .
The results of our study imply that the PSC is useful for the early detection of psychosocial problems by PCH, especially if an adjusted cut-off is used. The PSC mostly detects behavioural and emotional problems, which are common in this age group. However, questions on more extreme behaviours such as the abuse of alcohol and drugs are not asked. Screening using the PSC is best carried out as a first step in a two-step process on the way to referral. A relatively low-cut-off can then be used to avoid missing too many cases. In a second step, cases flagged by the PSC should then be assessed by a CHP before making a final decision about referral. Parental responses show that some questions may require revision. In any event, the PSC is a useful aid for the early detection of psychosocial problems that could be considered for use in other countries as well.
This study was financially supported by the Netherlands Health Research and Development Council (ZonMw), grant 2100-0087. The study sponsor had no influence on any aspect of this study, including the decision to submit this paper for publication. We thank Dr Michael S. Jellinek and Dr J. Michael Murphy, of Massachusetts General Hospital, Boston, USA, for their helpful comments.
- Murthy RS, Bertolote JM, Epping-Jordan J, Funk M, Prentice T, Saraceno B, Saxena S: The World health report 2001. Mental health: new understanding, new hope. 2001, Geneva, World Health OrganizationGoogle Scholar
- Verhulst FC, van der Ende J: Factors associated with child mental health service use in the community. J Am Acad Child Adolesc Psychiatry. 1997, 36: 901-909. 10.1097/00004583-199707000-00011.View ArticlePubMedGoogle Scholar
- Brugman E, Reijneveld SA, Verhulst FC, Verloove-Vanhorick SP: Identification and management of psychosocial problems by preventive child health care. Arch Pediatr Adolesc Med. 2001, 155: 462-469.View ArticlePubMedGoogle Scholar
- Reijneveld SA, Brugman E, Verhulst FC, Verloove-Vanhorick SP: Identification and management of psychosocial problems among toddlers in Dutch preventive child health care. Arch Pediatr Adolesc Med. 2004, 158: 811-817. 10.1001/archpedi.158.8.811.View ArticlePubMedGoogle Scholar
- Moffitt TE, Caspi A: Childhood predictors differentiate life-course persistent and adolescence-limited antisocial pathways among males and females. Dev Psychopathol. 2001, 13: 355-375. 10.1017/S0954579401002097.View ArticlePubMedGoogle Scholar
- Durlak JA, Wells AM: Evaluation of indicated preventive intervention (secondary prevention) mental health programs for children and adolescents. Am J Community Psychol. 1998, 26: 775-802. 10.1023/A:1022162015815.View ArticlePubMedGoogle Scholar
- Nelson G, Westhues A, MacLeod J: A meta-analysis of longitudinal research on preschool prevention programs for children. Prevention & Treatment. 2003, 6: 1-32.View ArticleGoogle Scholar
- Murphy JM, Reede J, Jellinek MS, Bishop SJ: Screening for psychosocial dysfunction in inner-city children: further validation of the Pediatric Symptom checklist. J Am Acad Child Adolesc Psychiatry. 1992, 31: 1105-1111.View ArticlePubMedGoogle Scholar
- Jellinek MS, Murphy JM: Screening for psychosocial disorders in pediatric practice. Am J Dis Child. 1988, 142: 1153-1157.PubMedGoogle Scholar
- Jellinek MS, Murphy JM, Little M, Pagano ME, Comer DM, Kelleher KJ: Use of the Pediatric Symptom Checklist to screen for psychosocial problems in pediatric primary care: a national feasibility study. Arch Pediatr Adolesc Med. 1999, 153: 254-260.View ArticlePubMedPubMed CentralGoogle Scholar
- Pagano M, Murphy JM, Pedersen M, Mosbacher D, Crist-Whitzel J, Jordan P, Rodas C, Jellinek MS: Screening for psychosocial problems in 4-5-year-olds during routine EPSDT examinations: validity and reliability in a Mexican-American sample. Clin Pediatr (Phila). 1996, 35: 139-146.View ArticleGoogle Scholar
- Jutte DP, Burgos A, Mendoza F, Ford CB, Huffman LC: Use of the Pediatric Symptom Checklist in a low-income, Mexican American population. Arch Pediatr Adolesc Med. 2003, 157: 1169-1176. 10.1001/archpedi.157.12.1169.View ArticlePubMedGoogle Scholar
- Stein BD, Jaycox LH, Kataoka SH, Wong M, Tu W, Elliott MN, Fink A: A mental health intervention for schoolchildren exposed to violence: a randomized controlled trial. JAMA. 2003, 290: 603-611. 10.1001/jama.290.5.603.View ArticlePubMedGoogle Scholar
- Kelley ML, Fals-Stewart W: Psychiatric disorders of children living with drug-abusing, alcohol-abusing, and non-substance-abusing fathers. J Am Acad Child Adolesc Psychiatry. 2004, 43: 621-628. 10.1097/00004583-200405000-00016.View ArticlePubMedGoogle Scholar
- Vogels T, Reijneveld SA, Brugman E, den Hollander-Gijsman ME, Verhulst FC, Verloove-Vanhorick SP: Detecting psychosocial problems among 5-6 years old children in Preventive Child Health Care: The validity of a short questionnaire used in an assessment procedure for detecting psychosocial problems among children. Eur J Public Health. 2003, 13: 353-360. 10.1093/eurpub/13.4.353.View ArticlePubMedGoogle Scholar
- Reijneveld SA, Vogels AG, Brugman E, Van Ede J, Verhulst FC, Verloove-Vanhorick SP: Early detection of psychosocial problems in adolescents: how useful is the Dutch short indicative questionnaire (KIVPA)?. Eur J Public Health. 2003, 13: 152-159. 10.1093/eurpub/13.2.152.View ArticlePubMedGoogle Scholar
- Word Medical Association Delaration of Helsinki: Ethical principles for medical research involving human subjects. 2006, [http://www.wma.net/e/policy/b3.htm]-Google Scholar
- Achenbach TM: Manual for the Child Behavior Checklist/4-18 and 1991 Profile. 1991, Burlington (VT), University of Vermont, Dept of PsychiatryGoogle Scholar
- Verhulst FC, van der Ende J, Koot HM: Manual for the CBCL/4-18 [in Dutch]. 1996, Rotterdam, Erasmus University/Dept of Child and Adolescent Psychiatry, Sophia Childrens' HospitalGoogle Scholar
- Guillemin F, Bombardier C, Beaton D: Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993, 46: 1417-1432. 10.1016/0895-4356(93)90142-N.View ArticlePubMedGoogle Scholar
- SPSS® for Windows®. 2006, [http://www.spss.com/spss]-Google Scholar
- Structural equation modeling to test telationships. 2006, [http://www.spss.com/amos/]-Google Scholar
- NIMH: Diagnostic Interview Schedule for Children. 1997, Rockville, National Institute of Mental HealthGoogle Scholar
- Boyle MH, Offord DR, Racine YA, Szatmari P, Sanford M, Fleming JE: Adequacy of interviews vs checklists for classifying childhood psychiatric disorder based on parent reports. Arch Gen Psychiatry. 1997, 54: 793-799.View ArticlePubMedGoogle Scholar
- Jellinek MS, Murphy JM, Burns BJ: Brief psychosocial screening in outpatient pediatric practice. J Pediatr. 1986, 109: 371-378.View ArticlePubMedGoogle Scholar
- Gardner W, Kelleher KJ, Pajer KA: Multidimensional adaptive testing for mental health problems in primary care. Med Care. 2002, 40: 812-823. 10.1097/00005650-200209000-00010.View ArticlePubMedGoogle Scholar
- Crijnen AA, Achenbach TM, Verhulst FC: Comparisons of problems reported by parents of children in 12 cultures: total problems, externalizing, and internalizing. J Am Acad Child Adolesc Psychiatry. 1997, 36: 1269-1277. 10.1097/00004583-199709000-00021.View ArticlePubMedGoogle Scholar
- Vogels AGC, Crone MR, Hoekstra F, Reijneveld SA: Three questionnaires to detect psychosocial problems among children aged 7-12 [in Dutch]. 2005, Leiden, TNO Quality of LifeGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/6/197/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.