This study is based on a community sample of children for whom PSC and CBCL data are available, and data on the identification and management of psychosocial problems by CHPs.
Population
The sample was obtained using a two-stage selection procedure. In the first stage, a national sample from 9 of the 41 Dutch Preventive Child Health Services was taken. In the second stage, each Service provided a sample of children aged 7–12 who were invited for routine well-child examinations. We aimed at a sample size of 700 respondents for evaluation, as earlier studies [15, 16] demonstrated that short questionnaires used in PCH settings allow for an area under the ROC curves (AUC) of about 0.90 with a clinical CBCL Total Problems score as criterion. A sample of 700 suffices to estimate this AUC with a 95% confidence interval of +/- 0.02.
Of the total sample of 815 eligible children, 687 participated and 674 provided complete data on both questionnaires (84.3% and 82.7% of the original sample, respectively). Both groups were representative of the total sample regarding age and gender, but non-response was higher for children of immigrant/minority origin (27.4% vs. 12.2%). Analyses were restricted to children with complete data for both questionnaires to make interpretation easier.
Data collection
The data were collected according to a standardised procedure during routine well-child examinations, from September 2004 to July 2005. The study was performed in compliance with the Helsinki Declaration [17]. The design of the study was approved by the local TNO Medical-Ethical Committee and includes verbal informed consent by parents.
The PSC [10] and the CBCL [18, 19] were mailed to children, along with the standard invitation for the preventive health assessment. Before attending the assessments, parents completed the questionnaires, placed them in sealed envelopes and gave them to the CHPs, who in turn passed them on to the researchers without opening them (in contrast with routine use, where the CHP would partially base the interview on the information from the PSC). The CHP interviewed each child and its parents regarding mental health and background, and examined each child. After each assessment, the CHP answered the following question: 'Does the child have a psychosocial problem, at this moment?' (yes, no) and scored its severity (mild, moderate or severe) and the type of problems identified using a pre-coded list. Children who only had risk indicators for the development of psychosocial problems, such as having parents with psychiatric problems or other family problems, had to be coded as having no problems.
The PSC was translated following the procedure proposed by Guillemin et al. [20]. Firstly, the original US English version of the questionnaire was translated into Dutch by three certified translators working independently of each other. Secondly, three further certified translators each translated one Dutch translation back into US English. The resulting US English versions were compared to the originals and all discrepancies were discussed by three researchers (SAR, MRC and AGCV) who spoke both Dutch and English. Discrepancies were also discussed with the developers of the PSC, Dr J.M. Murphy and Dr M.S. Jellinek, especially where items raised questions as to their intended meaning. The PSC consists of 35 items that are rated as never, sometimes or often present (0, 1 and 2, respectively). Item scores are summed; we dichotomised at 0–27 vs. 28–70, following the US cut-off [10].
The CBCL was used to assess parents' reports of the behavioural and emotional problems of their children over the preceding six months. Its (good) reliability and validity has been established [18, 19]. We used only the 120 problem items from the CBCL and computed scores for two broad-band groups of syndromes designated as Internalising and Externalising, and a Total Problems score. Children were also allocated to a normal range or a clinical range, using the 90th percentile of the Dutch normative sample as the cut-off [19].
Analysis
In the analysis we assessed the psychometric properties of the PSC and its added value in identifying psychosocial problems. Regarding psychometric properties, we first computed its internal consistency and examined the fit between the scale structure and the observed data using confirmatory factor analysis (CFA) with structural equation modelling. Next, we assessed the validity of the PSC using dichotomised CBCL scores (Total Problems score and Internalising/Externalising scales) and referral by the CHP due to psychosocial problems as criteria. Finally, we assessed whether mean PSC scores differed with the children's background.
Regarding the added value of the PSC in identifying psychosocial problems, we assessed the odds of identification of mental health problems (i.e. a clinical CBCL Total Problems score) using an elevated score on the PSC. This was repeated with adjustment for social and demographic risk indicators known to the CHP that might have helped in the identification of psychosocial problems [3, 4]. Regarding social and demographic risk indicators, we retained children with missing data in the logistic regression models by creating separate dummies for the missing category of each variable.
All analyses were done with SPSS 12.0 for Windows [21], except the CFA, which was done with Amos 5 [22]. All analyses were repeated for boys and girls separately. Results for these subgroups are provided only if they differed in a statistically significant way (p < 0.05).