Skip to main content

Evaluation of an early detection tool for social-emotional and behavioral problems in toddlers: The Brief Infant Toddler Social and Emotional Assessment - A cluster randomized trial

Abstract

Background

The prevalence of social-emotional and behavioral problems is estimated to be 8 to 9% among preschool children. Effective early detection tools are needed to promote the provision of adequate care at an early stage. The Brief Infant-Toddler Social and Emotional Assessment (BITSEA) was developed for this purpose. This study evaluates the effectiveness of the BITSEA to enhance social-emotional and behavioral health of preschool children.

Methods and Design

A cluster randomized controlled trial is set up in youth health care centers in the larger Rotterdam area in the Netherlands, to evaluate the BITSEA. The 31 youth health care centers are randomly allocated to either the control group or the intervention group. The intervention group uses the scores on the BITSEA and cut-off points to evaluate a child's social-emotional and behavioral health and to decide whether or not the child should be referred. The control group provides care as usual, which involves administering a questionnaire that structures the conversation between child health professionals and parents. At a one year follow-up measurement the social-emotional and behavioral health of all children included in the study population will be evaluated.

Discussion

It is hypothesized that better results will be found, in terms of social-emotional and behavioral health in the intervention group, compared to the control group, due to more adequate early detection, referral and more appropriate and timely care.

Trial registration

Current Controlled Trials NTR2035

Peer Review reports

Background

Psychosocial problems, such as social-emotional and behavioral problems, are prevalent among preschool children; in approximately 8-9 percent of preschool children, child health professionals identify psychosocial problems, such as anxious or depressed feelings and aggressive or disobedient behavior [1, 2]. Psychosocial problems in this age group can interfere with everyday functioning [3–5]. Research has demonstrated that problems at preschool age are associated with depressive symptoms, oppositional defiant or conduct disorder, poor peer relationships and social skills, parent- and teacher reported problems with externalizing and internalizing behavior, poor academic performance and psychiatric problems later in life [6–9]. A retrospective study [10] demonstrated that adolescents with psychosocial problems already had neurocognitive, temperament and behavioral problems at age two or three years old.

It has been recommended that psychosocial problems can be detected at a very young age and followed by appropriate management [11–13]. Research has shown that detection and treatment of psychosocial problems at a young age significantly reduces problems and increases competencies [14, 15]. Preventive youth health care, as part of community care or paediatrics, offers an opportunity for the early detection of psychosocial problems among preschool children. Child health professionals, such as physicians and nurses who provide preventive care, may apply early detection of psychosocial problems and if necessary adequate referral or short counselling [16]. In the Netherlands, participation of parents with their child in the preventive youth health care is free of charge and on voluntary basis; almost 95% of the parents of preschool children make use of the youth health care service [17].

Despite the potential impact of psychosocial problems at preschool age and the presence of easily accessible youth health care, only a relatively small number of children with psychosocial problems receive appropriate care [2, 5]. One study showed that only 29% of the children with severe problems, based on a Child Behavior Checklist (CBCL) total problem score in the clinical range, were identified by child health professionals [2]. And, in another study, only 13% of the children who scored in the clinical range of the CBCL total problem score were referred to mental health services [5].

In current preventive youth health care in the Netherlands, child health professionals apply a structured questionnaire about psychosocial problems that parents complete before coming to the youth health care center, which helps to structure the conversation between child health professionals and parents; there are no validated norm tables or cut-off scores that indicate when the questionnaire signals a problem [18].

As an alternative to this procedure, it has been recommend to evaluate the use of the Brief Infant-Toddler Social and Emotional Assessment (BITSEA) [19] for 1-3 year olds, to detect children at-risk for psychosocial problems and to act upon detection in a coherent, effective way [20].

Objectives

The objective of this study is to evaluate the effectiveness of the BITSEA as an early detection tool for preventive youth health care on children's psychosocial health at one year follow-up, compared to 'care as usual'. Additionally, the feasibility will be evaluated. In this paper we describe the design of this study.

Methods and Design

Study design

The design of the study is a cluster randomized controlled trial in which parents of children aged about 24 months are invited to participate. Information on the study is provided to the parents and the parents are asked to provide informed consent. The parents/children are invited by preventive youth health care staff for a regular health check. We identified 31 distinct youth health care centers that were numbered to the purpose of randomization. We use a block randomization so that within each of the 4 organisations, youth health care centers were randomly allocated to either the control group or the intervention group, using random numbers. The child health professionals in the intervention group use the scores on the BITSEA and the cut-off points to assess whether children are at risk for psychosocial problems. The child health professionals in the control group offer usual care by children using a questionnaire for structuring the conversation with the parents. The effect of the intervention will be evaluated after one year of follow-up by comparing CBCL1,5-5 [21] scores between the children in the intervention group and children in the control group, taking into account the baseline measurement on the CBCL1,5-5. The course of the study is presented in Figure 1. The Medical Ethics Committee of the Erasmus Medical Centre Rotterdam approved the study protocol (reference number MEC02009-092).

Figure 1
figure 1

Flow chart of the participants and allocation through the trial.

Study procedure and participants

Youth health care centers and randomisation procedure

Four youth health care organizations that consist of a total of 31 distinct youth health care centers that cover both urban and rural areas in the larger Rotterdam area, are participating in this study. Fifteen youth health care centers were randomly allocated to the intervention group; 16 youth health care centers were randomly allocated to the care-as-usual group, applying a block randomisation procedure as described above. Prior to the start of the study, the researchers arranged meetings to explain the study procedure and to instruct the child health professionals in the intervention group about the use and scoring of the BITSEA, with support of a specialized psychiatrist.

Children and their parents

Parents of 3,000 children are invited to participate in the study. The study population consists of parents or caregivers of toddlers aged 24 months old at baseline, and 36 months old at one year follow-up. Parents of children in the age range of this study have a high attendance (95%) at the regular health checks at youth health care centers [17]. Children who receive treatment of a mental health professional at baseline will be excluded from the study.

Intervention condition

BITSEA

The 42-item BITSEA is an early detection tool for emotional or behavioral problems and delays in social-emotional competence, including autism spectrum disorders, in toddlers. The BITSEA was developed and applied in the USA, and since then also studied in Turkey and Finland [19, 22, 23]. It is appropriate for use among children of 12-36 months old and consists of 42 items with 3 response options ('not true/rarely', somewhat true/sometimes', 'very true/often'). That are part of one of 2 multi-item scales, a Problem scale (31 items) or Competence scale (11 items). Per scale the items are summed up into a scale score. In addition to the 42 items, the BITSEA has 2 single-item questions with regard to parents' concerns. Internal consistency of the Problem scale has been reported to be 0.79 and of the Competence scale 0.65 [19]. Ten to 45 day test-retest reliability (ICC) of the Problem scale has been reported to be 0.87 and of the Competence scale 0.85 [19]. Relative to typical parent/teacher agreement [24], the parent/child-care provider correlation was relatively high for the Competence scale (ICC = 0.59) and 0.28 for the Problem scale [19]. The BITSEA was translated into Dutch according to international guidelines [25]. Psychometric properties of the BITSEA for the Dutch population will be assessed in the present study.

At the intervention centers child health professionals use the BITSEA [26] as an early detection tool during the regular health check. The child health professionals are trained to score the answers given by parents on the BITSEA and use the cut-off points provided in the literature [19] in their assessment whether children are at risk for or currently experiencing psychosocial problems. Additional information given by parents about timing, duration, intensity of problematic emotions and behavior is also considered in the assessment of the risk for problems. If, for instance, the problematic emotions/behaviors are mild or are considered to be temporarily, e.g. after a major life event, the child health professional may offer advice about how to cope with the circumstances and may choose to ask the parent back in a few weeks for a follow-up.

The child health professional may choose to refer a child and his/her parents to specialized care when the child is at risk for or currently experiencing problems based on the BITSEA scores and cut-off points, when the problematic behaviors/emotions are severe and not considered to be temporarily. A referral to specialized care is always made after consultation with the physician at the youth health care center.

Control condition

In the control condition youth health care centers at child's age 2 years provide care as usual; i.e. the child health professionals use a short questionnaire [18] that serves as a guide through the conversation between child health professionals and parents. Based on this information the child health professionals may choose to invite parents back for a follow-up visit or to refer to specialized care after consultation with the physician at the youth health care center.

Measurements

Primary outcome measures

The primary outcome of the study is the children's psychosocial health, measured with the Child Behavioral Checklist (CBCL1,5-5) [21]. Child health professionals are blind to this measurement. The 99-item CBCL1,5-5 is designed for children 18 months through 5 years and has two domains (Internalizing and Externalizing Problems and a Total Problem score). Answers are given on a 3-point scale with the following response options: 'not true', 'somewhat or sometimes true' and 'very true or often true'. We apply subclinical and clinical cutpoints for the Dutch population [27]. The primary outcome will be measured at baseline at child's age 24 months and one year after the intervention, at the child's age of 36 months. We hypothesize that children in the intervention group will have a lower Total Problem score on the CBCL1,5-5 at follow-up compared to children in the control group, due to more adequate screening, referral and more appropriate and timely care. For an overview, see table 1.

Table 1 Primary and secondary outcome measures and co-variates in the study

Secondary outcome measures

A secondary outcome is health related quality of life, i.e. General Health Perceptions subscale and the Growth and Development subscale of the Infant and Toddler Quality of Life Questionnaire (ITQOL) [28, 29], measured at follow-up at child's age 36 months. For an overview, see table 1.

Co-variates

Information on parental characteristics (date of birth, ethnicity, immigration characteristics, cultural identity, socio-economic status), children's characteristics (date of birth, sex, ethnicity, day-care attendance, presence of (mental) health problems and treatment for those problems), and household composition, major life events and the perceived health of the child rated by parents are obtained from the questionnaires at baseline and at follow-up. For an overview, see table 1.

Statistical analyses

Given the cluster design of the study, multilevel analyses will be applied [30, 31]. Linear multilevel analysis will be applied for continuous outcome variables and logistic multilevel analysis for dichotomous outcome variables. Interaction effects of gender and ethnic background with the outcomes will be explored.

Power of the study

Power calculations indicated that a total of 3,000 children (and their parents) are needed to detect a difference of 8 points on the CBCL1,5-5 between the control and experimental group, assuming a standard deviation of the CBCL1,5-5 of 26.5 points [32] and an intra-cluster coefficient of 0.1, with a power of 0.80 and alpha 0.05. Assuming a participation of 50% and a lost to follow-up of 30%, we will have complete data at follow-up of 2,100 children (1,050 in both the intervention and the control group).

Process evaluation

In addition to the effect evaluation, a process evaluation will be carried out, in which both the perspectives of parents and professionals will be taken into account. All parents that are included in this study, are asked to evaluate the use of the early detection tool (i.e. level of difficulty, level of understanding, consumed time and satisfaction with the early detection tool as a preparation for the regular health check at the youth health care center). All child health professionals are invited to complete a computer-based process evaluation questionnaire at 6 months after the start of the study. The process evaluation questionnaire consists of items about consumed time, adherence to work instructions, satisfaction with the early detection tool, general perception of the use of the early detection tools in the youth health care and perceived contribution of the early detection tool (a) to the quality of the conversation with parents, (b) to the assessment of the development of the child, and (c) to deciding whether or not to refer.

Furthermore, referrals and consumed care in the year after baseline measurement are assessed at 1-year follow-up, at child's age 36 months; i.e. if a referral to specialized care is made and to which professional; the extent to which parents pursue received referrals, and the diagnosis if one is made are measured at the 1 year follow-up, when children are age 36 months.

Discussion

This paper describes the design of a cluster randomized controlled trial. The trial evaluates the effectiveness of the BITSEA as an early detection tool when used by preventive child health professionals on children's psychosocial health at one year follow-up, compared to 'care as usual'. We hypothesize to find better results, in terms of psychosocial health in the intervention group at one year follow-up, compared to the control group, due to more adequate early detection, referral and more appropriate and timely care.

Strengths of the study are the cluster randomized controlled design, the power of the study, and the setting of the study, which is the daily practice of regular health checks at the youth health care centers that are highly attended by parents. The one year follow-up measurement allows evaluation of the medium term effect of the BITSEA. The study sample will include families with a non-Dutch background, which we expect will add to the generalizability of the results.

Because the study relies primarily on self-report by parents, misclassification might occur. Parents might provide socially desirable answers, e.g. by understating problems or overstating competencies.

A limitation of the study is that the questionnaires are only available in Dutch. For this reason it might be possible that parents with a relatively low level of knowledge of the Dutch language will have some difficulty with the completion of the early detection tool. However, parents have the opportunity to ask for help regarding this issue at the youth health care centers. Furthermore we assess the extent in which parents have understood the questions in the early detection tool as a process measure.

In conclusion, the study evaluates the effectiveness of the BITSEA as an early detection tool to be applied by child health professionals, with the purpose of promoting children's psychosocial health at one year follow-up, compared to 'care as usual'.

References

  1. Velderman M, Crone M, Wiefferink C, Reijneveld S: Identification and management of psychosocial problems among toddlers by preventive child health care professionals. European Journal of Public Health. 2010, 20 (3): 332-338. 10.1093/eurpub/ckp169.

    Article  Google Scholar 

  2. Reijneveld SA, Brugman E, Verhulst FC, Verloove-Vanhorick SP: Identification and management of psychosocial problems among toddlers in Dutch preventive child health care. Archives of Pediatrics & Adolescent Medicine. 2004, 158 (8): 811-817. 10.1001/archpedi.158.8.811.

    Article  Google Scholar 

  3. Brugman E, Reijneveld SA, Verhulst FC, Verloove-Vanhorick SP: Identification and management of psychosocial problems by preventive child health care. Archives of Pediatrics & Adolescent Medicine. 2001, 155 (4): 462-469.

    Article  CAS  Google Scholar 

  4. Briggs-Gowan MJ, Carter AS, Skuban EM, Horwitz SM: Prevalence of social-emotional and behavioral problems in a community sample of 1-and 2-year-old children. Journal of the American Academy of Child and Adolescent Psychiatry. 2001, 40 (7): 811-819. 10.1097/00004583-200107000-00016.

    Article  CAS  PubMed  Google Scholar 

  5. Verhulst FC, van der Ende J: Factors associated with child mental health service use in the community. Journal of the American Academy of Child and Adolescent Psychiatry. 1997, 36 (7): 901-909. 10.1097/00004583-199707000-00011.

    Article  CAS  PubMed  Google Scholar 

  6. Briggs-Gowan MJ, Carter AS: Social-emotional screening status in early childhood predicts elementary school outcomes. Pediatrics. 2008, 121 (5): 957-962. 10.1542/peds.2007-1948.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Campbell SB, Spieker S, Burchinal M, Poe MD: Trajectories of aggression from toddlerhood to age 9 predict academic and social functioning through age 12. Journal of Child Psychology and Psychiatry. 2006, 47 (8): 791-800. 10.1111/j.1469-7610.2006.01636.x.

    Article  PubMed  Google Scholar 

  8. Hakman M, Sullivan M: The Effect of Task and Maternal Verbosity on Compliance in Toddlers. Infant and Child Development. 2009, 18 (2): 195-205. 10.1002/icd.599.

    Article  Google Scholar 

  9. Mesman J, Koot HM: Early predictors of preadolescent internalizing and externalizing DSM-IV diagnoses. J Am Acad Child Adolesc Psychiatry. 2001, 40: 1029-1036. 10.1097/00004583-200109000-00011.

    Article  CAS  PubMed  Google Scholar 

  10. Moffitt TE, Caspi A: Childhood predictors differentiate life-course persistent and adolescence-limited antisocial pathways among males and females. Development and Psychopathology. 2001, 13 (2): 355-375. 10.1017/S0954579401002097.

    Article  CAS  PubMed  Google Scholar 

  11. American Academy of Pediatrics. Committee on children with disabilities: development surveillance and screening of infants and young children. Pediatrics. 2001, 108: 192-196.

  12. Bricker D, Schoen Davis M, Squires J: Mental health screening in young children. Infants and Young Children. 2004, 17: 129-144. 10.1097/00001163-200404000-00005.

    Article  Google Scholar 

  13. Horen, zien, niet zwijgen [See all, hear all, say something]. Inspectie voor de Gezondheidszorg, Inspectie van het Onderwijs, Inspectie Openbare Orde en Veiligheid, Inspectie Jeugdhulpverlening, Jeugdbescherming. 2003

  14. Durlak JA, Wells AM: Evaluation of indicated preventive intervention (secondary prevention) mental health programs for children and adolescents. American Journal of Community Psychology. 1998, 26 (5): 775-802. 10.1023/A:1022162015815.

    Article  CAS  PubMed  Google Scholar 

  15. Elliot J, Prior M, Merrigan C, Ballinger K: Evaluation of a community intervention programme for preschool behaviour problems. Journal of Paediatrics and Child Health. 2002, 38 (1): 41-50. 10.1046/j.1440-1754.2002.00713.x.

    Article  CAS  PubMed  Google Scholar 

  16. Basistakenpakket Jeugdgezondheidszorg 0-19 jaar [Youth Health Care Tasks 0-19 years]. 2002, Ministerie van Volksgezondheid, Welzijn en Sport

  17. Centraal Bureau voor de Statistiek [Statistics Netherlands]. [http://statline.cbs.nl/StatWeb/publication/?DM=SLNL&PA=7042MC&D1=397-401&D2=0-1,5,10,15,20,25-29&VW=T]

  18. Kousemaker NPJ: Zoeken, vinden, zorgen delen: de ontwikkeling van een praktijkparadigma voor onderkenning en pedagogische preventie van psychosociale problematiek in de Jeugdgezondheidszorg [Searching, finding, sharing care; the development of a practise paradigm for the descernment and pedagogical prevention of psychosocial problems in Youth Healthcare]. 1996, Dissertatie - Universiteit van Amsterdam Leiden: Rijksuniversiteit Leiden - Drukkerij Faculteit Sociale Wetenschappen

    Google Scholar 

  19. Briggs-Gowan MJ, Carter AS, Irwin JR, Wachtel K, Cicchetti DV: The brief infanttoddler social and emotional assessment: Screening for social-emotional problems and delays in competence. Journal of Pediatric Psychology. 2004, 29 (2): 143-155. 10.1093/jpepsy/jsh017.

    Article  PubMed  Google Scholar 

  20. Hermans J, Ory F, Schrijvers G: Supporting development and parenting: sooner, faster and better: an advice for early detection and interventions in regard to developmental and parenting problems. 2005, Julius Centrum: Utrecht

    Google Scholar 

  21. Achenbach TM, Edelbrock C, Howell CT: Empirically based assessment of the behavioral/emotional problems of 2- and 3-year-old children. J Abnorm Child Psychol. 1987, 15 (4): 629-650. 10.1007/BF00917246.

    Article  CAS  PubMed  Google Scholar 

  22. Karabekiroglu K, Rodopman-Arman A, Ay P, Ozkesen M, Akbas S, Tasdemir GN, Boke O, Peksen Y: The reliability and validity of the Turkish version of the brief infant-toddler social emotional assessment (BITSEA). Infant Behavior & Development. 2009, 32 (3): 291-297. 10.1016/j.infbeh.2009.03.003.

    Article  Google Scholar 

  23. Haapsamo H, Ebeling H, Soini H, Joskitt L, Larinen K, Penninkilampi-Kerola V, Carter A, Moilanen I: Screening infants with social and emotional problems: A pilot study of the Brief Infant Toddler Social and Emotional Assessment (BITSEA) in Northern Finland. International Journal of Circumpolar Health. 2009, 68 (4): 386-393.

    Article  PubMed  Google Scholar 

  24. Achenbach TM, McConaughy SH, Howell CT: Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin. 1987, 101: 213-232.

    Article  CAS  PubMed  Google Scholar 

  25. Wild D, Grove A, Martin M, et al: Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) Measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005, 8: 94-104. 10.1111/j.1524-4733.2005.04054.x.

    Article  PubMed  Google Scholar 

  26. Briggs-Gowan MJ, Carter AS: Brief Infant-Toddler Social and Emotional Assessment (BITSEA) mannual, version 2.0. 2002, New Haven: CT: Yale University

    Google Scholar 

  27. Verhulst FC, Van der Ende J, Koot HM: Handleiding voor de Teacher's Report Form (TRF). [Manual for the Theacher's Report Form (TRF)]. Rotterdam: Afdeling Kinder- en Jeugdpsychiatrie, Sophia Kinderziekenhuis/Academisch Ziekenhuis Rotterdam/Erasmus Universiteit Rotterdam. 1997

    Google Scholar 

  28. Raat H, Landgraf JM, Oostenbrink R, Moll HA, Essink-Bot ML: Reliability and validity of the Infant and Toddler Quality of Life Questionnaire (ITQOL) in a general population and respiratory disease sample. Quality of Life Research. 2007, 16 (3): 445-460. 10.1007/s11136-006-9134-8.

    Article  PubMed  Google Scholar 

  29. Raat H, Monhangoo AD, Grootenhuis MA: Pediatric health-related quality of life questionnairese in clinical trials. Current Opinion in Allergy and Clinical Immunology. 2006, 6: 180-185. 10.1097/01.all.0000225157.67897.c2.

    Article  PubMed  Google Scholar 

  30. Campbell MK, Elbourne DR, Altman DG, Grp C: CONSORT statement: extension to cluster randomised trials. British Medical Journal. 2004, 328 (7441): 702-708. 10.1136/bmj.328.7441.702.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Twisk JWR: Applied Multilevel Analysis: A practical guide. 2006, Cambridge: Cambridge University Press

    Book  Google Scholar 

  32. Achenbach TM, Rescorla LA: Manual for the ASEBA Preschool Forms & Profiles. 2000, Burlington: VT: University of Vermont, Research Center for Children, Youth, and Families

    Google Scholar 

Pre-publication history

Download references

Acknowledgements

This study is funded by a grant from the funding body ZonMw, the Netherlands Organization for Health Research and Development (project number: 80-82435-98-8058).

We express our gratitude to J.C. Visser, S. Smeekens and J.M.A. Riksen-Walraven for the forward translation and I. Flink and Z. Op de Macks for the back translation of the BITSEA.

Study website

http://www.vestonderzoek.nl

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hein Raat.

Additional information

Competing interests

AC gets royalties on the sale of the BITSEA from Pearson Assessment (not in the context of this study). The other authors declare that they have no competing interests.

Authors' contributions

HR and WJ had the original idea for the study and its design and were responsible for acquiring the study grant. IK is responsible for the data collection, data analysis en reporting of the study results. AC provides expert input during the study. HR and WJ supervise the study. All authors regularly participated in discussing the design and protocols used in the study. All authors read and approved the final manuscript.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Rights and permissions

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Kruizinga, I., Jansen, W., Carter, A.S. et al. Evaluation of an early detection tool for social-emotional and behavioral problems in toddlers: The Brief Infant Toddler Social and Emotional Assessment - A cluster randomized trial. BMC Public Health 11, 494 (2011). https://doi.org/10.1186/1471-2458-11-494

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1471-2458-11-494

Keywords