A cross-sectional pilot study of the Scottish early development instrument: a tool for addressing inequality

Abstract

Background

Early childhood is recognised as a key developmental phase with implications for social, academic, health and wellbeing outcomes in later childhood and indeed throughout the adult lifespan. Community level data on inequalities in early child development are therefore required to establish the impact of government early years’ policies and programmes on children’s strengths and vulnerabilities at local and national level. This would allow local leaders to target tailored interventions according to community needs to improve children’s readiness for the transition to school. The challenge is collecting valid data on sufficient samples of children entering school to derive robust inferences about each local birth cohort’s developmental status. This information needs to be presented in a way that allows community stakeholders to understand the results, expediting the improvement of preschool programming to improve future cohorts’ development in the early years. The aim of the study was to carry out a pilot to test the feasibility and ease of use in Scotland of the 104-item teacher-administered Early Development Instrument, an internationally validated measure of children’s global development at school entry developed in Canada.

Methods

Phase 1 was piloted in an education district with 14 Primary 1 teachers assessing a cohort of 154 children, following which the instrument was adapted for the Scottish context (Scottish Early Development Instrument: SEDI). Phase 2 was then carried out using the SEDI. Data were analysed from a larger sample of 1090 participants, comprising all Primary 1 children within this school district, evaluated by 68 teachers.

Results

The SEDI displayed adequate psychometric and discriminatory properties and is appropriate for use across Scotland without any further modifications. Children in the lowest socioeconomic status quintiles were 2–3 times more likely than children in the most affluent quintile to score low in at least one developmental domain. Even in the most affluent quintile though, 17% of children were ‘developmentally vulnerable’, suggesting that those in need cannot be identified by socioeconomic status alone.

Conclusions

The SEDI offers a feasible means of providing communities with a holistic overview of school readiness for targeting early years’ interventions.

Background

Evidence on the effectiveness of early childhood interventions is accumulating [1]. Early development has been shown to influence proximal factors such as behaviour, academic outcomes, and school drop-out rates [2], and later life outcomes such as social participation, criminality, obesity, and mental health [3, 4]. Furthermore, neurodevelopmental research confirms that the brain is most malleable in the early years [5, 6], hence both public health professionals and economists argue that investing in this period is necessary for tackling inequality [7, 8]. Indeed monetary returns to society over an individual’s life span are expected to more than repay the initial investment [9]. Social gradients in health outcomes have been reported for infant and child mortality, low birth weight, injuries, malnutrition, infectious disease and usage of healthcare facilities [10]; and in cognitive outcomes for school admission, mathematical and language attainment, and literacy [11]. Public health professionals have consequently called for studies on interventions aimed at reducing socioeconomic inequalities in healthy child development [12].

Dubbed the ‘sick man of Europe’ [13], Scotland is a fitting location for addressing inequalities observed across a range of health outcomes [14–16]. Unfortunately should appropriate interventions be implemented, it would be difficult to determine their success on developmental and health outcomes as, with a few exceptions such as child dental health and weight at school entry, routine population-level measures currently collected in Scotland are mostly concerned with either birth, or later life end-stage events [17]. There is a strong case for more ‘upstream’ population outcome measures, taken earlier in the life-course, that predict later life chances and wellbeing outcomes and which may be changed within the usual policy-maker time horizon, say five years, by intervention [17, 18]. Commonly used individual-level child development measures are typically designed to be administered by particular groups of education and health professionals such as psychologists, or physicians, thus making population-level data collection unfeasible or at least very costly. Moreover they focus on specific developmental domains such as language, cognitive or physical development, rather than providing a holistic measure. These measures, though useful for the purposes for which they were designed, are less useful as ‘upstream’, routinely-collected outcome measures suitable for community use.

The Early Development Instrument

The Early Development Instrument (EDI) [19, 20] was devised in Canada in response to this need for a universal, holistic, sensitive population measure of early child development outcomes that measures how well communities prepare their children for school [21]. It has been extensively validated [22, 23], and is now used in Canada and Australia, and parts of the USA, Central and South America, Asia, the Middle East, Africa and Europe. It provides a holistic, community-level overview of child development with findings readily accessible and interpretable by health, educational and social work agencies for planning and evaluation purposes. Class teachers complete the EDI for all school entrants, typically children aged 4–6 years, based on their knowledge of the children’s skills, competences and behaviour after four months of classroom contact. The instrument evaluates five key developmental domains: Physical health and wellbeing, Social competence, Emotional maturity, Language and cognitive development, and Communication and general knowledge. The EDI is never interpreted or reported at the individual child level or used for clinical diagnostic purposes. Instead, it is intended for community-level monitoring and as a catalyst in planning and action to improve aspects of the local environment that influence early child development [19]. Results are fed back to communities, allowing local stakeholders, planners and policy makers to judge developmental progression in order to reflect on the effectiveness of support services, and to locally plan early childhood interventions for infants, children, parents, families, and communities that address identified collective developmental vulnerabilities and so better prepare children for school.

EDI items are scored from 0–10 for each developmental domain. A mean domain score is then calculated. Children whose scores fall in the bottom 10% are classed as vulnerable in that domain. If scores are beneath the 10th percentile on one or more domains, then children are classed as ‘overall developmentally vulnerable’ (Note that in EDI terminology, ‘vulnerable’ does not have connotations with child protection or social services but merely refers to children scoring within the lowest 10%). EDI results can then be reported in three ways. The first is mean scores (out of ten) for each domain. Secondly, the percentage of children who are ‘overall developmentally vulnerable’ can be reported. Finally, for each of the five domains of development, the percentage of children in an area scoring in the bottom 10% (‘vulnerable’), between 10 and 25% (‘at risk’), and above 25% regarded as ‘on track’ for development, are reported. This uses 10th and 25th percentile ‘cut-offs’ scored by the whole cohort. Each can be examined at different geographical levels: region, neighbourhood or school level. Indeed, it is common practice in both British Columbia (Canada) and Australia to depict local communities’ EDI results on coloured maps, to facilitate the use of these data by the general public [18, 21].

In Canada, data linkage of routinely-collected population level measures of numeracy, reading comprehension and writing skills with the EDI dataset was conducted [24]. This allowed them to develop the Community Index of Child Development, a means of summarising development in individuals over time, rather than relying on cross-sectional data of different sets of children. Data linkage experts in Manitoba have also linked databases to explore the role of the socioeconomic gradient and infant health in child development, the comparative influence of family and neighbourhood in later wellbeing, and the long-term effects of poverty reduction [25, 26].

The Scottish context

In Scotland more than 95% of 4-year olds attend free pre-school education funded by local authorities [27]. Children start attending formal school at age 4.5 to 5.5 years, depending on birth dates, in August each year and only 1% of children attend an independent (non-government funded) school [27]. This first year of formal school is called Primary 1 (P1) and is broadly equivalent to the reception year in England and Wales, and the kindergarten year in Canada and Australia.

Within Scotland, however, there have been few means by which geographical or socioeconomic inequalities in child development have been widely and consistently measured to identify inequalities in early childhood. The aim of this study was to pilot a Scottish EDI within one school district in central Scotland. There were two phases of data collection with the following objectives:

Phase 1:

• to test the feasibility of implementing the Canadian EDI within the Scottish school context with a small sample

• to adapt the Canadian EDI to the Scottish context

Phase 2:

• to incorporate Phase 1 adaptations into a Scottish version (Scottish Early Development Instrument: SEDI) implemented across the whole school district

• to provide the community with information about children’s strengths and vulnerabilities

• to assess the psychometric properties of the SEDI (e.g. internal reliability of the questionnaire items) and its discriminatory ability, that is the tool’s ability to identify communities where children are well prepared for school compared to communities where children and their teachers are managing complex challenges

• to establish the tool’s feasibility for use with a wider Scottish population.

Methods

Measure

The EDI questionnaire evaluates five domains of child development: Physical health and wellbeing (items A2-A13 and C58/17–28 and 126); Social competence (items C1-C25 and C27, 69–93 and 95); Emotional maturity (items C28-C57/96–125); Language and cognitive development (items B8-B33/36–61); and Communication and general knowledge (items B1-B7 and C26/29-35 and 94). In addition, all domains, with the exception of Communication and general knowledge, comprise a number of sub-domains. For example, the Physical health and wellbeing domain comprises the sub-domains, Gross and fine motor skills, Physical readiness for the school day and Physical independence[20].

Procedure

Both phases of this EDI pilot study were conducted in a Scottish local authority district where new initiatives to improve early years’ outcomes had recently been implemented, and thus much local support had been raised for early years’ interventions – including this pilot study. Paper-based questionnaires were used in both phases. Teachers completed the assessments in their normal working hours while substitute teachers, funded by the study, were provided to teach their classes. One hundred and fifty-four P1 pupils (82 female, 72 male) from the school district participated in Phase 1. Questionnaires were completed by 14 teachers. Mean pupil age was 5 years 7 months (SD = 4 months, range 4 years 3 months - 6 years 9 months). Qualitative methods, including focus groups and semi-structured questionnaires, were used to gather feedback from teacher participants on the tool and the process. Cronbach’s α was used to determine internal consistency. Phase 1 results suggested that the EDI could be feasible to administer and implement in Scotland and that only minor changes were required to reflect the Scottish context, e.g., changing ‘repeating grade’ to ‘repeating Primary 1’, changing teaching qualifications to denote recognised Scottish qualifications, and to reduce administration errors, e.g., pre-entering part of year details for child date of birth and date of completion.

The larger Phase 2 sample was recruited from all 1180 P1 pupils in the school district from its 35 schools, including independent schools. Sixty-eight P1 teachers completed the EDI questionnaires (100% teacher response rate). Eighteen children were opted-out^a of participation by their parents. An additional 72 children were excluded because they had been in school for less than one month, or had missing data, or had special needs^b. Pupil postcodes were grouped into five quintile categories using the Scottish Index of Multiple Deprivation (SIMD)^c, a postcode-based index of socioeconomic status assigned to each of 6,505 data zones across Scotland’s population of about 5 million, on the basis of census, unemployment and other data. A score of 1 was given to the most deprived area and 5 the most affluent. For seven children there was no quintile information available hence they were excluded from results requiring socioeconomic status quintile.

Sample characteristics

The final sample then consisted of 1090 sets of pupil data (524 females, 563 males, 3 where gender was not recorded). Mean age was 5.51 years, SD = 0.32, range = 4.49 – 6.94. Most had English as their first language with 26 pupils identified as speaking English as well as another language. Seven pupils spoke only Polish or Urdu.

Ethics

The research was approved by: the School of Psychological Sciences and Health Ethics Committee of the University of Strathclyde, Glasgow, UK; the Education Authority of the relevant school district; and the Chief Scientist Office of the Scottish Government. In line with EDI data collection in other countries, opt-out consent was utilized for parents of pupils. Parents received detailed information sheets with clear instructions on how to refuse participation of their children. All teachers provided written, informed consent.

Results

Reliability

Cronbach’s α for the SEDI overall was .97, indicating high levels of internal reliability. Cronbach’s α for four of the five individual domains was greater than .9, and the fifth was .78, showing good reliability. Sub-domains also indicated good levels of reliability except for Physical independence and Physical readiness for school day with Cronbach’s α of .22 and .51 respectively. Item deletion did not significantly improve reliabilities. Indeed, a reliability of .26 reported for a Canadian sample was similarly poor for the Physical independence sub-domain [20]. Five items were noted as having low item-total correlations (0.2). Independent in toilet habits most of time (Physical independence sub-domain of Physical wellbeing scale); sucks thumb/finger, upset when left by parent/guardian and is shy (Emotional maturity); and knows how to handle a book (Language and cognitive).

Mean scores by domain

All yes/no scores and three-point Likert scores on the SEDI were converted into scores on ten-point scales to allow comparison across domains. Tables 1 and 2 summarize mean scores, score ranges and percentile cut-offs of score distributions, by EDI domain and gender.

Table 1 School enterers mean score and 10/25/50% cut-off scores in overall sample by EDI domain

Table 2 EDI developmental vulnerability of school enterers by gender and domain

SEDI vulnerability by domain and by SES quintile

Scores were then further categorised according to SES quintile groups (Figure 1). For each SEDI domain, quintile scores appeared to follow a SES gradient. One-way analysis of variance indicated significant differences between quintile groups for Physical health and wellbeing (F (4, 1081) = 7.55, p = .000); Social competence (F (4, 1082) = 5.07, p = .000); Language and cognitive development (F (4, 1082) = 9.69, p = .000) and Communication & general knowledge (F (4, 1082) = 5.11, p = 0.000). No significant differences between quintiles were found in the Emotional maturity domain (F (4, 1079) = 1.78, p > .05). Allowing that there was only a small quintile 1 group (4%) in this sample, results suggested that the SEDI can discriminate school readiness skills across socioeconomic gradients.

Figure 1

EDI mean domain score of school enterers by socioeconomic status quintile. The Scottish Index of Multiple Deprivation (SIMD) ranks small postcode areas (called data zones) according to level of deprivation by combining indicators such as current income, employment, health, education, skills, housing, and crime. Data zones can then be grouped into quintiles (five groups). Quintile 5 is most affluent and quintile 1 least affluent.

Domain cut-offs: vulnerable, at-risk and on-track by quintile

Mean scores for the sample and 10%, 25% and 50% cut-off vulnerability scores for each domain were used to calculate the frequency and percentage of children categorised as developmentally vulnerable in each quintile for each SEDI domain, i.e., those with scores below the 10th percentile. In addition, children whose score fell between the 10th and the 25th percentile were classed as ‘at-risk’, those whose scores were between the 25th percentile and 50th percentile were ‘on-track 1’, and those above the 50th percentile were ‘on-track 2’. Tables 2 and 3 show vulnerability frequencies and percentages for each domain, by gender and by SES quintile group.

Table 3 EDI developmental vulnerability of school enterers by SIMD quintile and domain

Inspection of these figures demonstrates that the majority of participants were ‘on-track’, regardless of quintile group or domain. The Communication and general knowledge domain further displayed ceiling effects for all quintile groups highlighting that for this SEDI component children from all quintiles tended to score highly, although this part of Scotland does not contain the most deprived areas.

Developmental vulnerabilities in one, two or more domains by quintile

Data were then examined for developmental vulnerability in one, and two or more, domains. 297 children in the sample (27.3%) were recorded as vulnerable in one or more domains, and 168 (15.4%) in two or more domains. Developmental vulnerabilities by SIMD quintile can be seen in Figure 2. Quintile 5, the most affluent SES grouping, showed the lowest vulnerability rates (16.7%) in one or more domains, and quintile 1, the least affluent, demonstrated the highest rate of vulnerability (38.5%). Figure 2 also illustrates percentage vulnerabilities in two or more domains by quintile. A similar pattern of results was found, with quintile 5 (the most affluent areas) showing the lowest vulnerability rates (7.2%) and quintile 1 the highest vulnerability rate (25.6%).

Figure 2

Percentage children vulnerable on one or more, and two or more developmental domains by quintile. Vulnerable=Scoring in the bottom 10% in a developmental domain. The five developmental domains were: Physical health and wellbeing; Social competence; Emotional maturity; Language and cognitive development; and Communication and general knowledge. Quintile 5 is most affluent and quintile 1 least affluent.

Analysis of our results by gender showed that 193 boys (34.3% boys) and 104 girls (19.8% girls) were vulnerable in one domain. Adjusted for age, the odds ratio of a boy being vulnerable in one domain was 2.38 times that of a girl. Similarly 111 boys (19.7%) and 56 girls (10.7%) were vulnerable in two or more developmental domains. Adjusted for age, the odds of a boy being vulnerable in at least two domains were 2.51 times that of a girl. The odds of a child who did not communicate adequately in his/her first language presenting as vulnerable in two or more domains were 8.04 times that of children who were able to communicate adequately in their first language.

Analysis by SIMD quintile, adjusted for age and gender, showed that in the most deprived quintile (1) children were 2.77 times more likely than those in the most affluent quintile (5) to be vulnerable in one or more domains. Compared to children in quintile 5, children in quintile 2 were 3.04 times more likely, quintile 3 were 2.21 times more likely, and quintile 4 were 1.45 times more likely, to be vulnerable in one or more domains. Similarly for vulnerability in two or more domains, children in quintile 1 were 3.36 times more likely to be vulnerable than those in quintile 5. Children in quintile 2 were 3.68 times more likely, quintile 3 were 2.09 times more likely and quintile 4 were 1.61 times more likely, to be vulnerable in at least two domains compared to children in quintile 5.

Chi square analyses also showed each of the following indicators to be significantly associated with vulnerability on one or more domains: child struggling to do his/her school work; child needing further assessment; child on a waiting list for further assessment; and child receiving school based supports.

Cost

Finally, we costed the data collection process for the EDI in the school district, based on actual Scottish costs of teacher time buy-out, i.e. to purchase substitute teacher services, for training (one half day) and EDI completion (about one day). If we assume direct web-based data-entry by teachers, as is normal in Canada and Australia, then the total cost of data collection and entry comes to about £20 sterling per child, 95% of which goes to teacher time buy-out. This would translate, for an “average” Scottish Local Authority with a total population of 150,000 and a crude birth rate of about 1%, to £30,000 for each wave of data collection – typically every three calendar years in Canada and Australia. Thus, for the average Local Authority the annualized cost would be £10,000.

Discussion

The study’s aim was to test the usefulness of a Scottish version of the EDI as a tool to assess global development in Scottish children at school entry. Results showed that it displayed adequate psychometric and discriminatory properties. Internal reliabilities for domains were good, ranging from .78 to .96. Five items identified as having low item-total correlations (independent in toilet habits most of time, sucks thumb/finger, upset when left by parent/guardian’, is shy and ‘knows how to handle a book’) should be monitored in any further Scottish roll-out. Mean domain scores demonstrated the expected social gradient across five SES postcode-based quintiles: Quintile 5, the most affluent quintile, had the highest mean scores across all domains and quintile 1 the lowest.

As well as providing information across a region, EDI results can contribute to international comparisons. For example, mean domain scores for our Scottish school district sample were slightly higher on all five domains than those reported for Canadian children [28] and slightly lower than reported Australian scores [29]. Nonetheless our school district scores followed the same pattern as the Australian and Canadian studies, suggesting that English-speaking Western children may have similar shortfalls and strengths in their early development. Percentage vulnerability in one or more domains for our sample (27.3%) was also similar to those reported in EDI studies in Canada (27.2%) and Australia (23.5%) [28, 29]. Gender differences in the Scottish sample were also similar to those reported in other countries where boys were more likely to be developmentally vulnerable than girls [30]. However vulnerability in two or more domains was 15.4% for our Scottish district sample, which appears higher than 12.4% reported for the Canadian sample, and 11.8% for Australia [28, 29]. At this point we do not know whether this local data represents developmental vulnerabilities across the whole of Scotland. Nonetheless it exemplifies the kind of information that EDI results produce for policy makers’ consideration.

Although an SES gradient of vulnerability by quintile is evident, the study findings also highlighted that significant developmental vulnerability was found across all five quintiles and not only in the most deprived. Such detailed data provide evidence needed to implement what Michael Marmot has called “proportionate universalism,” [8] including targeting of local programmatic interventions tailored to individual developmental domains and delivered in those communities where it is most needed. This is particularly relevant in Scotland where, in 2005, the core Child Health Surveillance programme was cut back in that the number of routine health visitor contacts in the first four years of life was reduced by three contacts thereby limiting opportunities to identify children and families with problems [31, 32]. In addition, the government utilizes a decentralised model allowing local authorities and/or health boards to make their own decisions with regard to additional services. Thus different parts of the country are using different licensed parenting programmes such as Triple P (Glasgow Health Board), Incredible Years (West Lothian Council), Mellow Parenting and numerous others [1]. Some programmes, such as David Olds’ Family Nurse Partnership, have been centrally funded and managed, starting with a ‘test site’ in NHS Lothian, Edinburgh, and subsequently being rolled out to limited sites in NHS Tayside, Greater Glasgow and Clyde, Ayrshire and Arran, Fife, Lanarkshire and Highland. Wider adoption of the EDI, or indeed any single measure of early child development, is currently therefore difficult to achieve in Scotland, due to decentralization, and the attendant local heterogeneity in stakeholder practices. However, adopting the EDI may provide stakeholders with knowledge about their communities enabling them to make wise choices with regard to child and parenting programmes.

In other regions that have utilised EDI evaluation, it has acted as a catalyst for public health, social work and education to collaborate to examine available resources for families with young children and identify areas of additional need to improve young children’s opportunities for achievement [21]. Furthermore, there is suggestive evidence from early Western Australian use of the EDI, in disadvantaged communities such as Mirrabooka, that community EDI scores can be substantially shifted within a half-decade, using improved local preschool programming [33]. This contrasts with epidemiological conventional wisdom regarding the much longer lag-times required to change the usual, routinely collected health indicators (mostly based on birth, hospitalization and death) which typically require much longer time-horizons for their alteration, including the reduction of health inequalities by social class [17].

The SEDI uses teachers’ ratings of aspects of children’s development hence reporting bias is a potential limitation of the study. A vast body of literature exists which considers proxy versus direct assessment of child development and wellbeing. Reviews of child wellbeing assessment using Health-Related Quality-of-Life (HR-QOL) instruments considered parent-provided and self-rated approaches. In several studies a high correlation was shown for ‘observable’ components such as physical, and a low correlation for non-directly observable components such as emotional or social areas [34, 35]. Frameworks have been developed to delineate between various proxy perspectives and to guide inquiries into the validity and interpretation of viewpoints [36]. The age of children has been found to have a moderating function on the closeness of correspondence [35]. In fact, 5-year-old children have been found unable to understand a sufficient number of items to describe their health adequately [37].

The EDI assesses physical, cognitive, emotional and social aspects of children 4–6 years. Reliability testing has been conducted by the designers of the EDI [38–40]. Both test-retest reliability (using the Pearson correlation coefficient) and intra-rater or within-teacher reliability (using the intra-class correlation coefficient) tests have been conducted. Both have shown high coefficients of between 0.7 and 0.96 [38].

Conclusion

The SEDI’s psychometric properties are shown to be robust, and it is able to highlight developmental differences in children between socioeconomic and geographic areas. The tool’s simplicity and usability lend themselves easily to community-wide implementation across Scotland without further modification. It has the potential to offer communities a holistic overview of school readiness in their children, to both fuel support and provide a baseline for targeted interventions.

Endnotes

^aThe opt-out group comprised 11 females and 7 males. One of the opt-out pupils had been in class less than one month; two pupils’ data were recorded ‘opt-out’ because they had moved school.

^bOf the further 72 pupils excluded, 42 were recorded as having special needs and 30 pupils had been in class for less than one month or the length of time in class had not been recorded. Participants who had 30% of the questionnaire items missing and all those who had more than 1 scale of domain data missing were excluded. Only 1 pupil fell into the latter category and had already been excluded as he/she had been in the class for less than one month.

^cThe Scottish Index of Multiple Deprivation (SIMD) ranks small postcode areas (called data zones) according to level of deprivation by combining indicators such as current income, employment, health, education, skills, housing, and crime. Data zones can then be grouped into quintiles (five groups).