Accreditation standards
The Alberta childcare accreditation program, which was developed by the Government of Alberta in consultation with relevant childcare stakeholders, is voluntary but has historically had a high participation rate [17]. The accreditation process takes several months to complete and there is variability between centres on the timeframe for completion. The process starts with applying for pre-accreditation followed by a self-evaluation, the development of a quality enhancement plan, and the implementation of strategies within the plan to meet the standards. Once the strategies have been implemented and supporting documentation has been compiled, a formal on-site evaluation is requested [17]. Childcare programs can access support from coaches when developing their quality enhancement plan.
The accreditation program has existed since 2004, but in 2013 the Alberta government introduced new accreditation standards for childcare settings to be phased in until 2019. The previous accreditation standards and provincial childcare legislation did not mention physical activity or sedentary behaviour [15]. However, a new physical activity and sedentary behaviour standard criterion and related indicators were included in the new accreditation standards. Specifically, Alberta Childcare Accreditation Program Quality Standards included six standards, each with several criteria and related indicators. Standard 2 in the new document (“Program planning and practices support every child’s optimal development in an inclusive early learning and care environment that incorporates the value and importance of play”) includes criterion 2.2, “Child care programs promote physical wellness in all children and incorporate physical literacy in everyday programming” [17]. This criterion includes seven indicators, and the indicator this study primarily focused on was, “promote physical activity and minimize the time that children are sedentary” [17].
Participants and procedures
The supporting Healthy AcTive CHildcare settings study used a quasi-experimental two-group pre-post design. Participants were children aged 19 to 60 months, and their educators from Canadian childcare centres in and around Edmonton, Alberta (accreditation group) and Ottawa, Ontario (control group). Due to logistical reasons (i.e., the locations of the research team and the number of trips needed to participating centres) data were collected in only one city and its surrounding area in both provinces. Ottawa was selected as the control group because its population size is very similar to Edmonton (Edmonton: 932,546; Ottawa: 934,243 [19]) and no provincial policy changes that would impact physical activity and sedentary behaviour in childcare settings were expected for the course of the study. Children under 19 months of age who typically attend infant programs in Canadian childcare centres were excluded because some children in this age group are not walking independently making it challenging to accurately measure our primary outcome described below [20]. Data collection occurred in two phases, with half of accreditation and control centres participating in 2017/2018 and the other half participating in 2018/2019. Logistically, this ensured at both phases, baseline data were collected between October and early December, and follow-up data were collected approximately 6 months later between April and June. Similar to our pilot study, these time points were selected to minimize seasonal effects of winter and summer and minimize loss to follow-up due to summer vacations and transitions to kindergarten [21, 22].
For childcare centres to be eligible for the study, they had to be licensed with full-time toddler and preschool programs. Childcare centres in the accreditation group were also required to be starting the process of meeting requirements for the updated accreditation program (i.e., had recently applied for pre-accreditation in the applicable phase of data collection) and not be affiliated with another childcare centre that had already completed the program. When contact was made with the director, a total of 12/18 childcare centres in the accreditation group and 8/42 in the control group agreed to participate. Data could not be included from one childcare centre in Edmonton because they dropped out mid- study for unknown reasons, leaving 11 childcare centres in the accreditation group. The lower participation rate in the control group may be due to the lower interest in the study topic of Alberta policy. Both accreditation and control group directors were aware we were focusing on an accreditation criterion related to active behaviours. After completing the study, all childcare centres received a detailed report and $100 resource/equipment of their choice.
To be eligible for participation in the study, children had to be enrolled full-time, aged 19 to 60 months, and not expecting to change childcare arrangements for the duration of the study. A total of 269 children (accreditation: 141; control: 128) had a parent agree on their behalf to participate (participation rate: total = 47%; accreditation = 67%; control = 36%). However, after excluding 16 ineligible children (age: n = 7, part-time: n = 7, leaving centre n = 2), there were 253 eligible children for the study. To be eligible for participation in the study, educators had to work with children in the eligible age range and not anticipate leaving their job for the duration of the study. A total of 83 educators (accreditation: 44; control: 39) agreed to participate in the study (participation rate: total = 69%; accreditation = 88%; control =55%). However, after excluding 3 ineligible educators (educator in infant age group: n = 1, leaving the centre n = 2), there were 80 eligible educators. Ethics approvals were obtained from the University of Alberta and the University of Ottawa Research Ethics Boards. Written informed consent was obtained from all directors and participating educators as well as from parents/guardians for all participating children.
Data collection at baseline and 6 month follow-up in the accreditation and control groups included questionnaires for directors, educators as well as a one-day environmental assessment by research staff. The director and educator questionnaires both included questions on the strategies and barriers to physical activity and sedentary behaviour in childcare settings, awareness and familiarity of the Alberta Child Care Accreditation Standards, and demographic characteristics. The director questionnaire also included questions on the characteristics of their centre and experiences of the children in their care. The parent questionnaire included questions on demographic characteristics and their children’s activities outside of childcare. Additionally, at both time points, children wore accelerometers and children’s height and weight, and cognitive development were measured. However, cognitive development assessments were not conducted in toddler rooms, since the assessments are only validated for preschool-aged children [23].
Process evaluation measures
Three process evaluation measures were captured, including awareness and familiarity of the Alberta Child Care Accreditation Standards (released in December 2013) as well as stage of accreditation. Awareness and familiarity were assessed with one question each at baseline and follow-up in the educator and director questionnaires for both groups. The awareness question was answered with a dichotomous scale that had yes and no response options, and the familiarity question was answered with a Likert scale that had 5 response options ranging from extremely familiar to not at all familiar. Stage of accreditation was assessed at follow-up in the accreditation group by asking directors.
Primary outcome measures
In-care ST and physical activity
Children’s in-care ST and physical activity were measured with waist-worn wGT3X-BT ActiGraph accelerometers for six consecutive weekdays. Childcare staff were provided with verbal and written instructions regarding accelerometer procedures, including putting assigned accelerometer belts on children’s right hip when they arrived in the morning and taking them off before the children went home. Data were downloaded in 15-s epochs. Non-wear time data, defined as ≥80 consecutive 15-s intervals of zero counts (equivalent to ≥20 min of consecutive zero counts, were removed [24]). For children who napped, it was assumed that nap time data were removed with the non-wear time definition. To be included in the analyses, children were required to have ≥3 days with ≥240 total 15-s intervals (equivalent to ≥1 h) of wear time. This definition maximized the sample of children and has been used before in previous childcare studies [25, 26]. Wear time data were classified into ST (≤25 counts/15-s), light-intensity physical activity (LPA; 26–419 counts/15-s), and moderate- to vigorous-intensity physical activity (MVPA; ≥420 counts/15-s) based on validated cut-points for this age group [27,28,29]. Average minutes per hour of ST, LPA, and MVPA were calculated. SAS version 9.4 (SAS Institute, Cary, NC, USA) was used to conduct all accelerometer data reduction.
Secondary outcome measures
Body Mass Index (BMI)
At both baseline and follow-up, trained research staff measured children’s height to the nearest 0.1 cm using a portable stadiometer and their weight to the nearest 0.1 kg using a calibrated scale. Shoes and bulky clothing were removed prior to the measurements. Height and weight were measured twice unless there was a > 0.2 cm/kg difference between the two measurements then a third measurement was taken. An average of the two closest measurements were recorded. Age- and sex-specific BMI z-scores were calculated based on World Health Organization growth standards [30].
Cognitive development
Cognitive development was assessed in preschool rooms only by trained research staff at both baseline and follow-up using two iPad-based tasks from the Early Years Toolbox [23]. The Mr. Ant task measured the working memory aspect of executive functioning, and the Expressive Vocabulary task measured the expressive language aspect of language development [23]. Mr. Ant raw scores had a possible range of 1–8 and Expressive Vocabulary raw scores has a possible range of 0–45. A previous study has reported on psychometric properties for these tasks in 3–5 year olds [23]. Specifically, for convergent validity, significant correlations with medium to large effect sizes [31] were observed for comparable tasks in the National Institute of Health (NIH) Toolbox and British Ability Scales (r = 0.46–0.60) [23]. Similar to this previous study [23], good internal consistency reliability [32] was observed in our sample for Expressive Vocabulary (Baseline Cronbach’s α = 0.92). Since the Early Years Toolbox was developed for preschoolers aged 3 to 5 years, cognitive development data were excluded if children were less than 36 months of age in the preschool room.
Educator perceptions of strategies and barriers
Strategies and barriers for physical activity and sedentary behaviour in childcare settings were measured with 6 questions at both baseline and follow-up in the HATCH educator questionnaire. The questions were developed by the research team and advisory committee, which included key childcare stakeholders, based primarily on observations and director feedback from the pilot study that informed the present study [18]. Strategies to support regular physical activity were measured with 13 items and barriers were measured with 11 items. Strategies to limit extended periods of sedentary behaviour were measured with 8 items and barriers were measured with 6 items. Strategies to limit screen time were measured with 7 items and barriers were measured with 6 items. All questions also had space for other strategies or barriers. All items were answered with a Likert scale that had 5 response options ranging from never to always. An average score was calculated for each subscale based on the mean of available item scores. Except for strategies to limit extended periods of sedentary behaviour (α = 0.60), good internal consistency reliability [32] was observed for all questions at baseline (α = 0.73–0.90) in educators that had complete data on all items of the subscale (n = 50–64). The items were derived from the research team and took into account feedback from key stakeholders in the Alberta childcare sector.
Potential moderators
Children’s baseline age group and the childcare environment were considered as potential moderators. Children’s baseline age was calculated based on the first date of accelerometer data collection at baseline and their birth date as reported in the baseline HATCH parent questionnaire. Children’s age in months was categorized into toddler (19–35 months) and preschooler (36–60 months) age groups. The childcare environment was objectively assessed at baseline and follow-up during a one-day observation by a trained researcher using three tools: 1) Environment and Policy Assessment and Observation (EPAO [33]; 2017 version; physical activity component only), 2) Movement Environment Rating Scale (MOVERS) [34, 35], and the Children’s Physical Environments Rating Scale (revised version 5; CPERS5; part C and D only) [36, 37]. During pre-study training, this researcher along with another trained researcher achieved inter-rater agreement above 85% for all tools (85.2% – EPAO and MOVERS, 96.2% – CPERS5) after assessing two childcare centres. The one-day observations were scheduled on a typical day for each centre and focused on preschool rooms and areas because one of the tools (EPAO) has only been validated in preschoolers [33, 38]. Due to low variability of CPERS5 scores observed between centres and low internal consistency for part C, this tool was not included in the present study.
The physical activity component of the EPAO has 13 subscales, which include several aggregated items related to physical activity, screen time, and/or outdoor play and learning. The subscale scores were added to create an overall EPAO physical activity score, which could range from 0 to 39 points. A higher score indicates higher compliance with best practices. To our knowledge, psychometric properties have not been previously reported for the slightly modified EPAO 2017 version. However, previous research on the original EPAO has found similar inter–rater agreement as the present study (87.3% – observation, 79.3% – document review) [33]. MOVERS includes 11 items, which aggregate into 4 subscales related to physical development, movement, and practice. Each item was rated on a scale from 1 (inadequate) to 7 (excellent), and an average of item scores were calculated to create a MOVERS total score [34]. A previous study has reported on the reliability (test-retest: intraclass correlation coefficient (ICC) = 0.96, Weighted Kappa = 0.90, percentage agreement = 69–100%; internal consistency: Cronbach’s alpha = 0.89–0.94) and validity (concurrent validity with some sub-areas of the EPAO and Sustained Shared Thinking and Emotional Wellbeing (SSTEW) scale: Spearman’s rho = 0.57–0.87) of this tool [35].
Potential covariates
Children’s age (months), sex, and race/ethnicity as well as parental education were considered as potential covariates in statistical models for the primary outcomes and child secondary outcomes (BMI z-score, cognitive development). Additionally, weather variables including, mean temperature, and mean precipitation were considered as potential covariates in statistical models for the primary outcomes. Children’s birth date, sex, race/ethnicity, and parental education were reported in the baseline HATCH parent questionnaire. Questions were adopted from the Canadian Health Measures Survey [39]. Children’s age at baseline and follow-up were calculated as described above in the potential moderators section. Children’s race/ethnicity was categorized as Caucasian or other (Aboriginal, South Asian, Chinese, African, Filipino, Latin American, Arab, Southeast Asian, West Asian, Korean, Japanese, and other race/ethnicity) based on the frequency distribution. Parental education was categorized as below bachelor level (less than high school diploma, high school diploma, college or trade certificate or diploma), bachelor’s degree, or above bachelor level based on the frequency distribution. Mean temperature and mean precipitation were calculated at baseline and follow-up by averaging the mean daily temperature (i.e., average of the maximum and minimum temperature during a day) and daily total precipitation at the main Edmonton or Ottawa weather stations for the 6 days that children wore the accelerometer (Environment Canada).
Educators’ age (months), sex, and race/ethnicity were considered as potential covariates in statistical models for educator secondary outcomes (i.e., educator strategies and barriers). Educators reported on their birth date, sex, and race/ethnicity in the baseline HATCH educator questionnaire. Educator’s age at baseline and follow-up were also calculated using the first date of data collection and the educator’s birth date. Race/ethnicity was also categorized as Caucasian or other.
Statistical analysis
Statistical analyses were performed using SPSS version 26.0 (SPSS Inc., Chicago, IL, USA). Descriptive statistics were calculated for child, educator, weather, and childcare centre environment variables at baseline and follow-up (where applicable). T-tests or non-parametric equivalent (Mann-Whitney U Tests) and chi-squared tests were conducted to determine if baseline group differences (accreditation vs. control), and if applicable follow-up group differences, existed for the above-mentioned variables. Additionally, these tests were used to compare child age (months), sex, and group (accreditation and control) variables between children who had baseline and follow-up data and children who had only baseline or follow-up data for in-care ST and physical activity. Finally, descriptive statistics were calculated, and Mann-Whitney U Tests and chi-squared tests were conducted to compare baseline and follow-up process evaluation measures between groups (accreditation vs. control) for both directors and educators.
Statistical assumptions for linear mixed models were assessed by visual inspection. MVPA was log- transformed to meet the assumption of normality. Data for working memory was also not normally distributed but transformations did not improve the distribution. Therefore, working memory was categorized as a dichotomous variable based on the median score (< 1 [range: 0.00–0.67] and ≥ 1 [range:1–3]) [40]. Additionally, LPA and log MVPA were truncated for one participant because they were identified as outliers according to pre-specified criteria [41]. Linear mixed models were conducted to estimate marginal means and standard errors at baseline and follow-up as well as mean differences and standard errors for each outcome variable within each group (accreditation and control). In model 1, the clustering effect of childcare centres and the repeated effect of time were included as random intercepts, and group (accreditation vs. control), time (follow-up vs. baseline), and time*group were included as fixed effects. Model 2 also included covariates as fixed effects where baseline differences and applicable follow-up differences were observed between accreditation and control groups.
To address the primary study objective, linear mixed models were conducted separately for ST, LPA, and log MVPA. In model 1, the clustering effect of childcare centres and the repeated effect of time were included as random intercepts, and group, time, time*group were included as fixed effects. Additionally, in model 2, covariates were included as fixed effects where group (accreditation vs. control) baseline and applicable follow-up differences were observed. If the interaction term (time*group) was significant, stratified analyses were conducted to examine the change in the primary outcome (follow-up vs. baseline) separately in the accreditation and control groups. If the interaction term was not significant, then models were rerun with the interaction term removed to examine the change in the primary outcome (follow-up vs. baseline) in the entire sample, with the accreditation and control groups combined. For these models, additional covariates were considered. Specifically, if a significant difference was not observed between accreditation and control groups but was observed between baseline and follow-up time points, the covariate was included in the model.
To address the first secondary objective, child baseline age group or the childcare environment and a three-way interaction term between group (accreditation vs. control), time (follow-up vs. baseline), and baseline age group (preschoolers vs. toddlers) or continuous environment data were added to the model 1 and model 2 as fixed effects. Once again, if an interaction term was significant, stratified analyses were conducted. To address the second and third secondary objectives, the same modelling strategy was used that addressed the primary objective, except generalized linear mixed models were used for working memory, and a different set of potential covariates were considered. For all mixed models, participants were included if they had data on the model outcome at baseline and/or follow-up, and for model 2 if they also had complete covariate data for that model. Sensitivity analyses were conducted by rerunning the models in the sample of children that had complete data at both points. Additionally, models for the primary outcome were rerun based on the accreditation status at follow-up. Statistical significance was set a priori at p < 0.05.