Population and Sample
Participants were students from low-income (90-99% of students qualified for the US Federal Free or Reduced Price Lunch Program) elementary schools in the Houston Independent School District of Houston, Texas, the fourth largest US city. For Objective 1 (school travel survey validation), students were recruited from 4th grade classrooms of two schools that predominantly enrolled either African American (85%) or Hispanic (74%) students. Eligibility was restricted to 4th grade students since this study was conducted in preparation for a randomized controlled trial evaluating a walking school bus intervention among 4th grade students in the Houston Independent School District. For Objective 2 (pedestrian safety behavior checklist validation), students from one elementary school (kindergarten through 5th grade) that predominantly enrolled African American students (90%) were observed. Low-income ethnic minority participants were recruited because they are generally underrepresented in childhood obesity and injury prevention research, yet are substantially impacted by both obesity and unintentional injuries [16].
Surveys
SRTS travel survey (Objective 1)
We adapted the publicly available SRTS travel survey from the US National Center for Safe Routes to School website [23]. Instead of asking students to raise their hands to indicate how they traveled to school (which could have a strong social influence bias on responses), we administered a written survey [Additional file 1]. The survey obtained each child's name, parents' contact information, and asked one question, "How did you get to school today?" The students could chose among seven potential responses, "rode school bus, came by carpool, came by car, rode metro bus, walked with an adult, walked without an adult, or biked". Students were instructed to mark the one answer that best showed how they got to school. Parents were contacted on the same day by study staff and asked, "How did [child's name] get to school today?" Parents also answered questions on the child's date of birth, gender, and race/ethnicity.
Pedestrian safety behavior checklist (Objective 2)
The principal investigator (PI) developed a pedestrian crosswalk behavior observation checklist based on previous observation elements that had good reliability [28–31]. Specifically, we observed and scored (yes/no) the following pedestrian safety elements: crossed at a corner or crosswalk, crossed with an adult or safety patrol, stopped at the curb, looked left-right-left, kept looking while crossing, walked and did not run across the street, and followed the traffic signal (if present). We added an item to assess if the participant was a part of a walking school bus, operationally defined in this study as a group of children wearing bright reflective vests and led by an adult [Additional file 2].
Procedure
SRTS travel survey (Objective 1)
We recruited 4th grade students, aged 9-11 years, to complete in their classrooms the one-question written survey, available in English and Spanish. Students could "opt-out" of the study if asked by their parents, who received an informational letter describing the study, or if they did not wish to participate in the study themselves. Study staff administered the survey by asking the students, "how did you get to school today?" in English or Spanish (as appropriate) and directing them to indicate their answer on the survey. Students also provided their full names and telephone numbers, to facilitate contact with their parents, who were asked the same question in English or Spanish along with brief demographic questions. Test-retest reliability was determined on the same day by repeating the student survey 3-4 hours later in the same classroom. Convergent validity was determined in comparison to parents' report for that day, similar to previous school travel validation studies that compared student report to parent report [26, 27].
Pedestrian safety behavior checklist (Objective 2)
The PI trained 10 research assistants in a 1.5 hour session to complete the pedestrian safety behavior checklist. The first 1/2 hour of training was spent discussing pedestrian safety and the checklist, while the final hour was spent in the field observing and scoring pedestrians with the PI, who also gave feedback on correct scoring. After training, the PI and research assistants unobtrusively observed child pedestrians walking toward the study school in the morning prior to classes. Children on bicycles, skateboards, scooters, or riding in strollers were excluded, since this checklist was designed to assess pedestrian safety only. Ten research assistants and the PI observed a convenience sample of 29 students chosen by the PI (about 1/2 were female) at a major school intersection. For groups of children crossing the street, the PI chose only one student for the research assistants to observe, consistent with a previous study [30]. The PI and research assistants did not interact with the children (to avoid influencing their behaviors), thereby eliminating the possibility of collecting socio-demographic information on individual children. The PI served as the comparison because he was a board-certified pediatrician with advanced public health training in injury prevention, physical activity, and SRTS research. He also developed the pedestrian safety behavior checklist, trained the research assistants in the use of the checklist, and observed the study participants simultaneously with the research assistants. While an objective measure would have been ideal, no such standard exists for child pedestrian safety observations.
This study was approved by the Institutional Review Board of Baylor College of Medicine and the Research Department of the Houston Independent School District.
Analysis
SRTS travel survey (Objective 1)
Frequencies and percentages were used to describe participant characteristics. To assess the reliability of the children's rating of the SRTS travel survey, the percentage of agreement and the kappa test statistic for agreement assessed the reliability between the children's initial SRTS ratings and the child ratings given 3-4 hours later. Similarly, the percentage of agreement and the kappa statistic for agreement assessed the convergent validity between the children's initial SRTS survey responses and their parent's responses, which served as the comparison.
Pedestrian safety behavior checklist (Objective 2)
To assess systematic differences between rater values and the PI, an analysis of variance with a priori contrasts on the pairwise comparisons was used. Spearman correlation was used to determine the reliability between the raters's and the PI's total scores. The mean of each rater and the PI's score was correlated with the difference between each rater and the PI's score to evaluate whether errors from the rater was associated with the mean difference (a Bland-Altman plot) [32]. Relative reliability, the extent to which the rater's scores were the same as the PI, was assessed using the two-way mixed intra-class correlation (ICC). Generalizability theory identified the contribution to variability from different sources (the behavior, the rater) and provided a generalizability coefficient, a form of the intra-class correlation, which incorporated the different sources of error [33].
To determine the probability that the rater reported that the behavior was performed, when the individual actually performed the behavior (based on comparison to the PI), the sensitivity was computed. Conversely, the specificity was computed to determine the probability of the rater reporting that the behavior was performed when the individual did not perform the behavior. All analyses were performed using SAS 9.1.3 (SAS Institute Inc., Cary, North Carolina, USA).