Physical Literacy Knowledge Questionnaire: feasibility, validity, and reliability for Canadian children aged 8 to 12 years

Background Physical literacy is defined as the motivation, confidence, physical competence, and knowledge and understanding to engage in physical activity for life. Physical literacy knowledge and understanding encompasses movement (how to move), performance (evaluation of movement), and health and fitness (value of exercise, need for relaxation and sleep, etc.). This paper describes the development and evaluation of a standardized assessment of physical literacy knowledge and understanding for Canadian children in grades 4, 5, and 6. Methods Proposed Physical Literacy Knowledge Questionnaire (PLKQ) content was identified through expert consultation and a review of provincial/territorial physical education curricula for grades 4 to 6. Open-ended questions verified language and generated response options. Feasibility was assessed via completion time and error frequency. Item validity assessed scores by age, gender, and teacher ratings of student knowledge. Test-retest reliability was assessed over short (2-day) and long (7-day) intervals. Results Subsets of 678 children (54% girls, 10.1 ± 1.0 years of age) completed the feasibility and validity assessments. Response errors (missing or duplicate responses, etc.) were minimal (2% or less) except for one question (7% error) about the use of safety gear during physical activity. A Delphi process among experts in children’s physical activity and fitness achieved consensus on the core content and supported an item analysis to finalize item selection. As expected, knowledge scores increased with age (partial eta2 = 0.07) but were not related to gender (p = 0.63). Teacher ratings of children’s knowledge of physical activity behaviour (r = 0.13, p = 0.01) and fitness (r = 0.12, p = 0.03), but not movement skill (r = 0.07, p = 0.19) were associated with PLKQ scores. Test-retest reliability for PLKQ score and individual questions was substantial to excellent for 71% of comparisons over a 2-day interval, but lower over a 7-day interval (53% substantial or excellent). Items with low reliability had high or low proportions of correct responses. Conclusions This study provides feasibility and validity evidence for the Physical Literacy Knowledge Questionnaire as an assessment of physical literacy knowledge for Canadian children in grades 4, 5, and 6. Completion rates were high and knowledge scores increased with age. Streamlining of the content in accordance with Delphi panel recommendations would further enhance feasibility, but would also focus the content on items with limited reliability. Future studies of alternative item wording and responses are recommended to enhance test-retest reliability. Electronic supplementary material The online version of this article (10.1186/s12889-018-5890-y) contains supplementary material, which is available to authorized users.


Background
Physical literacy is defined as the motivation, confidence, physical competence, and knowledge and understanding to engage in physical activity for life [1]. It is expected that children who have progressed further along their physical literacy journey are better able to adopt a healthy active lifestyle. It is also desirable that children with lower physical literacy be identified in order to provide them with additional support. According to Whitehead [2], physical activity motivation reflects a "willingness and eagerness" to take action that is demonstrated as a joy of movement, confidence in one's own physical abilities, a positive attitude toward participation, and an expectation of successful participation. Physical competence reflects not only competence in movement skill but also the capacity (e.g., strength, endurance, etc.) for movement. Knowledge and understanding encompasses movement (how to move), performance (evaluation of movement), as well as health and fitness (value of exercise, need for relaxation and sleep, etc.). For brevity, we will use the term "knowledge" throughout this paper to represent the knowledge and understanding domain of physical literacy. Standardized protocols are available to assess the elements of motivation and confidence [3,4], physical competence [5][6][7][8] and engagement in physical activity [9] in some age groups. However, although physical activity knowledge is a universally stated outcome of Canadian physical education curricula [10], a standardized measure of physical literacy knowledge and understanding has not been identified.
The purpose of this study was to develop and evaluate a standardized assessment of physical literacy knowledge and understanding. The target population was Canadian children in grades 4, 5, and 6 [11]. This pre-adolescent age group was selected because they have not yet experienced the decline in physical activity that occurs during adolescence [12], but are still able to independently respond to survey questions. The goal was to include an assessment of knowledge and understanding within the Canadian Assessment of Physical Literacy (CAPL) to ensure that the four domains of the CAPL (Knowledge and Understanding, Physical Competence, Motivation and Confidence, and Daily Behaviour) would be consistent with the current Canadian consensus definition of physical literacy [1].

Study design overview
The development of the Physical Literacy Knowledge Questionnaire (PLKQ) was completed through a series of studies as described in Fig. 1. Initially, proposed content was identified through a review of physical literacy knowledge components of the physical and health education curricula, combined with input from education professionals and expert advisors. Potential questions (Additional file 1) were then tested by providing students in grades 4, 5, and 6 with the opportunity to respond in an open-ended format. Responses were qualitatively analyzed to optimize item wording and to generate a list of response options suitable for a closed-ended format. Feasibility of the initial PLKQ (Additional file 2) was evaluated by having students respond, in pencil and paper format, to the closed-ended questions. Response errors and teacher reports of each student's knowledge were evaluated. Reliability was assessed by having students complete the PLKQ on two separate occasions. The datasets supporting the conclusions of this article are available from Dr. Patricia Longmuir.

Participants
Study participants were convenience samples of children attending schools and summer camps in Ontario, Canada, who had agreed to cooperate with our research. All children at collaborating schools and camps were approached to participate, and those who assented and whose parents provided written consent were enrolled. Children tested in schools were in grades 4, 5, or 6. Fig. 1 Overview of research to develop the Physical Literacy Knowledge Questionnaire*. * All participants in all phases of this research described above were students in grades 4, 5, or 6, or children attending summer camps who would be in grades 4, 5, or 6 when returning to school after summer vacation Children tested in summer camps were going to be in grades 4, 5, or 6 when they returned to school in September. The majority of study participants were aged 9 (Grade 4), 10 (Grade 5) or 11 (Grade 6) years. There were smaller numbers of children 8 and 12 years of age. Children 8 years of age were in Grade 4 and born late in the year (October/November/December) but who were tested in the fall (September to December) before their ninth birthday. Children 12 years of age were in Grade 6 and born early in the year (January/February/March/ April) and tested after their 12th birthday.
Informed written consent was obtained from the parents of all children before enrolment. Teachers who participated in the rating of student knowledge also provided written informed consent prior to study participation. Verbal child assent was obtained before the commencement of study activities. Study activities were approved by the Children's Hospital of Eastern Ontario and University of Windsor Research Ethics Boards as well as by the research committees of participating school boards and camps.

Process of development for the PLKQ
Content areas for the PLKQ were systematically identified through a review of physical and health education curricula from all Canadian provinces and territories, supplemented by the recommendations of an international Delphi process [13]. Key learning objectives from each curriculum document for grades 4, 5, or 6 were identified (Table 1). A content analysis of the key learning objectives identified the following areas of knowledge as being common across all of the reviewed curricula: importance of physical activity, definition of cardiorespiratory fitness, guidelines for daily physical activity and sedentary time, definition of "healthy", recognition of movement skills, understanding of fitness and its impact on physical activity, safety practices during physical activity, identification of healthy foods, and methods of skill and fitness improvement. Open-ended questions on the proposed topics to be assessed were then provided to children in grades 4, 5, and 6 as well as their teachers (Additional file 1). Feedback was obtained on the clarity and wording of the questions from both teachers and students. In addition, the children's responses to the open-ended questions were used to identify the closed-ended response options that would be included in the initial PLKQ (Additional file 2).

Feasibility of the PLKQ
Feasibility analyses were conducted among children assessed at schools in eastern Ontario during development of the CAPL. Children completed the initial PLKQ (Additional file 2) during class time. The proportion of children in grades 4, 5, and 6 able to complete the initial PLKQ without missing or other completion errors (e.g., multiple responses to one item) was evaluated. Completion time was analyzed in relation to instrument feasibility. A factor analysis assessed the difficulty of each question based on the proportion of correct/incorrect responses.

Validity of the PLKQ
Content for the initial PLKQ was verified through an international Delphi process [13]. Experts in children's physical activity, movement, motivation, and fitness achieved consensus on initial PLKQ content through an iterative process. Results from administration of the initial PLKQ to children assessed in schools in eastern Ontario were analyzed by the child's self-reported age and gender. It was hypothesized that knowledge would not vary by gender but would increase with age.

Reliability of the PLKQ
Test-retest reliability data were collected from two samples of children who were asked to complete the PLKQ twice. One sample (n = 31) attended a summer camp in eastern Ontario, and completed the initial PLKQ on 2 separate days, at an interval of 2 days. A second sample of children (n = 35) completed the final PLKQ (Additonal file 3), a more streamlined version, at their school in southwestern Ontario, on 2 occasions over a 1-week interval. Test-retest reliability of the PLKQ responses was assessed using Pearson correlation coefficients.

Statistical analyses
Descriptive statistics were used to summarize the data (mean ± SD, frequencies) and the participants in each phase of this research. Regression analyses investigated the impact of independent variables on study outcomes. Correlations were used to evaluate test-retest reliability. Due to the large sample size used for the feasibility and validity analyses, results were interpreted based on measures of effect size. According to Murphy and Myors [14], small, medium, and strong effects are observed when partial eta squared exceeds 0.01, 0.06, or 0.14, respectively. Correlations are defined as small (> 0.10), moderate (> 0.30), strong (> 0.50), substantial (> 0.70), or excellent (> 0.90).

Feasibility of the PLKQ
Feasibility and validity analyses were conducted on the same sample of children, whose demographic information is summarized in Table 2. A total of 678 children (54% girls), mean age 10.1 ± 1.0 years completed the feasibility assessment of the initial PLKQ. Of these, a calculated PLKQ score was available for 487 children (28% had response errors [i.e., were missing or responded Gr. 5 Gr. 6 Gr. 4 Gr. 5 Gr. 6 Gr. 4 Gr. 5 Gr. 6 Gr. 4 Gr. 5 Gr. 6 Gr. 4 Gr. 5 Gr. 6 Gr. 4 Gr. 5 Gr. 6 Gr. 4 Gr. 5 Gr. 6 Gr. 4 Gr. 5 Gr. 6 Gr. 4 Gr. 5 Gr. 6 Gr. 4 Gr. 5 Gr. inappropriately to two or more items]). Among the 191 children with response errors, the number and type of errors for each item in the initial PLKQ are summarized in Table 3. A factor analysis demonstrated a good distribution of easy, moderate, and difficult questions ( Table 4). The most common response error occurred with the question regarding safety gear. Children were presented with pictures of different types of physical activity (see question #16, Additional file 2). They were asked to circle the activities that they themselves perform, and then add a check mark to the activities for which they wear safety gear (e.g., helmet, elbow pads). The score assigned was based on the proportion of correct responses among only the activities that the child actually performs. Fifty (7%) children showed a limited comprehension of the question instructions, as they checked pictures that they had not circled, indicating that they knew safety gear was required but did not understand the instruction to  circle the activities that they actually performed. Other common errors, each affecting about 2% of responses, were missing information or lack of understanding for the questions regarding whether the child uses active transportation to get to school, and multiple answers in response to questions about activity preferences after school and reasons for being active. Mean completion time was assessed in order to further clarify the feasibility of the PLKQ. Among 117 children who were timed completing the PLKQ using the pen and paper format, the mean completion time was 43, 42, and 38 min for students in grades 4, 5, and 6, respectively. Among 851 children who completed the PLKQ using the online platform (www.capl-ecsfp.ca), completion time was 27 min for Grade 4 students, 26 min for Grade 5 students, and 14 min for students in Grade 6. These analytics track only the time that each student required to complete the knowledge component of the PLKQ, which is comprised of 9 questions requiring 38 responses. The time required for questions #2, #3, and #12 of the PLKQ, which are part of the Motivation and Confidence domain score, were excluded. Table 5 summarizes the consensus recommendations from the Delphi panel with regard to the PLKQ content. The Delphi expert panel (n = 19, 4 female [21%]), who had 25 ± 15 years of research experience within their field (range: 5 to 65 years) and a combined total of 4181 peer-reviewed publications (range 15 to 1500), agreed that children's knowledge of daily physical activity and screen time guidelines, the meaning of cardiorespiratory fitness and muscular strength and endurance, and how to improve sport skills and fitness were important areas of knowledge that should be included in the PLKQ. There was also consensus for the inclusion of questions about the child's use of active transportation and self-reported days per week of moderate to vigorous physical activity. Questions probing the child's understanding of the meaning of "healthy" and the need for safety gear during physical activity were also close to consensus (70% and 72% respectively, where 75% was required for consensus). Other areas of consensus among the Delphi panel participants were that factor item analysis should be used to determine the final content for the PLKQ (see paper by Gunnell et al. in this In a linear regression model, knowledge of physical literacy as assessed by the initial PLKQ was significantly associated with increased age (F = 174.5, p < 0.001, partial eta squared = 0.068 (medium effect [15]); Table 6). There was no relationship between initial PLKQ score and self-reported gender (F = 0.24, p = 0.63). Table 7 compares teacher ratings of the children's knowledge about physical activity behaviour, fitness, and movement skill to the children's initial PLKQ score. Teachers were asked to rate each child's knowledge on a scale from 1 (low knowledge) to 10 (excellent knowledge). Separate ratings were provided for knowledge of physical activity behaviour, physical fitness, and movement skill. Teachers provided ratings for those students to whom they taught physical education classes. Twenty-five teachers (5 [20%] male) rated 516 students (median 23 students per teacher, range 2 to 37). Significant correlations were observed between the initial PLKQ score and teacher ratings of the child's knowledge of physical activity behaviour and fitness. Teacher ratings of the child's knowledge about movement skill were not associated with the initial PLKQ score. In a multi-variable regression model adjusted for the child's self-reported age and gender, only the teacher rating of the child's fitness knowledge was significantly associated with initial PLKQ score (F = 4.3, p = 0.005, partial eta 2 = 0.034 [small effect]). For each 1-year increase in self-reported age, the initial PLKQ score increased by 0.36 points. For each 1-point increase in the teacher rating of the child's fitness knowledge, the initial PLKQ score increased by 0.19 points. Gender was not significantly related to PLKQ score (p = 0.71). The test-retest correlation coefficients for the PLKQ total score, as well as for individual items on the PLKQ, are provided in Table 9. Reliability of the PLKQ total score was strong (r = 0.62 and 0.69 over the 2-and 7-day intervals, respectively). Adjusting the correlation by age did not alter the reliability (r = 0.60 and 0.70 over the 2-and 7-day intervals, respectively). Over a 2-day interval, the reliability of most individual items was substantial to excellent. Responses to questions about the meaning of "cardiorespiratory fitness", a story about sport training and fitness, and how to get in better shape had moderate reliability. Item reliability over a 7-day interval was similar for most questions. Reliability was lower over a 7-day interval, compared to the 2-day interval, for questions asking about the use of safety gear during physical activity and the meaning of "healthy".

Validity of the PLKQ
Reliability for the question that asked children about the recommended amount of daily sedentary time was moderate over both the 2-and 7-day intervals (r = 0.33 and 0.34, respectively). Even though the proportion of children answering correctly was similar (21 and 20%, respectively), a paired comparison of responses was significantly different (p = 0.045) between the first and second trials over the 7-day interval. Over a 7-day interval, 72% and 77% of children correctly identified the recommended daily physical activity (1st and 2nd trial, respectively) and the correlation was strong (r = 0.61). Over a 2-day interval, reliability was low (0.20) and may have been influenced by the content of the initial PLKQ, which asked separately about moderate to vigorous activity at school and total throughout the day.

Discussion
Generally, the PLKQ proved to be feasible, valid, and reliable for children in grades 4, 5, and 6 (8 to 12 years of age). Rates for missing or incomplete responses, assessed using the pen and paper format of the initial PLKQ, were low. Less than 2% of all questions had incomplete or missing responses, except for the question asking about safety gear, for which 50 children indicated the need for safety gear without indicating that they participated in the activity. Use of the online format for the PLKQ requires complete responses in order to log off from the website. Validity of question content was supported through a Delphi process, a balanced array of item difficulty, and the finding that PLKQ scores increased with age but did not differ by gender. Test-retest reliability was substantial to excellent for most questions over a 2-day interval, with some questions having moderate reliability, particularly over a 7-day interval.
The Delphi panel achieved consensus for the inclusion of 6 content areas within the PLKQ: knowledge of daily physical activity and screen time guidelines, the meaning of cardiorespiratory fitness and muscular strength/endurance, and how to improve fitness or sport skills [13]. Content related to the use of safety gear and the Maximum score for the initial PLKQ was 18 points Table 7 Association between Physical Literacy Knowledge Questionnaire score and teacher ratings of a child's knowledge meaning of "healthy" came very close to achieving consensus. The Delphi panel was either neutral or in agreement with content items for the benefits of sport and fitness participation and the identification of movement skills. These items were retained in the PLKQ to be inclusive of relevant theoretical concepts and to more fully reflect the physical and health education curricula. The Delphi panel was either neutral to, or disagreed with, the inclusion of item content related to self-reported sleep time and the use of sunscreen; these items were subsequently removed when the streamlined final PLKQ was created. The inclusion of self-reported sedentary time was supported by 47% of the Delphi panel, but opposed by 42% of those experts. Those opposed indicated that self-reported sedentary time was a measure of behaviour rather than knowledge. Therefore, the item was removed from the final PLKQ. Teacher ratings of each child's knowledge of fitness, physical activity behaviour, and movement skill were intended to contribute to the assessment of PLKQ validity. Although there were statistically significant (p < 0.05) correlations between PLKQ total score and teacher ratings of children's physical activity and fitness knowledge, the correlations were small in magnitude (r < 0.15) and, therefore, explained less than 2% of the variance in initial PLKQ responses. PLKQ total score was not significantly correlated with teacher ratings of children's knowledge of movement skill. If teacher ratings of the children's knowledge were assumed to be the "gold a All children were in grades 4, 5, or 6. Children 8 years of age were in Grade 4 and born late in the year (October/November/December) who were tested in the fall before their ninth birthday. Children 12 years of age were in Grade 6 and born early in the year (January/February/March/April) and tested after their 12th birthday  [14]. Substantial or excellent correlations are shown in bold text PLKQ Physical Literacy Knowledge Questionnaire standard" reference, these results would seem to suggest that the initial PLKQ may have limited validity as an assessment of physical literacy knowledge. However, the three teacher ratings by content area (fitness, behaviour, movement skill) were strongly associated (p > 0.60) with each other, suggesting that teachers had consistent expectations for physical literacy knowledge, such that students were rated as having either good or poor knowledge in all content areas. Teacher ratings were provided by each child's physical education teacher. At times this was the regular classroom teacher and at other times it was someone else (e.g., physical education specialist, camp or program leader). Therefore, the limited association between teacher ratings and initial PLKQ scores may reflect the fact that some teachers had no knowledge of the child's classroom learning or performance. Future research should evaluate the relationship between student knowledge and teacher ratings, adjusting for teacher familiarity with the student's classroom performance. These results may also reflect the ability of the initial PLKQ to discriminate between the child's knowledge in different physical literacy content areas. While test-retest reliability for most items (71%) on the initial PLKQ was substantial to excellent over a 2-day interval, the reliability of items on the final PLKQ was lower over a 7-day interval (53% substantial or excellent). Items with limited reliability over both short and longer intervals included daily recommendations for physical activity and screen time, benefits of sport training and fitness, and how to "get in better shape". It is possible that differences in reliability of these items occurred because children were curious about the correct answer to these questions after the first trial, and were prompted to seek out the correct response prior to the second trial. Data supporting this explanation come from the question about recommended daily physical activity. Over a 2-day interval, only 4 children had an incorrect response on the first day of testing, and 3 of these 4 answered the question correctly on the second day. Over the 7-day interval, only 9 of 35 responses were incorrect on the first day, with 3 becoming a correct response when the assessment was repeated. In spite of the limited reliability for these items, they were retained in the final PLKQ based on the Delphi panel consensus support and theoretical arguments that this knowledge is important to evaluate.
Low reliability for the question asking how to "get in better shape" may have reflected different interpretations. Examiners reported that some children asked if the question meant to improve one's physical appearance. Although "get in better shape" was the preferred wording identified in response to the initial open-ended questions (Additional file 1), a change to the wording of this item may improve understanding of the question's intent, ultimately improving the reliability of responses.
Responses to the items relating to daily screen time recommendations and the benefits of fitness and sport training suggest that only a small proportion of children could respond correctly. The screen time item has 4 response options (1 correct and 3 incorrect). The "benefits of fitness and sport training" item was a paragraph / story that required children to fill in 9 (initial PLKQ) or 5 (final PLKQ) blank spaces from a list of word options provided. If most children did not know the correct responses to these items, and therefore were "guessing", it is reasonable to expect that the lower reliability measures for these questions may reflect the children's need to randomly choose among multiple response options.
The mean time required to complete the final PLKQ was estimated to be 14 min for students in Grade 6, but 27 min for students in grades 4 and 5. This represents a substantial time burden for the younger students, particularly if there is also a desire to assess their physical literacy motivation. Future research is needed to reduce the response burden for younger students.

Strengths and limitations
To our knowledge, this study is the first report of the psychometric properties of a physical literacy knowledge assessment for children. Feasibility was assessed in a large sample of Ontario children attending grades 4, 5, and 6 classes. The content of the assessment was matched to the physical and health education curricula published for each Canadian province and territory, and verified through a Delphi expert consensus process [13]. Participants in all phases of this research were convenience samples of children attending local schools or day camps in Ontario. Therefore, the extent to which the study sample is representative of the population of Canadian children as a whole remains unknown. Additional investigations of item reliability are recommended given the relatively small sample sizes reported here, and differences in the item content between the 2-(initial PLKQ) and 7-day (final PLKQ) intervals. The psychometric properties of questions modified in the future should also be assessed and reported.

Conclusions
The results of this study provide evidence for the feasibility, reliability, and validity of the PLKQ as an assessment of physical literacy knowledge and understanding among Canadian children in grades 4, 5, and 6. Completion rates were high and knowledge scores increased with age. Streamlining of the content in accordance with Delphi panel recommendations would further enhance feasibility, but would also focus the content on items with lower reliability. Future studies of alternative item wording and responses are recommended to enhance test-retest reliability.