Measuring sport experiences in children and youth to better understand the impact of sport on health and positive youth development: designing a brief measure for population health surveys

Understanding youths’ experiences in sport

The impact of youth sport has garnered significant interest across disciplines, with studies examining the potential health impacts of sport participation [20, 21], academic impacts [22, 23], social and psychological impacts on the youth [7], impacts on delinquency [10], and societal benefits [24, 25]. Measuring youths’ experience in sport is essential to advance research across disciplines. To develop the SEM:CY, we will draw heavily from the positive youth development (PYD) in sports literature. Youth experience in sport has garnered significant attention in the PYD in sport literature. PYD is an area of psychology that is interested in the competencies and processes that result in positive development [3].

Recently, Holt and colleagues’ [26] developed the Grounded Theory of Positive Youth Development through Sport model based on a recent synthesis of 63 qualitative studies on positive youth development through sport. This model includes distal social-ecological systems and individual characteristics that influence the sport, features of the sport climate that impact sport experiences, and program focus that contribute to positive youth development outcomes (see Fig. 1).

Consistent with our focus on youths’ subjective experience in sport (more proximal influences), we focus on the three elements of the sport climate that support positive youth development outcomes in the model: relationships with coaches (identified as adults in the model), relationships with teammates (identified as peers in the model), and involvement of parents. As our intention for the SEM:CY is to apply to outcomes beyond PYD, we also draw on the broader sport literature to ensure comprehensive coverage of youth sport experiences. Not surprisingly, the importance of relationships with coaches, parents, and teammates in sport has also been identified in the broader sport literature as essential to youths’ experiences in sport [5, 27–31]. Additionally, other influencers such as family members other than a parent (e.g., siblings), parents of others on the team, and spectators at games has been identified as important to youth’s sport experience, though they have received less attention in the literature to date [9, 32].

Most research on youths’ sport experience has focused on the role of the coach. In particular, the importance of the coach in establishing the motivational climate of the sport is one of the most established aspects of youths sport experience. Motivational climate is grounded in Achievement Goal Theory [33, 34] and considers different ways for interpreting competence: self- referenced competence that focuses on skill development and mastery and external referenced competence that evaluated competence in reference to others (e.g., winning and comparing teammates to each other). When coaches that emphasize skill development and mastery, youth are more likely to show positive outcomes whereas coaching behaviours that emphasize winning and comparisons with others are associated with negative outcomes including anxiety and stress, conflict with peers, less sportsman-like behaviour, and negative developmental outcomes [35].

Teammate and parents are also important in establishing the motivational climate of the sport. While the role of both teammates and parents have received less empirical attention than coaches, [35] the impact of both teammates and parents has been demonstrated independent of the influence of coaches [36–39]. When teammates emphasis skill development and mastery (i.e., self-referenced motivational climate), youth report more enjoyment of sport, pro-social sport attitudes (e.g., congratulating an opponent after a loss) and fewer antisocial sport attitudes (e.g., attitudes towards cheating and gamesmanship). The same pattern of results has been found with parental motivational climate: emphasis of skills is related to positive outcomes, while emphasizing winning is related to negative outcomes [35].

The importance of the coach-athlete relationship is another established element of youths’ sport experience. A positive relationship and rapport between athletes and coaches is related to a number of positive developmental outcomes including higher perceived competence, pro-social norms, and lower stress whereas negative rapport with coaches is associated with higher stress and negative peer dynamics [40–42]. There is a dearth of research on the relationship of teammates and parents relative to coaches. However, the research that has been conducted reports positive relationships with teammates and parents in sport is associated with positive development including perceived physical competence and personal and social skills [43–46].

While there is recognition that coaches, teammates, and parents each play an important role in youths’ sport experience, there is some evidence of other influences on youth sport experience. For example, positive spectator behaviours (e.g., cheering for members of both teams) and negative spectator behaviours (e.g., taunting players or arguing with officials) has been associated with the sportsmanlike behaviour of young athletes during games [32]. We propose that these types of behaviours can influence the motivational climate of the sport. Arguing or aggressiveness towards officials can indicate that winning is highly valued whereas recognizing skill and achievements on both teams can indicate the value of skill. Similar to relationships within sport, the impact of relationships with other family members (e.g., siblings) and peers aside from teammates is an area that has been identified as warranting further investigation [11].

In summary, a sport experience where coaches, teammates, parents, and others engage in behaviours that foster skill development, do not overemphasize winning, and youth have positive relationships is likely to produce a more quality sport experience and enhance developmental outcomes. Alternatively, an experience that focuses on winning, where negative behaviour is modeled, and there are conflicts with relationships is more likely to produce a lower quality experience and be associated with negative developmental outcomes.

Based on this review, we propose an initial conceptual model for the SEM:CY measure that includes the following domains: relationships with coaches, relationships with parents, relationships with teammates, relationships with others, coach motivational climate, parental motivational climate, teammate motivational climate, and other motivational climate (see Fig. 2). This model will act as a starting point for item generation and will be thoroughly tested and refined through expert input and statistical analysis as detailed in our methodology.

SEM:CY in relation to existing measures

While there are a number of surveys that capture some elements of youths’ experience of sport, there is no single measure that fully captures all of the domains of our conceptual model. In fact, even providing partial coverage of the proposed SEM:CY domains would require several measures. For example, there are separate measures of motivational climate for coaches, parents, and peers respectively [47–49]. These measures were designed specifically for motivational climate, so do not capture the quality of relationships. Other measures that focus on the source of influence in sport such as the Coaching Behaviour Scale for Sport [50] and the Coach Athlete Relationship Scale [51] were not designed to measure motivational climate. Using existing measures to capture youths’ sport experience would require significant time to complete which is not feasible for survey research. This is especially true for surveys that are designed to measure health and youth development across multiple domains. There are also potential gaps in existing measures of youths’ experience in sport. For example, the perceived impact of others in the sport community on motivational climate and the impact of sport on relationships with others outside sport are not captured with an existing measure.

It is also important to recognize that while some measures might initially appear to overlap with the proposed SEM:CY, there are important distinctions. The Youth Experiences Survey- Sport (YES-S) [52] was developed to capture positive youth development such as personal, social, and cognitive skills. As such, the YES-S [52] does not address most of the factors we propose in youths’ experience in sport. Referring back to Holt and colleagues’ [26] model, the YES-S captures the PYD outcomes box. This is further evidenced in prior research that has employed the YES-S as an outcome measure of positive youth development in sport [45, 53–55]. Therefore, the YES-S and our proposed SEM:CY measure are distinct in both purpose and in coverage.

Phase I. Identifying and testing content for the SEM:CY (year 1)

Phase I will focus on generating a pool of items that provide comprehensive coverage of the proposed domains of youth sport participation. We will review existing literature to identify existing measures that capture relevant components of the proposed domains included in our model. We will also hold focus groups and interviews with coaches, sport policy and community leaders, parents, and young athletes separately to inform item development where there are gaps not covered by existing measures. We anticipate a minimum of five focus groups with approximately five to seven participants per group to obtain saturation for each group. Focus groups will be analyzed using an inductive approach where any salient text coded and later grouped into higher order codes by at least two members of the research team. Interviews will be conducted until saturation is reached in each group. Data collection will take place concurrently with data analysis, so as to ensure that new themes are fully explored in subsequent interviews and analyses.

Once the initial pool of items is generated, we will engage with a panel of at least 15 experts in youth sport participation for initial content validation [58]. The panel will include experts with research expertise and those with significant experience in sport (e.g., coaches; leaders in sport organizations such as Sport4Life, Sport Matters, and Sport Research Intelligence Sportive). Expert panel members will be asked to provide feedback on the conceptual model of the SEM:CY measure, including the relevant constructs, their definition, and the initial pool of items for testing. At this stage, the panel will be asked to rate each item with regard to relevance, comprehensiveness, redundancy, and clarity [59]. The panel will be asked to score the items on a 5-point scale based on the degree to which the item fits with the content of the construct it is intended to measure. The panel will also be asked to provide written feedback about each item (e.g., suggested edits for wording and developmental appropriateness), the conceptual framework (e.g., identifiable gaps), and the item assessment procedure as a whole.

The content validity ratio (CVR) will be used to assess the panels’ content relevance ratings for each item and each set of items [56]. Items that are deemed relevant will be retained, items deemed relevant but in need of modification will be revised, and items deemed irrelevant will be removed. The exact number of items for the initial pool will depend on the results of the expert panel process, however we estimate the process would result in about 5 items for each domain of the conceptual model. Based on our initial conceptual model with 8 domains, this would result in an initial pool of about 40 items.

We will test items from the expert panel using cognitive interviews with approximately 10 youth in the target age range for the SEM:CY. Cognitive interviewing is a common technique used to evaluate item meaning, comprehension, and wording as well as response option choice [60–62]. We will assess children’s perceptions of difficulty and understanding of each item to ensure items are developmentally appropriate. Based on the results, items will be revised as necessary. At the end of phase I we will have an initial pool of items for testing.

Phase II. Testing the item pool of the SEM:CY (years 2–3)

Phase II will involve testing the factor structure of the model and conducting item level analysis for item reduction and refinement. We propose to test a convenience sample of youth ages 10 to 18 (sample size estimate below) across elementary, junior, and senior high school (grades 6 to 12). The sample will be recruited from school settings, rather than from sport programs, to capture a more heterogeneous patterns of sport participation than when sampling from a particular sport club or program. Furthermore, the measure is intended for community-based youth surveys, thus a school-based sample is appropriate. Students will only be asked to complete the SEM:CY items if they have participated in organized sport in the last year.

We propose to test and refine the pool of items with 2 consecutive samples across 2 steps: (1) conduct initial item level analysis and explore the structure of the SEM:CY and (2) confirm the factor structure of the SEM:CY and conduct any further item refinement required. These steps will ensure that we will have sufficiently refined and examined the internal validity and structure of the measure prior to final validation. Moving to external validation prior to establishing internal validity can result in problems with interpretation in the meaning of external associations [63]. After the factor structure is confirmed, the relationship between the SEM:CY and perceived self-competence will be examined. Perceived competence is one of the most common psychological factors that has been examined in relation to sport participation among youth [8, 9].

Initial item analysis and reduction

For the first sample, we anticipate requiring approximately 600 students to complete the measure. This sample will provide a ratio of 15:1 participants for each item with a 40-item scale for the exploratory factor analysis (EFA). This should provide adequate power for the EFA. While some guidelines recommend a lower 10:1 ratio, recent work suggests that these guidelines underestimate the required sample size when ideal scenarios (e.g., about 4-5 items that at least moderately load onto each factor) are not met [64, 65]. We will aim to recruit about 938 students to provide a final sample of 600. This is based on attaining a 75% response rate among students (N = 750) and a 75% participation rate in sports in the past year.

Confirming the factor structure and initial validation

Once the revised pool of items is selected we will recruit another sample to confirm the factor structure of the measure and assess the initial validity of the SEM:CY. Finally, we will examine the time required to complete the revised SEM:CY measure to assess whether it is sufficiently brief (about 5 min) for inclusion into community-based surveys of youth [8]. This step also provides an opportunity for further item refinement if needed prior to validating the final measure. We will aim to recruit about 781 students (assuming a 75% response rate and 75% complete SEM:CY items) to provide a final sample of 500 to provide a sufficient sample to test the model in subgroups in line with recommendations for confirmatory factor analysis (CFA) [68].

Phase III. Validating the final SEM:CY measure (year 3)

Phase III will focus on a more robust test of the measures construct validity (e.g., association with youth development and well-being) and reliability (e.g., internal consistency of scales and short-term stability) and confirming any changes to the structure from phase II. We propose to recruit 391 students (assuming a 75% response rate and 75% complete SEM:CY items) to assess the final SEM:CY in 250 students ages 10 to 18. We will retest a random subset of the sample (n ~ 75) on the SEM:CY to examine test-retest reliability at about 2 weeks.

To test the validity of the measure we will assess its relationship with perceived competence, sport anxiety, and PYD in sport. Along with perceived competence, positive youth development in sport, and anxiety have been commonly studied in relation to sport participation. Based on both qualitative and quantitative research we would expect a positive relationship between the SEM:CY with perceived self-concept and positive youth development, and negative correlation with sport anxiety [8, 9].