Sport-based youth development interventions in the United States: a systematic review

Abstract

Background

The growing number of sport-based youth development interventions provide a potential avenue for integrating sport meaningfully into the U.S. public health agenda. However, efficacy and quality must be reliably established prior to widespread implementation.

Methods

A comprehensive search of databases, peer-reviewed journals, published reviews, and both published and unpublished documents yielded 10,077 distinct records. Title and abstract screening, followed by full-text screening using 6 criteria, resulted in 56 distinct studies (coalescing into 10 sport-based youth development intervention types) included in the synthesis. These studies were then independently assessed and critically appraised.

Results

Limited efficacy data were identified, with the quality of methods and evidence largely classified as weak. Processes likely to contribute to the outcomes of sport-based youth development interventions were identified (e.g., predictors of ongoing engagement, alignment between target population and intervention, intervention design), although more rigorous research is needed on these and other processes. Physical health outcomes were only studied in 3 of the 10 intervention types.

Conclusions

The evidence base does not yet warrant wide-scale implementation of sport-based youth development interventions for public health goals within the U. S., although there is promising research that identifies areas for further exploration.

Background

Despite widespread recognition of physical inactivity as a global public health issue [1], there continues to be a trend toward sedentary lifestyles, particularly for youth [2]. This includes the United States (U.S.), with 37.4% of 12–19 year olds classified as overweight and 20.9% of these youth identified as obese [3]. Consequently, the promotion of physical activity is a national health priority. Despite physical activity being an inherent feature of sport, which is immensely popular among youth in the U.S. (e.g., more than 44 million participants) [4], the role of sport in promoting physical activity is frequently marginalized in the public health agenda [5, 6].

The perception of sport as a competitive activity permeated by aggressive masculinity, expressions of violence, and a high tolerance for injury, may partially explain the lack of sport in public health discourse [7]. Others have argued that practices are filled with drills and a focus on strategy over activity [5, 8]. From this perspective, the public health agenda appears incompatible with the performance-oriented focus of sport [6], often positioned as an “afterthought in U.S. public health campaigns” ([5], p. 20). Moreover, competitive sport can be exclusionary [9], particularly when social inequalities (e.g., socioeconomic position) and neighborhood characteristics (e.g., social fragmentation, crime incidence) disproportionately affect low-income youth [10, 11].

A response to these concerns is the growing number of sport-based youth development interventions founded on principles of inclusion and participation [12]. These interventions expand their focus beyond physical health, promoting holistic development in physical, cognitive, affective, social, and lifestyle domains [13]. This emphasis broadens the traditional public health narrative that takes a deficit-reduction approach, instead embracing the notion that “problem-free is not fully prepared, and that fully prepared is not fully engaged” ([14], p. 25). Thus, sport-based youth development interventions involve youth in physical activities that intentionally foster developmental assets (e.g., values, commitment to learning, social competencies, positive identity) and surround youth with protective factors (e.g., support, relationships, experiences, resources, opportunities) that have been shown to be important facilitators of health for at-risk youth [15, 16]. This represents a paradigm shift in which public health efforts are embracing strength-based approaches in addition to problem-prevention activities.

Despite the potential for sport-based youth development to contribute to U.S. public health goals, evidence is needed to support the effectiveness of these interventions prior to wide-scale implementation. As Berg and colleagues pointed out, “all public health programs must guard against the idea that merely providing opportunities will get people more physically active” ([5], p. 22). This notion is particularly true for sport-based youth development interventions, as they have been criticized on account of theoretical [17] and methodological rigor [18]. As such, the purpose of the current study was to systematically review sport-based youth development interventions in the U.S. The primary research question was: What is the quality of evidence reported for sport-based youth development interventions in the U.S.? Two secondary research questions also guided the study: In youth populations, are sport-based youth development interventions effective for improving public health-related goals (i.e., physical, social, mental), and What are the processes that contribute to the outcomes of sport-based youth development interventions?

Methods

This systematic review was conducted and reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (see Additional file 1 for PRISMA checklist) [19]. An electronic search was conducted of 6 databases (i.e., PsycINFO, Embase, SPORTDiscus, Education Source, Scopus, Web of Science; see Additional file 2 for detailed search terms and strategy), along with manual searches of 20 field relevant peer-reviewed journals and the reference lists of 12 recent reviews. Searches focused on literature published from 1995 through August 2017. Additionally, published and unpublished research documents were sought from 45 sport-based youth development experts, representing a diverse array of organizations, institutions, and groups, with the websites of these entities also searched. Overall, the search yielded 10,077 distinct records published in English (see Fig. 1), with the title and abstract of each record screened. Full text review of 430 articles (4.3%) was completed by 2 independent investigators using 6 screening criteria: (a) full text articles/evaluations with enough methodological data (i.e., a description of participants, data collection, and data analysis) to critically appraise the study; (b) data collected completely/partly in the U.S.; (c) empirical studies reporting primary data; (d) average participant age between 10 and 24 years old, per the United Nations’ definition of adolescents and youth [20]; (e) evidence of a plus-sport (i.e., sport adapted to maximize developmental objectives) or sport-plus (i.e., sport used as a vehicle for development, with precedence on non-sporting outcomes) intervention [21]; and (f) a minimum of 2 independent records for an intervention type (in response to consistent criticisms of one-off evaluations) [17, 22]. Disagreements were resolved through discussion with the primary investigator. The final sample contained 56 distinct studies, which coalesced into 10 intervention types: (a) Summer Sport and Life Skills Camps, (b) Teaching Personal and Social Responsibility, (c) Girls on the Run, (d) Playworks, (e) The First Tee, (f) Play It Smart, (g) Urban Squash, (h) Coach Across America, (i) Doc Wayne, and (j) Sport Hartford. A full reference list of all included studies is provided in Additional file 3.

Fig. 1

PRISMA flow diagram for research on sport-based youth development interventions in the U.S.

Quality assessment

Two investigators independently assessed and critically appraised the study methods. For studies containing quantitative data, the Quality Appraisal Tool for Quantitative Studies [23] was used, based on these criteria: (a) selection bias, (b) allocation bias, (c) control of confounding variables, (d) blinding, (e) data collection methods, (f) follow-up rates, (g) statistical analyses, and (h) intervention integrity (e.g., fidelity assessment, likelihood of co-intervention). Each study received a strong, moderate, or weak rating for each applicable criterion. When specific criteria were not applicable to a study (e.g., withdrawal and follow-up in a cross-sectional design; allocation bias in single group designs), the Quality Appraisal Tool for Quantitative Studies gives explicit instructions as to how to rate said criteria. An overall assessment was then derived: (a) strong if all criteria were rated moderate or strong; (b) moderate if 1 criterion was rated weak; and (c) weak if 2 or more criteria were rated weak. The investigators took a meta-theory and meta-method approach for qualitative studies [24]. A methodological coherence rating was given, which determined whether a study demonstrated consistency between ontology, epistemology, theory, methodology, sampling strategy, data collection techniques, data analysis, and study quality procedures.

Data synthesis

Due to high levels of heterogeneity (see Table 1 for study details), a narrative description of the evidence is presented below. Specifically, the extant literature was examined within each intervention type, with the study outcomes assessed in the context of the quality of evidence. Where discrepancies across studies occurred (i.e., positive vs. negative results), these results are contextualized within the quality of the evidence.

Table 1 Summary table¹

Results

Summer Sport and Life Skills Camps

Study quality

Summer Sport and Life Skills Camps (4–5 weeks in length) grounded in principles of positive youth development (e.g., caring climate, support for autonomy and competence) and sponsored by the National Youth Sport Program or universities were assessed in 11 studies (8 quantitative, 2 qualitative, 1 mixed methods). Eight studies with quantitative components were classified as weak evidence, with only McDavid, McDonough, Blankenship, and LeBreton classified as moderate evidence [25]. Most studies were single-group, pre-test post-test designs. Of the 2 qualitative studies and 1 mixed methods study, 1 was assessed as having no methodological coherence, while 2 were assessed as partial methodological coherence [26, 27].

Outcomes

Overall, the findings suggest a limited effect of Summer Sport and Life Skills Camps on youth outcomes, with major limitations being short assessment periods and/or a lack of follow-up. In 1 follow-up study, slight increases were reported in assessment scores for those who returned to camp, but the findings did not account for those who did not return [28]. Given that the outcomes assessed could be considered rather stable (e.g., perceptions of competence, hope, self-worth), it is possible that the short intervention and observation phases were not sensitive enough to show possible changes.

One strength of this body of evidence is the use of mediational and moderation analyses to identify key critical factors. For example, establishing a caring climate increased empathic concern [29], and affective self-regulatory efficacy and empathic self-efficacy helped mediate the link between caring climate and social behaviors [30]. Moreover, leader behavior and perceived leader support were important predictors of child-level outcomes and camp participation in subsequent years [28, 31]. These findings imply that ongoing intervention involvement may enhance outcomes. Ulrich-French and McDonough reported that lower BMI, higher self-worth, and higher intervention attendance were also predictors of ongoing engagement [28]. Finally, findings suggested that the camp interventions were most effective for older children [32] and those who entered with higher levels of baseline risk [33]. Qualitative findings suggested that summer camp interventions were safe places where relationships were developed [27]. No physical health data were assessed.

Teaching Personal and Social Responsibility

Study quality

Teaching Personal and Social Responsibility is a professional practice model that has been used for decades by practitioners and scholars, with intentional programming supporting the development of personal and social responsibility in youth. All 17 studies (2 quantitative, 13 qualitative, 2 mixed methods) were designed and conducted by university faculty and/or students. Three of the 4 studies with a quantitative component were judged as having a weak level of evidence, with 1 study [34] rated as strong evidence. Of the 15 studies with a qualitative component, 6 studies were assessed as having no methodological coherence, 8 studies partial methodological coherence, and 1 study full methodological coherence [35]. Common weaknesses of qualitative studies were a lack of philosophical or theoretical basis to the study, an over-reliance on deductive/confirmatory analyses, and an over-reliance on participant perspectives, without triangulation with other agents in the intervention. Strengths of qualitative studies included multiple sources of data.

Outcomes

Overall, lower-rated qualitative studies reported that participants experienced some development related to the Teaching Personal and Social Responsibility levels (e.g., respect, effort, leadership), and the higher-rated qualitative study [35] identified life skills learned through the intervention including, but not limited to, these levels (e.g., self-belief, courage, responsibility, self-control, self-direction), although the findings were based on participants’ self-reporting. Critical intervention elements that may have enhanced youth outcomes included quality adult-youth relationships, youth leadership, and a youth-centered task-oriented climate [35,36,37,38,39,40,41,42]. Additionally, sustained intervention engagement was perceived to enhance youth development outcomes [36, 38].

While studies often discussed participants transferring skills learned in the intervention to other life domains, actual evidence of transfer was scarce and inconsistent. Hayden et al. [36] and Cryan and Martinek [43] reported a transfer effect into classroom settings, while Jacobs [44] and Wright et al. [45] reported no group differences for intervention and control participants. Despite the fact that the ultimate goal of the Teaching Personal and Social Responsibility model is to have participants transfer life skills outside of the intervention, the current evidence base does not support this outcome. Findings did indicate the need to design a curriculum with opportunities to practice life skills within and outside of the intervention context [41], which may support life skill transfer.

The 1 study with strong evidence [34] examined an intervention within the context of self-determination theory and goal perspective theory, while targeting moral reasoning. Results showed positive effects on distributive justice reasoning and perceived competence for those in the intervention. Thus, it appears that the Teaching Personal and Social Responsibility model can yield some desired outcomes; however, a more substantive and theoretically based intervention may need to be used in conjunction with this professional practice model to elicit positive changes in youth development outcomes.

Girls on the Run

Study quality

Girls on the Run is a non-profit organization that engages girls in community running interventions designed to build physical, psychological, and social assets. Five published studies and 3 evaluation reports (6 quantitative, 2 mixed methods) were assessed. All 8 studies were rated as weak evidence, with 6 single-group studies and 1 cross-sectional study examining exposure and no exposure to the intervention along with 1 study with a 3-group, quasi-experimental design. For the 2 mixed-methods studies, the qualitative components were assessed as having no methodological coherence.

Outcomes

While pre-post data provided consistent evidence of small, but positive effects for self-esteem and body image satisfaction, these findings were not upheld by multiple-group designs. Pettee Gabriel, DeBate, High, and Racine [46] found a group x time interaction for self-reported levels of physical activity, with those not exposed and those newly exposed reporting increases in physical activity, while those previously exposed not reporting physical activity increases. More rigorous research should be conducted to examine how changes in physical activity might mediate mental health and wellness-related outcomes. Overall, findings suggested that initial exposure to Girls on the Run likely has the greatest effect and that there may be a ceiling effect for intervention benefit [47]. Social support and the development of self-worth also appear to be important factors in intervention success [48].

Playworks

Study quality

Playworks is a non-profit organization which places trained coaches in schools serving low-income communities, with a focus on providing opportunities for developmentally appropriate physical activity. Two quantitative studies (published across 4 reports [49,50,51,52]), 2 qualitative studies [53, 54], and 1 mixed methods study [55] assessed the Playworks intervention. Two quantitative studies were rated as strong level of evidence, and the mixed methods study by Massey et al. [55] was rated as moderate level of evidence. The 2 qualitative studies [53, 54] and the mixed methods study [55] showed partial coherence, as none discussed their underpinning ontology/epistemology or provided a clear rationale for sampling.

Outcomes

Madsen et al. [52] reported that with each additional year of exposure to Playworks, youth reported significantly higher scores in physical activity, school participation, problem solving skills, and goal aspirations. While the effects reported were small on a yearly basis, they were clinically meaningful when considered cumulatively (e.g., approximately 1 SD increases in physical activity, meaningful participation, and problem solving across the analysis period). Similarly, other researchers reported higher levels of objectively measured physical activity; higher levels of teacher-reported safety and inclusion both at recess and in school; lower levels of transition difficulties and bullying behaviors; and higher levels of student-reported positive behavior and attention in the classroom [49,50,51]. However, these same researchers reported no differences between Playworks and control schools on several variables (e.g., student perceptions of safety at recess and aggressive behavior at school, teacher-reported classroom behavior, positive youth development, academic performance). Finally, Massey and colleagues reported improved classroom behavior over time for those in the Playworks peer leadership training intervention, and both within and between-group improvements in playground conflict and positive playground interactions [55]. Qualitative data from London et al. [53] and Massey et al. [54, 55] documented increased recess quality over time, as well as students’ perceptions of an improved recess experience and their own roles as leaders on the playground. Given the quality of the reported evidence, in conjunction with the findings, data suggest that the Playworks intervention facilitates several positive outcomes aligned with public health goals.

The First Tee

Study quality

The First Tee is a non-profit organization that provides youth with facilities and education interventions designed to foster the development of character and values through golf. Three studies conducted on The First Tee were assessed. One study used a qualitative methodology and was assessed to have partial methodological coherence, while 2 studies used quantitative designs, with 1 rated as weak evidence and the other as moderate evidence.

Outcomes

The qualitative study [56] identified various life skills that youth perceived to have developed (e.g., ability to meet/greet others, show respect, control emotions), with findings corroborated by a quantitative longitudinal study [57]. Notably, findings implied significant group differences (when controlling for parent education and socioeconomic status) on 5 of 8 life skill transfer domains and 6 of 8 development outcome domains for those with and without exposure. Longitudinal findings also implied that 3 life skills (i.e., meeting/greeting, appreciating diversity, getting help) increased over time and with increased intervention exposure. Brunelle, Danish, and Forneris’ study was more limited in scope, as it only tested outcomes after a 1-week intensive intervention [58]. Results showed short-term improvements in social responsibility and goal knowledge, although no findings were reported on physical health outcomes.

The First Tee findings suggest several potential moderating variables. First, those who entered the intervention with the lowest life skills scores improved the most [57], suggesting this intervention is effective when targeting the appropriate youth. Additionally, those engaging in community service had more favorable outcomes [58], suggesting a need to engage youth outside of the intervention. Finally, girls reported greater increases in perspective taking, while those who identified as White reported greater increases in social interests [58], suggesting a need to better tailor the intervention and outcomes to the unique needs of males, as well as youth from racially and ethnically diverse backgrounds.

Overall, while limited, the findings suggested a positive effect on life skills development for youth in The First Tee, with those findings strengthened by the inclusion of multiple-group data, longitudinal data, qualitative data, and first-person reports, as well as the identification of various critical features (e.g., baseline risk, gender, race, engagement outside of the intervention). No physical health data were assessed.

Play It Smart

Study quality and outcomes

Play It Smart is a school-based intervention that promotes social and personal development for student-athletes, with approximately 50 schools still implementing the intervention in various forms. Two studies were assessed, with the intervention founder an author on both articles. The quantitative study was rated as weak evidence, while the qualitative study was assessed as having no methodological coherence despite using data from exit interview forms from 1361 youth. Overall, findings partially support positive youth development and academic outcomes for participants in Play It Smart, although an overall lack of methodological processes and procedures limits the interpretation of the findings. Physical health findings were not reported.

Urban Squash

Study quality and outcomes

Two Urban Squash interventions facilitated by non-profit organizations were evaluated. Interventions focused on academic enrichment, sports, mentoring, and community service through after-school and summer programming. Three studies (2 quantitative, 1 qualitative) were assessed. Doctoral dissertations represented the quantitative studies, which were rated as moderate evidence and limited by possible selection bias and high attrition rates. The qualitative study was assessed as having no methodological coherence. Overall, there is inconclusive evidence to suggest that these Urban Squash interventions have a positive impact on academic skills, and physical health findings were not reported.

Coach Across America

Study quality and outcomes

Coach Across America is the flagship intervention of Up2Us Sports, where coaches are trained and supported in their efforts to provide quality sport-based youth development for participants in community-based organizations. Coach Across America was assessed with 2 evaluation reports (1 quantitative, 1 mixed methods). The mixed methods study was rated as moderate level quantitative evidence, with the qualitative component assessed as having no methodological coherence. The quantitative study was rated as weak due to missing data and unclear data collection procedures (e.g., reliability and validity of instruments, blinding). Despite findings suggesting improvements in self-reported physical activity and physical fitness tests and the development of high-impact attributes, the findings should be examined cautiously, given methodological concerns. Better results on youth development outcomes were reported for younger participants and those with higher levels of organizational contact.

Doc Wayne

Study quality and outcomes

Doc Wayne is a non-profit organization that provides sport-based group therapy interventions for at-risk youth, along with facilitating the use of its sport-based therapeutic curriculum in varied settings. Two quantitative studies (1 published, 1 evaluation report) were rated as weak evidence, although results demonstrated positive trends for those participating in a trauma-informed sports league on participant behavior (e.g., internalizing and externalizing behavior) and levels of emotional regulation. No findings were reported on physical health outcomes.

Sport Hartford

Study quality and outcomes

Through Sport Hartford (designed for preadolescent females) and Sport Hartford Boys (designed for preadolescent males), sport and physical activity are used to help participants develop their intellectual, social, and physical competencies. Both interventions are designed and conducted by university faculty and/or students. Three qualitative studies were published. One study was assessed as having no methodological coherence, while the other 2 were assessed as having partial methodological coherence. All studies were limited by low participant numbers. Findings suggest that Sport Hartford shows promise but currently lacks efficacy data, along with findings on physical health outcomes. Broader principles ascertained from these studies suggest that interventions need to maintain a sense of fun, provide unique experiences, help youth develop broader life and community outcomes, and create a positive structure that youth can count on for consistency and stability.

Discussion

This systematic review of research on sport-based youth development interventions conducted within the U.S. over the past 23 years provides a portrait of the current state of evidence. Specifically, there is limited efficacy data, with the quality of methods and evidence largely classified as weak and incoherent. A number of recommendations are offered from this review. These include the need to: (a) assess intervention quality and fidelity; (b) use multiple groups, incorporate multi-site comparison, and use longitudinal designs; (c) account for confounding variables (e.g., maturation bias, selection bias); and (d) integrate studies across philosophical, theoretical, methodological, and analytical perspectives. Additionally, the norms for reporting research on sport-based youth development interventions must be critically examined and improved. Despite more than 700 organizations operating in the U.S. [59], only 56 studies assessing 10 intervention types met the inclusion criteria for this review, with many of the articles limited in their transparency (e.g., unclear sampling procedures, infrequent null/negative outcomes). Likely reasons include funding concerns (e.g., fear of reporting null/negative outcomes), time limitations, word count limits, and publication bias. This is alarming if we consider that the evidence base is grounded in what is publicly shared. Not only does this limit intervention and research efficacy, but it also means that decisions related to funding and policy may be grounded in research that does not meet high quality standards. Thus, the quality of methods and evidence–and the reporting of these–needs to improve so that intervention efficacy can be assessed systematically to fully appreciate the potential for sport-based youth development to contribute to public health goals, and to more precisely understand its limitations. There are certainly challenges inherent to this, as research in this field is fraught with hurdles and limitations (e.g., respondent burden, degradation of data quality), but this does not change the need for higher quality of methods and evidence which are widely and transparently reported.

Limited efficacy data constrains the conclusions that can be made from this systematic review, with very few higher-quality studies scattered across the 10 reported intervention types. Nevertheless, processes likely to contribute to intervention outcomes were identified, although more rigorous designs are needed to fully examine these. For instance, there is a need to design curricula around predictors of ongoing engagement (e.g., leader behavior, higher self-worth, safe climate), with extended engagement in interventions likely to lead to improved participant outcomes. Additionally, the most appropriate target population for each intervention must be carefully considered, from participant age, gender, and race/ethnicity to life skill scores and baseline risk levels. As for intervention design, the following features were identified as variables influencing youth experiences and outcomes: (a) climate (e.g., safety, sense of caring/support, trust, stability); (b) leadership (e.g., support, adult-youth relationships, training/education); (c) youth engagement (e.g., youth leadership, ability to practice life skills); and (d) activities (e.g., fun, novelty), including engagement outside the intervention itself (e.g., community service, ability to practice life skills, connections to broader life/community outcomes). These and other processes must be rigorously studied in greater depth to understand whether, how, and why sport-based youth development interventions may facilitate certain outcomes and impacts. Additionally, if sport-based youth development interventions are going to contribute to the public health agenda, there is a need for concurrent assessment of the physical, cognitive, affective, social, and lifestyle domains identified by Bailey [13], with a particular focus on physical health. In the 10 intervention types featured in this systematic review, only 3 (i.e., Girls on the Run, Playworks, Coach Across America) directly measured physical activity or physical fitness levels, despite widespread concerns about physical inactivity and overweight/obese classifications in the U.S. [2, 3]. This suggests that sport-based youth development interventions may be prioritizing the cognitive, affective, social, and lifestyle domains, thereby eschewing the physical domain that is inherent to sport-based interventions; this must be addressed in future studies. This knowledge can inform future theory, research, praxis, policy, and funding, along with the broader dialogue surrounding sport-based youth development within the public health agenda.

Limitations

The limitations begin with the scant methodological descriptions provided in many of the included studies, but extend to publication bias, self-censorship, and the geopolitics of knowledge production in these studies. As is the case with many systematic reviews, the research team could not pool the data for meta-analysis due to heterogeneity. The decision to require a minimum of 2 independent records for an intervention type excluded 25 studies, although only 1 study published in 2002 [60] was rated above weak evidence and no studies were assessed as having full methodological coherence, suggesting this criterion did not overlook higher-quality published research on smaller, newer interventions.

Conclusions

Returning to the primary research question guiding this systematic review, the quality of evidence reported for sport-based youth development interventions in the U.S. was largely classified as weak and incoherent. This limited efficacy data therefore constrained the conclusions that could be made about the secondary research questions, which related to intervention efficacy in improving public health-related goals and identification of processes which contribute to intervention outcomes.

Thus, the quality of the reviewed evidence base does not yet warrant wide scale implementation of sport-based youth development interventions for public health goals within the U.S., although there is promising research that identifies areas for exploration. Further investigation is needed to elucidate the processes that contribute to the outcomes of sport-based youth development interventions. This includes ideal adult-youth ratios in interventions, effective leader training and education, and optimal intervention size and dosage, along with the recommendations offered above. Additionally, more questions need to be asked about the efficacy of sport-based youth development interventions contributing to public health-related goals, particularly within larger systems (e.g., schools, communities). For example, given that social inequalities and neighborhood characteristics disproportionately affect the health of low-income youth, how should interventions be adapted to best serve this population? Further, what is the social return on investment for sport-based youth development interventions related to public health goals? These questions, and others, must be rigorously and comprehensively studied to appreciate the most effective, efficient approach for sport-based youth development within the public health agenda.