- Study protocol
- Open Access
- Open Peer Review
Rationale and study protocol to evaluate the SunSmart policy intervention: a cluster randomised controlled trial of a primary school-based health promotion program
BMC Public Healthvolume 15, Article number: 42 (2015)
Previous evaluations of the SunSmart Program have supported the link between a written sun protection policy and improved sun protection behaviours in New South Wales (NSW) primary schools. However these evaluations have relied on self-reported data and research suggests that direct observations are required to better represent schools’ usual sun protective practices.
Data will be collected in the summer months of 2014, 2015, and 2016 as part of an 18-month cluster-controlled trial in NSW primary schools (n = 20). Researchers will conduct three direct observations to record students’ hat use and teachers’ use of sun protective measures during recess and lunch periods in each school. Researchers will also record the volume of sunscreen that the Year 6 classes in each school utilise over the term. At the conclusion of baseline data collection, five schools will be randomised into an intervention group that will work with researchers to develop a policy-driven intervention to improve sun safety behaviour in NSW primary schools.
An initial review of relevant Australian and New Zealand literature suggests that provision of policy support is likely to improve school sun protection practices; however there is no suggested model for this support. This will be the first objective analysis of sun safe behaviours leading to a policy-driven intervention conducted in Australian primary schools since the 1990s, and will inform the future direction of sun safety in our schools.
Australian and New Zealand Clinical Trials Register ACTRN12614000926639 Registered 28th August 2014.
Australia has among the highest incidence of skin cancer in the world with approximately 750,000 treatments for melanoma and skin cancer every year . Protecting skin from overexposure to ultraviolet (UV) radiation is the simplest and most effective way to reduce the risk of developing melanoma and other skin cancers [2,3]. The World Health Organization (WHO)  suggests that school sun protection programs are the key to skin cancer prevention. The New South Wales Department of Education and Communities (NSWDEC) has demonstrated its commitment to sun safety, releasing updated Sun Safety for Students guidelines , which strongly encourage each school community to implement a comprehensive Sun Safety Action Plan.
Cancer Council New South Wales (CCNSW) is a not for profit research and advocacy organisation for all forms of cancer within New South Wales (NSW). The SunSmart Program is their flagship skin cancer prevention program, and supports primary schools to develop and implement comprehensive sun protection policies. The program is based on the World Health Organization’s Sun Protection and Health Promoting Schools principles  It applies these principles by; a) negotiating a school-endorsed sun protection policy with schools and the systems that govern them, b) providing sun protection education resources, c) advocating for a healthy and sun-safe school environment that include the scheduling of outdoor activities, providing shaded play and recreation areas, encouraging sun protective clothing, providing and encouraging the wearing of sunscreen and encouraging the role modelling of sun-safe behaviours by teachers and other significant adults at schools, and d) encouraging community and family involvement in sun protection behaviour and awareness.
Previous national and state surveys of school sun protection policies and practices demonstrate that being a member of the SunSmart Program improves sun protection practices in primary schools [7,8]. The national survey of Sun Protection Policy and Practice in Primary Schools demonstrated an association between a written sun protection policy and more effective sun protection practices . However survey results also demonstrate there are opportunities to embed sun protection practices more broadly in primary schools; in particular students’ use of sun-safe hats and sunscreen, and teacher role modelling of positive sun protection practices .
A review of relevant Australian and New Zealand literature suggests that provision of policy support is likely to improve school sun protection practices [9,10]. Change agents (project champions like teachers) are suggested as one potential approach for improved policy implementation [11,12]. Furthermore, any policy intervention must consider the nuance and idiosyncrasies of the school and the communities they serve. Given the complexity of the school setting, intervention strategies will need to be innovative and consider issues of design and measurement in dealing with school-based samples.
In 2014 more than three-quarters of NSW primary schools are signatories to the SunSmart Program . Previous evaluations of school sun protection policies have stated the data supporting the link between sun-protection policies and observations of sun protective behaviour at primary schools are lacking . This evaluation by Turner and colleagues also called for research involving independent assessment of policies and direct unannounced observations of behaviour to better represent usual sun protective practices (rather than self-reported data). This study seeks to build on the body of knowledge regarding sun protection behaviours in primary schools. The study will use a novel application of an existing objective observational tool to collect data on key sun-safe practices in schools that are members of the SunSmart Program, and develop and evaluate practical strategies to support schools in improving their sun protection policy.
The SunSmart Evaluation and Policy Intervention study is an 18-month primary school-based intervention and will be evaluated using a cluster randomised controlled trial. Ethics approval has been sought and obtained from an Australian University Human Ethics Committee (HREC 2014/062) and the New South Wales Department of Education and Communities (SERAP: 2014148). The SunSmart Evaluation and Policy Intervention study is registered with the Australian and New Zealand Clinical Trials Registry (ACTRN12614000926639). The study protocol was also reviewed internally by the research committees of the funding agencies.
Following the initial recruitment processes, researchers will conduct baseline assessments at participating schools. The design, conduct and reporting of this study will adhere to the Consolidation Standards of Reporting Trials (CONSORT) guidelines for a cluster randomised controlled trial . Principals, teachers and parents will provide written informed consent.
Sample size calculation
Power calculations were conducted to determine the sample size and number of observations required to detect changes in the outcome of students wearing sun protective headwear in a cluster design. Calculations assumed baseline-posttest expected effect size gains of d = 0.4/r = 0.19 and were based on 80% power, with alpha levels set at p < 0.05. Using the standard deviation (SD) of change of SD = .5, it was calculated that the study required 50 observations within the intervention and control groups would provide adequate power to detect a between group difference of d = 0.4 across the school day.
Recruitment and study participants
To be eligible to participate in the study, schools must be government primary schools and be a current signatory to the SunSmart Program in the Greater Western Sydney Region, NSW, Australia (Approx 33.75 deg S, 150.70 deg E).
All eligible schools (n = 167) will be sent an initial email with an invitation to participate in the study. CCNSW and NSWDEC will also identify a short-list of schools that may be receptive to participating in the study (n = 40). Schools that respond to the initial email and the short-listed schools will be pooled and receive a follow up call in random order from the project researchers to ascertain whether they would like to participate in the study. The first twenty schools that demonstrate interest will then be recruited into the study.
Randomisation into intervention and control group will occur after baseline assessments. A simple computer algorithm will be used to randomly allocate schools to either control (n = 15) or the treatment (n = 5) conditions by an independent researcher not involved in the study. This method will ensure all schools have the same likelihood of allocation into one of the two study arms. Trained research assistants and project researchers will conduct all assessments and perform focus groups and interviews. All researchers will complete training sessions prior to assessment to maintain consistency and where possible, the same assessors will be used at baseline, post-test and follow-up. Figure 1 shows the flow of participants through the study.
After randomisation, researchers will conduct focus group discussions with parents, Stage 3 (year 5 and 6) students, and individual interviews with teachers. This qualitative research will aim to investigate the results of the baseline data collection and inform the intervention design with these schools. These focus groups and interviews will unpack the positive and negative aspects of the school’s utilisation of sun-safe hats, sunscreen and staff role modelling of sun-safe behaviours.
Interviews and focus groups will also investigate schools’ understanding of being ‘SunSmart’ and other aspects of the existing school sun protection policy; including sun-safe school uniforms, the provision of shade, scheduling of outdoor activities, sun protection within the school curriculum and finally, inclusion of the local community into school sun-safe practices. This broader context will allow analysis of the relative importance of the three specific study outcomes and other factors that may have influenced the baseline results.
Data from this formative research will be analysed using coding and intra and inter-textual analysis to ascertain common themes across schools . Formative interview data will be utilised to refine the design and enactment of the intervention.
The intervention development and implementation will be based on Social Cognitive Theory (SCT)  which focuses on the interplay between personal, behavioural, and environmental factors. The personal factors identified in SCT (such as the thoughts, emotions and biological dispositions of an individual) will be addressed by the intervention program by influencing the value systems that may reinforce the low emphasis on the school sun protection policy. The nuanced behavioural factors associated with sun protection behaviours, such as the wearing of hats and sunscreen and playing in shaded areas also need to be identified objectively and addressed within the intervention program. Finally, the environmental factors (both social and physical) that often exist beyond the immediate control of individual schools need to be identified and addressed in order to remove potential barriers and to provide opportunities for social support. An example of influencing this factor may include the utilising and modifying of existing school infrastructures that were not used by the school in order to make them more accommodating for the sun-safe practices without any additional financial outlay.
The intervention will also be based on the key principles of the WHO Health Promoting Schools (HPS) Framework  that focuses on the connection between formal curriculum, school ethos and the school-home-community partnership. By addressing these overlapping and interconnecting components, schools are more likely to provide comprehensive and effective health promotion throughout the entire school community. The formal curriculum of the HPS framework refers to the teaching and learning programs provided by the school, and advocates these be distributed in a variety of subjects and classes rather than just traditional health education classes . A school ethos is derived by the components put in place which construct its values and atmosphere, including organisation, environment, policies and procedures . The partnership between the school and local community is a vital component for school health promotion. By incorporating families and the community into the life of the school, a supportive learning and healthy environment can be established .
In collaboration with CCNSW and each of the primary schools assigned to the intervention group, a policy driven intervention will be rolled out over the course of two school terms (June-December 2015). Post-test data will be collected during the latter half (October-December 2015) of the intervention implementation with follow-up data collected after the schools return from summer break between January and April 2016.
Evaluation of the SunSmart Policy Intervention will involve a variety of instruments to report on sun protection behaviours that occur in primary schools. Trained research assistants who will be blind to the control or intervention allocation of the schools will conduct all assessments. All sun protection behaviour practices will be measured at baseline, post-test (12 months), and follow-up (15 months).
Wearing of headwear and prevailing environmental conditions
The primary outcome will be children’s wearing of a sun protective headwear during break periods of the school day (i.e. recess and lunch). Currently no direct observation instruments to record sun-safe behaviours of children in school settings exist. However several direct observation tools of physical activity do exist, and one of these will be adapted to capture sun-safe behaviour data in children. The System for Observing Play and Recreation in Communities (SOPARC) is based on momentary time sampling techniques in which systematic and periodic scans of individuals and contextual factors within predetermined target areas are made . iPad tablets (Apple Inc, USA) installed with the iSOPARC Application Version 1.75 (CIAFEL, Portugal: http://ciafel.fade.up.pt) will be used to provide an objective measure of the wearing of hats by children during recess and lunch.
During a scan each subject is electronically coded and identified by; sex (male or female), intensity of activity (Sedentary, Walking, or Very Active), and whether they are a Child, Teen, Adult or Senior. For this study, given all the subjects will be children, the third battery of coding (Child, Teen, Adult or Senior) will be changed to detect whether the student is Unprotected (no hat), Partially-Protected (wearing a baseball cap), Fully-Protected (wearing a 360 degree brim; broad-brimmed or bucket hat) or Fully-Protected (wearing a peak cap with back flap; legionnaire hat).
Separate scans are made for females and males, and simultaneous entries are also made for time of day, temperature, UV radiation level, area accessibility, area usability and presence of supervision. Each observation is conducted twice during the recess and lunch breaks for both females and males. This results in each school having 24 observations conducted at each data collection period (i.e. baseline, post-test and follow-up).
Direct observations will be made in designated Target Areas that represent all standard locations likely to provide opportunities for students to have sun exposure during their recess and lunch periods (e.g. play equipment or outdoor sporting fields/courts). These Target Areas (one shaded and one non-shaded) will be predetermined and identified for observations prior to baseline assessments. A map will be provided to identify Target Areas and a standard observation order established for each school. Additional Target Areas may be added by observers on site and then documented.
During occasions of high student density, Target Areas will be subdivided into smaller Scan Spaces so that accurate measures can be obtained. Observers will use standard court or field markings to determine appropriate Scan Spaces within each Target Area. Data from these smaller spaces will be summed to provide an overall measure for each Target Area. A decision to subdivide a Target Area depends upon the (1) number of students in the area and (2) the type of student activity.
Additional data recorded prior to the direct observation scans includes;
1. Temperature and UV level at the start and end of the observation period; 2. Whether the observation was made at recess or lunch; 3. Start and finish times of recess and lunch; and 4. The condition of the target area. This is coded as follows:
A = Area is accessible (e.g. not locked or rented to others).
U = Area is usable for activity (e.g. is not excessively wet or windy).
SS = Area is supervised by designated school personnel who are role modelling sun safe behaviour (e.g. wearing a broad brimmed hat, wearing a shirt with a collar and minimum short sleeves, sunglasses, using shade).
SN = Area is supervised by designated school personnel who are not role modelling sun safe behaviour.
O = Organised physical activity (i.e. scheduled, with leadership by school personnel apparent) is occurring in the area (e.g. intramurals, interscholastic practices, fitness stations).
E = Equipment provided by the school or other agency is present (e.g. balls, jump ropes). This will not be coded as present if the only equipment is permanent (e.g. basketball hoops).
C = Covered area (e.g. the activity is taking place in an area where at least 50% of the target area is covered with shade provided by a permanent or natural feature such as shade cloth or tree foliage).
Researchers will record whether the teacher supervising activities in the Target Area is role-modelling sun protective behaviour. An observation note will be added to the final iSOPARC data on whether the teacher is wearing a) a hat (broad brimmed or baseball cap), b) sunglasses or other appropriate eye protection (i.e. transition lensed optical glasses, and c) a sleeved shirt and collar.
Wearing of sunscreen
The secondary outcome will be children’s wearing of sunscreen during the school day. Currently no direct observation instruments to record the wearing of sunscreen exist. As a proxy measure of children wearing sunscreen, Grade 6 classes in control and intervention schools will be provided with one-litre containers of sunscreen for each class free of charge. Sunscreen will delivered to, and be collected from schools on the same day in order to ensure the period of use is the same in all schools; 50 days (10 school weeks) per interval. Each of the containers will their have branding removed but maintain their active ingredient information as required by the Therapeutic Goods Administration (TGA) . A laminated instruction card will be attached to each bottle requesting classroom teachers to use the sunscreen as they normally would with their students.
The consumption of sunscreen from these containers will be recorded at the conclusion of the baseline, post-test and follow-up phases and will be replaced with full containers when they reach approximately 100 mL of sunscreen or less remaining.
In order to determine sunscreen consumption, 20 similar pump packs of sunscreen will be weighed to record an average starting weight for comparison. Consumption will then be analysed at the end of each data collection phase by comparing the net weight loss each pump pack of sunscreen issued to each school and the student number enrolled in Grade 6 classes who were issued the sunscreen.
Statistical analysis of the primary and secondary outcomes will be conducted with linear mixed models using SPSS statistics version 20 (IBM SPSS Statistics, 2012) and alpha levels will be set at p > 0.05.
The models will be used to assess the impact of treatment (SunSmart Policy Intervention or Control), time (baseline, post-test and follow-up) and the group-by-time interaction, these three terms forming the base model. The models will be specified to adjust for the clustered nature of the data and will include all randomised participants in the analysis. Mixed models are robust to the biases of missing data and provide appropriate balance of Type 1 and Type 2 errors . Mixed model analyses are consistent with the intention-to-treat principle, assuming the data are missing at random . Sex, temperature/UV, time of day, type of period (recess or lunch) and conditions of target area (as previously outlined) will be included as covariates in the models.
The focus groups and interviews of Stage 3 students, parents and teachers will be used for the intervention design and will be digitally recorded with the participants’ consent and transcribed verbatim into a Microsoft Word document. Two computer programs (NVIVO 10 and Leximancer) will be used to assist with the organisational aspects of data analysis. Analysis will be conducted by members of the research team using a standard general inductive approach to qualitative analysis with NVIVO 10 and a specialised method of transforming lexical co-occurrence information from natural language into semantic patterns for qualitative analysis with Leximancer. The algorithms used by Leximancer are statistical, but they employ nonlinear dynamics and machine learning methods and have been validated, deemed stable, and reproducible .
The primary aim of this study is to evaluate the impact of a SunSmart policy intervention on children’s daily school sun protection behaviours. The study will use a novel evaluation strategy that directly observes the behaviour of children and teachers in their natural school environments. Researchers will also monitor sunscreen consumption over a ten-week time period in Grade 6 students during three summer intervals.
Previous school-based health promotion intervention studies have highlighted the importance of teacher and parental behaviour on health related intervention outcomes in primary school children . A critical aspect of this study is that teachers, parents and the students themselves will be involved in the intervention design phase. Teacher, student and community ownership of intervention design has been identified in previous studies as a desirable factor to improve the effects of school-based interventions . This ownership of the program has the potential to lead to greater sustainability of increased sun protection practices and enable teachers to integrate effective sun protection policies into the school day.
A clear strength of this study is the rigorous evaluation process including quantitative and qualitative measures to explore program feasibility and potential efficacy. The empirically robust design with baseline, post-test and follow-up measures will help examine the views of participants (teachers, parents and students), and help distinguish between an intervention that is poorly designed and one that may be poorly delivered . We consider this a necessity in this study due to the multi-site delivery.
To our knowledge, no previous interventions have reported the effects of a policy-based intervention on sun protection behaviour outcomes using direct observation measures. Significantly, our study will be capable of collecting a host of covariates on the sun safety behaviour of primary school children namely daily temperature, period of UV exposure during recess and lunch periods, size of play area, nature of play children undertake and teacher role-modelling behaviour.
An additional study strength is the use of an objective measure of behaviour. Direct observation has a number of advantages over other possible measurement tools . First, it is an objective method that provides contextually rich data to identify other factors related to sun protection behaviours (e.g., physical and social factors). Second, it can provide information on the type of hats being worn by children. The main disadvantage of direct observation is the time-intensive nature of observer training and data coding. The use of a direct observation instrument like iSOPARC is advantageous when working with children because unlike self-report measures of behaviour, they help eliminate language and literacy difficulties, recall bias and social desirability bias . As research assistants will be blind to the randomisation of schools there is little to no chance of any bias impacting on the observations.
The findings of study will provide valuable information for other research groups looking for evidence based research on sun protection behaviours of primary school students in NSW. School-based sun behaviour interventions are infrequent, and seldom published in peer-reviewed journals. This study has the potential to change school policy and practice in relation to sun safety integration and enhance a range of key health promoting behaviours in schools.
New South Wales
World Health Organization
New South Wales Department of Education and Communities
Cancer Council NSW
State Education Research Approvals Process
Australian and New Zealand Clinical Trials Registry Number
Consolidation Standards of Reporting Trials
Social Cognitive Theory
Health Promoting Schools
System for Observing Play and Recreation in Communities
Therapeutic Goods Administration
Statistical Package for the Social Sciences
Fransen M, Karahalios E, Sharma M, English DR, Giles GG, Sinclair R. Non-melanoma skin cancer in Australia. Med J Austr. 2012;197(10):565–8.
Armstrong BK. How sun exposure causes skin cancer: An epidemiological perspective. In: Hill D, Elwood M, English D, editors. Prevention of skin cancer. The Netherlands: Kluwer Academic Publishers; 2004. p. pp. 89–116.
Armstrong BK, Kricker A. How much melanoma is caused by sun exposure? Melanoma Res. 1993;3(6):395–401.
World Health Organization. INTERSUN: the global UV project: a guide and compendium. Geneva: WHO; 2003.
New South Wales Department of Education and Communities. Sun safety. 2013: Retrieved 11th April, 2014 from http://www.schools.nsw.edu.au/studentsupport/studenthealth/sun-safety/index.php.
World Health Organization. Sun protection and schools (Vol. 55). Geneva: WHO; 1993.
Sharplin G, Roth F. Sun protection policy and practice in NSW primary schools: evaluation of national SunSmart schools program in NSW from 2005 to 2011. Cancer Council SA: Adelaide; 2012.
Sharplin G, Smith A, Roth F. The national survey of Australian primary schools’ sun protection policy and practices: Evaluating the national SunSmart schools program. Cancer Council SA: Adelaide; 2013.
Giles P, Roy CR, Javorniczky J, Henderson S, Lemus-Deschamps L, Driscoll C. Global solar UV Index: Australian measurements, forecasts and comparison with the UK. Photochem Photobio. 2004;79(1):32–9.
Reeder AI, Jopson JA, Gray A. Primary school sun protection policies and practices 4 years after baseline—a follow-up study. Hlth Edu Res. 2012;27(5):844–56.
Lynagh M, Knight J, Schofield MJ, Paras L. Lessons learned from the Hunter region health promoting schools project in New South Wales, Australia. J Sch Hlth. 1999;69(6):227–32.
Townsend JS, Pinkerton B, McKenna SA, Higgins SM, Tai E, Steele B, et al. Targeting children through school-based education and policy strategies: comprehensive cancer control activities in melanoma prevention. J Am Ac Derm. 2011;65(5 Suppl 1):104–13.
Cancer Council New South Wales. SunSmart program for primary schools. Retrieved 11th April, 2014, from http://www.sunsmartnsw.com.au/Default.aspx.
Turner D, Harrison SL, Buettner P, Nowak M. School sun-protection policies – does being SunSmart make a difference? Hlth Edu Res. 2014;29(3):367–77.
Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, et al. CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:698–702.
Maykut P, Morehouse R. Beginning qualitative research: a philosophic and practical guide. London: Falmer Press; 1994.
Bandura A. Social foundations of thought and action: a social cognitive theory. Englewood Cliffs: Prentice-Hall; 1986.
World Health Organization. WHO Expert Committee on Comprehensive School Health Education and Promotion. Geneva: WHO; 1995.
New South Wales Department of School Education. Towards a Health Promoting School. Sydney: NSW Government; 1996.
McKenzie TL, Cohen DA. SOPARC (System for Observing Play and Recreation in Communities): Description and Procedures Manual. 2006: Retrieved 11th April, 2014 from http://activelivingresearch.org/sites/default/files/SOPARC_Protocols.pdf.
Australian Government Therapeutic Goods Administration. Australian regulatory guidelines for sunscreens (ARGS). Office of Medicines Authorisation; 2012.
Mallinkrodt CH, Watkin JG, Molenberhs G, Carroll RJ. Choice of the primary analysis in longitudinal clinical trials. Pharm Sat. 2004;3(3):161–9.
White IR, Carpenter J, Horton NJ. Including all individuals is not enough: lessons for intention to treat analysis. Clin Trials. 2012;9(4):396–407.
Smith AE, Humphreys MS. Evaluation of unsupervised semantic mapping of natural language with Leximancer concept mapping. Beh Res Meth. 2006;38(2):262–79.
Catalano RF, Haggerty KP, Oesterle S, Flemming CB, Hawkins D. The importance of bonding to school for healthy development: findings from the social development research group. J Sch Hlth. 2004;74(7):252–61.
Poland B, Krupa G, McCall D. Settings for health promotion: an analytic framework to guide intervention design and implementation. Hlth Prom Prac. 2009;10(4):505–16.
Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Process evaluation in randomised controlled trials of complex interventions. Br Med J. 2006;332:413–5.
Loprinzi PD, Cardinal BJ. Measuring children’s physical activity and sedentary behaviors. J Exe Sci Fit. 2011;9(1):15–23.
Trost SG. Measurement of physical activity in children and adolescents. Am J Lifestyle Med. 2007;1(4):299–314.
This project is supported by Cancer Council NSW and the NSW Cancer Institute’s Evidence to Practice Grant.
This study is funded by CCNSW and the NSW Cancer Institute. JM and KJ are employees of CCNSW. The other authors (DD, MW, BW, WC) have no competing interests to declare.
DD, MW, WC and BW obtained funding for the research. All authors contributed to developing the protocols and reviewing, editing and approving the final version of the paper. DD is the guarantor and accepts full responsibility for the conduct of the study. All authors have read and approved the final manuscript.
Dean A Dudley, Matthew J Winslade, Bradley J Wright, Wayne G Cotton, Jackie L McIver, and Kirsten S Jackson contributed equally to this work.