Implementation of strategies to increase adolescents’ access to fruit and vegetables at school: process evaluation findings from the Boost study

This study has the following key findings:

The measurements of dose delivered showed that the proportion of teachers offering FV to the pupils every day declined noticeably from the time of the midway evaluation to the end of the intervention period, when measured at individual- as well as at school level. This may reflect a declining engagement of teachers over time which to some extent may be explained by the busier schedule of teachers by the end of the school year or by teachers and pupils growing tired of the project [35]. Deteriorating implementation over time have been found in several school-based studies using educational strategies [8,14-19]. This study contributes with new knowledge in the area of environmental strategies. Still, six schools were capable of maintaining a high level of implementation during the entire implementation period. This indicates that implementation does not have to decrease over time. Reporting results exclusively at an individual level would have led to an atomistic fallacy as we would have concluded that the teacher implementation declined at all schools. This point to the importance of reporting at an individual as well as at school level to avoid atomistic as well as ecological fallacy [36].

The dose delivered by FV suppliers was low in the first two months of implementation, then reached a high level for most of the months of intervention until it declined in the last month of the intervention. In the qualitative process evaluation of the Boost FV programme, the suppliers indicated that at the beginning of the intervention, the school delivery had not yet turned into a daily routine. This was a barrier for implementation, as they simply forgot their obligation to deliver the FV [21]. This finding contributes to the understanding that an introductory period must be expected before the delivery runs efficiently. The same tendency was found for fidelity concerning the freshness and appearance of the delivered FV. This increased throughout the implementation process, and may be explained by an improvement in the communication between teachers and suppliers regarding the delivery overtime. The increase might also be due to seasonal conditions where it might have been possible to deliver fresher FV over some seasonal periods compared to others. Qualitative findings from the Boost study confirm that several schools experienced poor FV quality at the beginning of the intervention period [21]. We assume that the decline of delivery from suppliers by the end of the intervention was caused by lack of clarity concerning the end date of the delivery. However, the decrease in delivery of FV in June did not affect the outcome the follow-up data collection as this was finalised early in June in all schools and as such before any decline took place.

Our findings suggest that changes in implementation levels over time vary between the different process measures of the two intervention components. Some tend to decrease throughout the intervention period while other aspects require an introductory period before being properly implemented. These findings point to the necessity of collecting data several times during the intervention period, as well as assessing each intervention component separately to obtain a complete and accurate picture of the implementation of an intervention as highlighted by Carroll et al. (2007) [25].

From the midway survey to the follow-up study, we found a decline in dose delivered by teachers in terms of the pupils getting FV every day, but an increase in the number of teachers cutting up FV during their lessons. This may indicate that at the end of the intervention period, teachers prioritised to allocate time for cutting up FV certain days at the expense of providing pupils with a piece of FV in class on a daily basis. In this way, the two components may become barriers for the delivery of each other. This is a hypothesis to be tested in the future e.g. by interviewing teachers about the reasons for positive and negative changes in implementation levels. We also found that the average percentages for teachers’ reporting ‘enough FV for every pupil to get one piece a day’ was rather low compared to teachers reporting ‘pupils offered FV 5 days a week’. We do not know the explanation of this finding but our suggestion would be that the difference is due to a too small amount of being delivered, while delivery it timely every day.

The pupil data reflected poor implementation of the pleasant eating environment component, as large proportions of pupils (32-40%) reported that FV were not cut up every day/most days and that adequate time was not allocated for them to eat FV in class. These findings show how teachers act as gatekeepers for implementation. As the teachers have not adhered to the implementation manual (low implementation of FV break and cut up FV) there is a risk that that the intended changes in the proximal outcomes of the pleasant eating environment component such as accessibility and social norms [29] have not been achieved as intended by the project group. This should be tested in mediation analyses.

Our data suggests that schools with stable high proportions of teachers offering pupils FV every day are characterised by being smaller and having fewer pupils with few economic resources. This may well be explained by the teachers at these schools having more time and energy to engage in the FV programme. This assumption is supported by the qualitative process evaluation of the Boost FV programme in which teachers’ lack of time was identified as a main barrier for the implementation of the FV programme [21]. Likewise, other studies have identified time issues as crucial for the implementation of interventions [32,37-43]. In agreement with our study, an Australian study found that the odds for adopting a FV break were significantly higher among small schools [13]. In line with other studies [12,24,26,44-46], we found that schools with stable high levels of dose delivered were characterised by having larger proportions of pupils and teachers appreciating the Boost project.

Schools with stable high levels of dose delivered were more likely to implement the educational intervention components in terms of curricular activities and parental newsletters. This indicates that engagement in one intervention component may be linked to an overall engagement in the entire intervention. This complicates the study of a separation of effects of each intervention component of a multicomponent intervention.

Schools with a stable high level of dose delivered were more likely to have a food policy. This may reflect a greater focus on health at these schools as well as a greater level of engagement and support to the teachers’ participation in the intervention from the principals [47].

Methodological issues

Study strengths include: a systematic and comprehensive approach to process evaluation [29], a large population of pupils, high response rates among pupils and principals, data source triangulation, teacher assessment at two different time points, assessment of implementation levels at the individual-, school-, and class-level and assessment of many process measures covering different aspects of the implementation process [9].

Study limitations include: low response rate among teachers and variations in number of teachers participating in the surveys at each school, lack of teacher data at the class level, differences in the phrasing of answer categories in questions to different data sources [48]. Furthermore the relatively small number of schools limited the power to detect significant differences in characteristics of different school types.

Selection bias: The low response rate among teachers challenges the generalizability of findings as it may be a certain group of teachers who have participated, probably those most engaged in the intervention. However, we do have at least one teacher response from each school and we received responses from schools reporting stable low or decreasing implementation levels, suggesting that also teachers with less engagement have completed questionnaires.

Misclassification bias: Responses from school principals, teachers and pupils are all at risk of recall bias as they have to report details that happened several months earlier. The risk is probably lower in the teacher data as several questions were asked twice during the implementation process, but they still report months back in time. Self-reported data may be affected by social desirability bias e.g. if teachers over report their implementation of the two components due to a feeling of obligations to the Boost project group [8,49]. This questions whether teachers are the best to rate dose delivered. Pupils’ reports’ of dose received may give a more accurate picture of the actual level of implementation [32]. Due to dichotomizing the variables used in the analyses we might have lost nuances in the respondents’ responses.

The data source triangulation provided us with information on the difference in experience of teachers and pupils, e.g. the appearance of the delivered FV was judged worse by pupils than teachers. A general trend for all process measures was that pupils assessed the implementation less successful compared to teachers. The differences between pupils’ and teachers’ perception of the quality and appearance of the FV delivered might be explained by the strong emphasis adolescents put on appearance and aesthetics of food [21,50-52]. Due to this emphasis the pupils may demand a greater quality of FV and the pupils may also be likely to have higher expectations concerning e.g. the variety of FV compared to the teachers. Also, the pupils are likely to have a different definition of the meaning of ‘having a good time’ in class. Another explanation may be that not exact the same wording was used in the questionnaires so they might not be fully comparable, e.g. the teachers were asked if the FV was fresh at delivery (from the FV supplier) while the pupils were asked if the FV was fresh. Pupils may have reported the freshness of FV which have been stored in class for several days while teachers may have reported the freshness at the time of delivery from suppliers.

As teachers are often responsible for several year 7 classes it was not possible to link the teacher data on implementation to specific school classes and examine between class variations in implementation levels. The aggregation of teacher information to the school level might have been too crude as pupil data showed large between class variations.

We measured pupils’ perception of implementation level only once: at follow-up. This may increase the risk of misclassification of pupil-reported implementation level. Computer tailoring was tested as a new tool for gathering process data on the pupils’ intake of the delivered FV at different times during the implementation period to supplement the information gathered at follow-up. Unfortunately, the response rate of the computer tailoring module was low and the data collection method was problematic as some pupils filled out the questionnaire in rapid succession and corrected their answers at a later point in time.

Judging from the low response rate and the risk of social desirability among teachers it seems to be an advantage to collect process measures from pupils compared to teachers.

It may be questioned whether the categorization of schools as stable low when under 80% is too high a cut point, as this cut-point leaves schools with e.g. 70% to be defined as having a stable low implementation level.

Implication for research

Future studies should collect process data at several time points to be able to document changes in the implementation process over time. This should however be balanced with the risk of tiring the implementers with more data collections possibly leading to lower response rates.

Our collection of process data using a computer tailoring module was not successful. The response rates were low and the pupils edited their answers along the way as they were able to log on to the module as many time as they wanted to over time. However, if pupils’ are already using computer tailoring module as part of an intervention, if ways are identified to increase response rates and if pupils’ opportunities for correcting answers are eliminated, we think that the computer tailoring has a great potential as a tool to collect process measures over time.

It was the intention of the Boost study to reach boys and adolescents from low SEP at least to the same extent as other pupils [28]. The environmental component Daily provision of free FV was successful in reaching pupils from different social classes and boys and girls equally. It remains to be investigated if the intervention was also equally effective across gender and socio-economic background measured by the total daily FV intake (the primary outcome). Girls and pupils from low socioeconomic background appreciated a pleasant eating environment more than others. Other studies also find that girls emphasize the social aspect of eating together more than boys [21,52,53]. Whether the detected differences in reach of this component are linked to socially differential intervention effects will be examined in effect analyses. The effect analysis will reveal whether other approaches than a social FV break are needed to change boys’ total daily intake of FV or if it is sufficient to provide FV for free.

Implications for practice

The increase in dose delivered by suppliers during the implementation period point to the importance of ensuring that the introduction period is long enough for implementation to take place.

As the proportion of teachers offering FV to the pupils decreased throughout the implementation period, it is important to maintain teachers’ engagement to ensure that pupils receive the intended dose. As lack of time has been found to be the main barrier for teacher implementation of a FV programme [21] one strategy to improve teachers’ engagement could be to secure the time needed for teachers to make a FV break mandatory in the national or local school curriculum.

The identification of characteristics of schools with stable high proportions of teachers providing FV to pupils every day can inform the development of strategies to increase programme implementation levels. Schools with many low SEP pupils may need more support for implementing FV programmes compared to schools with few low SEP pupils. However, we recommend that our findings on characteristics of different school types are tested in a larger school sample as our study was limited by involving 20 school units, only.