Two events during the study period influenced the Citizen Science approach and interventions implementation. These events included organizational changes at the terminal company and the Covid-19 pandemic. As a result, interventions were postponed because organization possibilities for intervention implementation were limited and some workers and health ambassadors dropped out.
Evaluation citizen science approach
Findings are reported according to the Nielsen and Randall framework, structured by three themes with accompanying process components (see Table 1). For the evaluation of the Citizen Science approach, the following two of the three evaluation themes were evaluated: intervention design and implementation, and context. Process components that were included in these themes and structured the study results consisted of: communication, participation, satisfaction, culture and events were included as process components for the evaluation of the Citizen Science approach.
Intervention design and implementation
At the construction company, workers were approached and informed by their managers. At the terminal company acquaintance with the intervention and the registration to participate was organized via sessions and through e-mail contact. Based on the interviews, some workers of the terminal company experienced that the overall communication went well. For example, one worker of the terminal company said: “yes, I definitely felt involved in the emails, and overall the communication went well.”
However, in both companies, some workers mentioned that the communication intensity decreased during the course of the intervention and the preference for continued communication. As one worker said: “I least liked the attention for it, because at a certain moment there was no communication.” Also, a need for a clearer introduction and information provision to the managers at the beginning of the intervention was mentioned, as one worker of the terminal company, for example, suggested: “We should have organized a meeting for the managers. That is something we should adapt in the future.”
According to the interviews, the majority of the interviewed workers at both companies indicated that they were actively involved in the intervention. One worker of the construction company, for example, reported: “I certainly felt involved, but that’s because it’s a subject that interests me.” Another worker said: “Yes, I did felt involved. But, as I said, I am into the subject around nutrition, so then it interests you.” Workers of the terminal company suggested that the involvement of managers could be optimized by organizing an information session for the managers.
As to the participation as health ambassador, some workers experienced to have little influence, and some workers mentioned that they actively motivated colleagues and promoted the intervention to colleagues. It was also expected that the impact of health ambassadors could be greater. As one worker of the terminal company reported: “Perhaps the expectation that they would start small projects.” Another worker added: “I also expected more from the ambassadors, that it would become more alive and that they would manage to involve more people.”
Interviewees experienced the involvement in content creation and the approach as positive and the open atmosphere created enough opportunities for questions. As one worker of the terminal company reported: “I liked it the most to be involved in the beginning and during the conversations to find out what should be included in the intervention. There was more room for conversations, more people were present and I was able to share my ideas.”
According to interviewees of the construction company, their company is open-minded, which offers room for opinions from workers. Besides, most workers indicated that they felt supported by their employer. Although workers at the terminal company indicated that they have a pleasant culture at the workplace, they indicated the need for more guidance and involvement from the employer/supervisor.
Furthermore, workers mentioned in the interview that the employer should emphasize the importance of participation in the intervention with special attention for the workers’ health. One worker, for example, expressed this opinion as: “It must become healthier, but not only to reduce absenteeism. Then people would start to think that it is only in favor of the employer.”
Some differences as to the culture among older and younger workers seemed to appear from the interviews. According to the interviewees, the older generation within the companies are less concerned about their health, compared to the younger generation of workers.
Workers in both companies mentioned in the interview that the mentality is characterized by an ‘alpha’ culture with dominant male leadership and an individualistic attitude. In addition, workers of the terminal company indicated that there is a conservative and distrustful attitude, which negatively influences the willingness. One worker said: “We have been used to something for years and when something changes, it is always an issue. The fear of changing something.” Besides, workers do not always feel comfortable to talk about personal topics such as health.
According to some workers, physical work-related factors and a high workload might have influenced the intervention implementation to some extent. For example, one worker of the construction company said: “The pressure is quite high. You also have to deal with deadlines.”
While some workers at the construction company were more aware about their health due to the pandemic, others indicated that they were eating less healthily and were struggling to be physically active. For example, one worker reported: “We talk more often about health, especially now during corona you are more aware of it.” Another worker indicated: “I notice that I became an emotional eater during that time.”
During the study period there were organizational changes that led to the drop out of workers and ambassadors and delay in the implementation of the intervention. According to the logs, it was difficult to schedule workshops and training courses. Moreover, another big ‘event’ that influenced the intervention process and approach was the Covid-19 pandemic. According to the logs, this led to a postponement of the intervention and limitation of the organization’s possibilities.
Evaluation resulting WHPPs
For the evaluation of the Citizen Science approach, the following three evaluation themes were evaluated: intervention design and implementation, context, and mental models. To evaluate the study results of the Citizen Science approach according to these themes, the following process components were included: communication, participation, reach, satisfaction, tailoring, exposure, participation/utility, culture, readiness for change and perception were included as process components for the evaluation of the resulting WHPPs. An overview of the three evaluation themes with accompanying process components and operationalization are presented in Table 1.
Intervention design and implementation
Some workers of the construction company reported in the interview they obtained some prior knowledge about the toolbox before the start, which involved mainly information about the toolbox in general and the included topics of the toolbox. Interviewees who indicated that they had no prior knowledge about the intervention, indicated that they started their intervention with no specific expectations.
Questionnaire data showed that a majority of the workers who had filled in the questionnaire (62%, n = 109/175) felt involved during toolboxes. With regard to the statement about the probability of expressing an opinion during sessions, 72% (n = 126/175) of the participants at the construction company agreed.
With regard to the participation, it was mentioned in the interview that involvement was especially experienced at the beginning of the intervention. According to the logs of the terminal company, during meetings with workers and employers about the approach of the intervention, there was little interaction with and input from the workers. As communicated by a worker, most workers did not feel comfortable having the meeting together with the employer/management. In contrast to the terminal company, logs of the construction workers indicated a positive atmosphere with a lot of interaction during toolbox sessions, especially in small groups. During sessions with larger groups, more resistance was felt.
According to the interviewees and the logs, improvement of the participation and involvement throughout the intervention mainly consisted of the advice to narrow the group sizes. One worker of the construction company said: “In a group it is just striking that some do not dare to ask questions, they are afraid that they will be laughed at. It would be better to organize it one on one or in small groups.”
From the questionnaire data, it appeared that about 83% of the workers who filled in the questionnaire (n = 151/175) indicated that they were present at toolbox 1 and 77% (n = 135/175) indicated that they were present at toolbox 2.
Workers of both companies agreed upon the fact that it is not feasible to reach everyone with the intervention, since it does not fit everyone’s interests. The older generation in particular was mentioned as difficult to reach, as one worker of the terminal company reported: “The older generation thinks very differently. They are much less concerned about their health. They don’t take it seriously.”
Moreover, planning outside working hours was experienced as a pressure on flexibility. As described in the logs, the management of the terminal company had decided that the workshops had to be followed in the workers’ own time because it was considered impossible for the company to give the workers time off to follow the workshops. According to workers of the terminal company, the relatively low attendance was probably due to a misfit with interests and the unfavorable planning. As indicated in the logs of both companies, not all workers were able to attend sessions because, for example, they had to work changing shifts or because they were following another course at that time.
Workers of the construction company indicated that they appreciated the non-committal nature of participation. In contrast, workers of the terminal company indicated that not making the session mandatory may have affected the reach negatively. One worker of the terminal company said: “It would be a very good way to oblige, because this is optional and the toolbox is mandatory.”
Satisfaction with the toolbox health was rated with a mean score of 6.7 (SD = 1.4). A minority of 14% (n = 25/175) of the workers rated the satisfaction as insufficient and scored it with a 5 or lower. According to the interviews, overall, the toolbox was experienced as fun and interesting, being a good initiative and concept to draw more attention to health in the workplace. One worker terminal company said: “I think the positive aspect about the toolbox is that you try to raise some awareness about health.” Another worker reported: “For me it was like a reset.”
From the poll about the ideas of Tip Top Fit intervention within the terminal company, the idea of presenting flyers with tips and tricks about nutrition/sleep/work during night shifts was perceived as the best idea with a total score of 265 (29%, n = 265/902). With a score of 223 points (25%, n = 223/902) the idea of health information provision became second, followed up by the idea of using stickers to visualize healthy food choices with 219 points (24%, n = 219/902), and the idea to encourage to take the stairs instead of the elevator with 195 points (22%, n = 195/902).
As to the appropriateness of tools and materials, some workers reported that they experienced the use of the pedometer and the food diary useful, others indicated that they used them not often or not at all. Moreover, it was mentioned that a one-hour session may not be functional enough and that there was a need for structural repetition.
Most workers agreed that they would recommend the intervention for other companies. One worker of the construction company, for example, said: “In this particular branch of construction work it is very difficult to eat really healthy, but I do think this toolbox is advisable for other companies.”
Based on the questionnaire, just over half of the workers at the construction company agreed on the statement concerning the alignment of ideas with individual wishes to make the workplace healthier (54%, n = 94/175). Moreover, a small majority of the participants (51%, n = 90/175) agreed on the statement “The health toolbox matches the wishes I have with regard to my own health”; 36% (n = 63/175) neither agreed nor disagreed and 11% (n = 20/175) disagreed with the statement.
As to the perceived intervention fit with the problems and needs of the workplace, some workers of the construction company mentioned that the intervention did not match the interests and needs of some of the construction personnel. One worker reported: “As I mentioned, we get enough daily exercise on a day of work.” Another worker said: “I think office workers would benefit more from it, because they have a sedentary profession.” Some felt it was not suited for their organization because it was not in balance with the high workload. For example, one worker said: “The pressure is quite high. You also have to deal with deadlines.” As noted in the logs, there was uncertainty as to whether the approach would match the target group.
In contrast, some workers of the terminal company reported that the intervention was well suited to their interests and work environment. One worker reported: “I thought it was useful. Especially the information regarding night shifts, what you can and cannot eat and how the biological clock works. Now I am aware of what to eat when I’m feeling tired.”
About half of the workers working at the construction company (49%, n = 86/175) neither agreed nor disagreed with the statement regarding the perceived health change. About 28% (n = 49/175) agreed that the workplace has become healthier, while 18% (n = 31/175) disagreed on the statement.
While some workers of both companies experienced health improvements, others mentioned they noticed little or no change at the workplace. One worker of the construction company reported: “I don’t think something changed yet. We’ve had the toolbox, but I don’t think people are working on it yet.” Another worker said: “I don’t think you can change that with a toolbox, that’s very difficult I think.”
Some workers at the construction company mentioned that they were not aware of the existence of an idea box nor the implementation of the ideas. One worker reported: “At least I haven’t seen an idea box.” Another worker said: “I think that idea box was used minimally.” A possible reason for the minimal use of the supplied ideas box may be related to the fact that the construction company already had a general idea box.
With regard to the use of implemented interventions, some of the workers at both companies indicated in the interviews that they had made some positive adjustments. One worker of the terminal company said: “I did start with fitness and also started living healthier. At least, I try.” According to some workers of the terminal company, the information and shared experiences were very helpful to make healthy improvements in their diet. Some mentioned that they eat more consciously and healthfully during night shifts and that they exercise more often.
At the construction company, a number of workers mentioned that they stopped eating more fruit and did not use the idea box anymore. In contrast, others indicated that they try to eat healthier, exercise more and snack less unhealthy. One worker said: “Eating healthier, you try to think about that more often.”
About half of the workers at the construction company (49%, n = 86/175) neither agreed nor disagreed on the statement about the experience that it was easier to talk about health with colleagues after the toolbox. With 22% (n = 38/175), the number of workers who agreed was almost equal to the number of workers who disagreed on this statement (21%, n = 37/175). Also, mixed results appeared on the perceived support from colleagues as well as from the employer.
A majority of 52% (n = 70/133) (T0) and 51% (n = 68/133) (T1) indicated that they were already living a healthy life. In addition, a percentage of 38% (n = 51/133) who intended to live healthier at T0, remained unchanged at T1.
Some workers were satisfied with their current lifestyle and therefore felt no need to change. One worker of the construction company said: “We are physically active enough already. I walk about 12,000 steps a day.” Another worker of the construction company reported: “I’ve just lived the same as I always live, I’m not going to change anything. Because I know that I eat healthy, eat enough vegetables and enough fruit, sometimes even too much.”
Data from the questionnaire underlines the influence of the workplace’s health on workers. While 31% (n = 54/175) neither agreed nor disagreed on the statement “It helps me personally if the workplace becomes healthier”, a total of 54% (n = 95/175) agreed to strongly agreed on the statement (47%, n = 83/175; 7%, n = 13/175).