Mixed-methods process evaluation of a residence-based SARS-CoV-2 testing participation pilot on a UK university campus during the COVID-19 pandemic

Abstract

Background

Regular testing for Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) is an important strategy for controlling virus outbreaks on university campuses during the COVID-19 pandemic but testing participation rates can be low. The Residence-Based Testing Participation Pilot (RB-TPP) was a novel intervention implemented at two student residences on a large UK university campus over 4 weeks. The aim of the pilot was to increase the frequency of asymptomatic SARS-CoV-2 saliva testing onsite. This process evaluation aimed to determine whether RB-TPP was implemented as planned and identify implementation barriers and facilitators.

Methods

A mixed-methods process evaluation was conducted alongside the RB-TPP. Evaluation participants were students (opting in, or out of RB-TPP) and staff with a role in service provision or student support. Monitoring data were collected from the intervention delivery team and meeting records. Data were collected from students via online survey (n = 152) and seven focus groups (n = 30), and from staff via individual interviews (n = 13). Quantitative data were analysed descriptively and qualitative data thematically. Barriers and facilitators to implementation were mapped to the ‘Capability, Opportunity, Motivation–Behaviour’ (COM-B) behaviour change framework.

Results

Four hundred sixty-four students opted to participate in RB-TPP (98% of students living onsite). RB-TPP was implemented broadly as planned but relaxed social distancing was terminated early due to concerns relating to national escalation of the COVID-19 Delta variant, albeit testing continued. Most students (97.9%) perceived the period of relaxed social distancing within residences positively. The majority engaged in asymptomatic testing (88%); 46% (52% of testers) were fully compliant with pre-determined testing frequency. Implementation was facilitated by convenience and efficiency of testing, and reduction in the negative impacts of isolation through opportunities for students to socialise. Main barriers to implementation were perceived mixed-messages about the rules, ambivalent attitudes, and lack of adherence to COVID-19 protective measures in the minority.

Conclusions

This process evaluation identifies factors that help or hinder the success of university residence-based outbreak prevention and management strategies. RB-TPP led to increased rates of SARS-CoV-2 testing participation among students in university residences. Perceived normalisation of university life significantly enhanced student mental wellbeing. The complexity and challenge generated by multiple lines of communication and rapid adaptions to a changing pandemic context was evident.

Trial registration number

UKAS 307727–02-01; Pre-results. ClinicalTrials.gov Identifier: NCT05045989; post-results (first posted, 16/09/21).

Ethical approval

Faculty of Medicine & Health Sciences Research Ethics Committee, University of Nottingham (Ref: FMHS 96-0920)

Strengths and limitations of this study

• This is the first study to report a process evaluation of an initiative aimed to increase student participation in SARS-CoV-2 testing in university residences. • This study provides a worked example of a pragmatic approach to process evaluation to explore the implementation of a rapid response intervention in the context of a pandemic. • A strength of the study is that the process evaluation uses both qualitative and quantitative data to illustrate intervention delivery, the facilitators and barriers to implementation and perspectives of multiple participants and stakeholders. • A limitation of the study is that the process evaluation included only a small number of students who had chosen not to take part in the scheme.

Background

The Coronavirus Disease 2019 (COVID-19) pandemic cause by the novel Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) is highly contagious [1] and the world’s population is susceptible to infection [2]. If not identified and controlled quickly, an outbreak on a university campus would have potential for explosive and extensive spread, threatening the immediate and wider community. National initiatives, such as the United Kingdom (UK) Track and Trace programme [3], to target symptomatic cases and their contacts are unlikely to identify university outbreaks rapidly, as published data shows that most infections in these individuals will be asymptomatic [4, 5]. Evidence shows that individuals with minimal or no symptoms can still transmit the virus [6, 7]. Therefore, prevention of largescale virus outbreaks within the University community has required appropriate mitigation (strict personal hygiene, improved estates cleansing etc.) as well as containment (testing, contact tracing and quarantine). High-frequency surveillance testing (i.e., once or twice per week) is considered to be an effective strategy for COVID-19 disease mitigation [8, 9]. Saliva testing is one approach to both asymptomatic and symptomatic detection of the presence of replicative SARS-CoV-2 RNA, with a reported accuracy of > 99% and a sensitivity of 1–10 viral copies/μl [10]. A systematic review and meta-analysis showed that the diagnostic accuracy of saliva nasopharyngeal swab nucleic acid amplification testing (NAAT) diagnostic accuracy is similar to that of nasopharyngeal swab NAAT, especially in the ambulatory setting [11]. As such, saliva-based SARS-CoV-2 surveillance testing programmes have been operationalised in university settings in various geographical regions (e.g., [12,13,14,15,16].

At the University of Nottingham, asymptomatic testing has been available from an internal Asymptomatic Testing Service (ATS) since September 2020. Students arriving at the University to residences on the campus are offered asymptomatic tests (for the detection of the presence of replicative SARS-CoV-2 RNA) on arrival. A pilot study conducted during the summer of 2020 found high adherence to regular testing and acceptability of socialising via ‘household bubbles’ [16], but this was on the University’s rural campus, with the first cohort of students to occupy university residences since the outbreak of the pandemic. By Autumn 2020, the local and national situation had markedly changed [17]. Students were arriving at, or returning to, campus in a context of a second surge of COVID-19 in the UK, and the highest rates of COVID-19 in a UK higher education setting [18], with an escalating number of positive cases requiring students to self-isolate [17].

Although most students tested on arrival, the majority did not continue with regular (weekly) testing and testing uptake rates rapidly declined (participation in testing dropping from 58 to 5% [17]). This was primarily associated with fear of the negative impacts of self-isolation, loneliness and the impacts of positive test results on peers [17]. Attempts at enforcing household bubbles and other social distancing rules and regulations that had worked in a different context and environment [16] were less acceptable to students living in large traditional residences, on campuses close to the city. While approaches to the delivery of testing in university settings have emerged internationally (e.g., [16, 19, 20], there is limited evidence on strategies for increase testing uptake. At the time of writing, solutions to increasing rates of testing participation are urgently needed to inform future higher education policy and practice around outbreak prevention and management.

Clustering of positive cases has been identified in university residences (up to 31%) [20]. Recent SARS-CoV-2 transmission modelling suggests that surveillance-based informative testing strategies targeting university residences are more effective at detecting positive cases than random or voluntary testing [21]. Therefore, a novel, Residence-Based enhanced SARS-2 coronavirus Testing Participation Programme (RB-TPP) was initiated, aiming to increase the uptake and frequency of testing for SARS-CoV-2 RNA, in university residences, whilst simultaneously allowing some relaxation of social distancing restrictions within buildings.

Process evaluation is vital for understanding how interventions function in different settings, including if and why they have different outcomes or do not work at all. This is particularly important in trials of complex interventions in ‘real world’ organisational settings where causality is difficult to determine. We report a process evaluation conducted alongside the RB-TPP that explored the impact and consequences of the programme for students and staff, and established views on key aspects of the programme in order to aid better understanding of how, why and for whom such approaches and interventions are effective. This process evaluation aimed to provide insight into the value of the RB-TPP approach to prevention of COVID-19 outbreaks on university campuses. The objectives were to explore (a) the intervention as it was implemented (to ascertain the extent to which it was implemented as planned); (b) how people participated in and responded to the intervention (to ascertain the barriers and facilitators to implementation); and (c) the contextual characteristics that mediated this relationship and may influence outcomes.

Methods

Study design

This is a convergent parallel mixed-methods [22] process evaluation following the UK Medical Research Council guidelines [23]. Intervention fidelity is the degree to which an intervention is delivered as intended. The components of implementation fidelity evaluated here are: Reach (the proportion of the target group who participated in RB-TPP and their socio-demographic characteristics), Dose and Timeliness (of the intervention delivered) and Adherence/Compliance (of students to the minimum programme requirements). This process evaluation corresponds with the inputs, activities and outputs detailed in the RB-TPP logic model and interrogates the assumptions underlying the model and the linkages between the intervention components and outcomes (Fig. 1). The framework for documenting RB-TPP programme implementation and data sources is shown in Table 1. Terms are explained in Additional file 1. The study reporting adheres to the consolidated criteria for reporting qualitative research guidelines [24] (Additional file 2), and the TIDieR (Template for Intervention Description and Replication) Checklist (Additional file 3) [25] has been used to describe the intervention. The Checklist for Reporting Results of Internet E-Surveys (CHERRIES) [26] guided the reporting of survey findings.

Fig. 1

Logic Model for Residence-Based SARS-CoV-2 Testing Participation Pilot

Table 1 Framework for documenting RB-TPP programme implementation and data sources

Study setting and participants

A total of 588 registered university students were listed occupants in two, similar, mixed-gender residences on a single UK university campus at the start of the study (April 2021) and eligible to participate (site 1: 366, site 2: 222). Of these, 116 were not living onsite (by choice) due to the COVID-19 pandemic (e.g., they had not physically returned to the university campus during the pandemic) and therefore did not take part in the RB-TPP. There were 472 students living onsite at the time of the study (80% occupancy; site 1: 311, site 2: 161), of whom 464 provided written informed consent online, to take part (98%; site 1: 306, site 2: 158). There was a two-step approval process to take part in RB-TPP. First step was an online privacy notice, which contained consenting to have saliva samples tested for presence of SARS-CoV-2 RNA and, if positive, for follow-up analysis of that sample to include virus sequencing but excluding any human DNA sequence analysis. This also asked if participants would be willing to be approached for the purposes of research. Next, there was a separate online consent form for research participation. Students who opted out of the programme were re-located to alternative temporary accommodation during the study period. Reasons for decline were perceived risk for COVID-19, and inconvenient timing of the programme due to its proximity to academic examinations. The settings were deemed to be more ‘traditional’ residences with large corridors, shared facilities, communal dining and socialising models and where a prior COVID-19 mitigation approach of the small group ‘student household’ (e.g., in [16]) was less relevant. Eligible process evaluation participants were in-house students who had either taken part in or opted out of the RB-TPP, and staff with a role in intervention delivery or student support.

The intervention: residence-based asymptomatic testing participation pilot (RB-TPP)

The aim of the RB-TPP was to increase and maintain participation of students in regular testing for SARS-CoV-2 RNA, in university residences. The RB-TPP (Fig. 1) was planned for delivery over 4 weeks in April–May 2021 and required asymptomatic students to take a saliva test to detect SARS-CoV-2 RNA, twice weekly for 4 weeks. This was combined with relaxed social restrictions within the residence during the study period (i.e., removing the need for 2-m distancing between students living in the same residence), devolved local contact tracing (i.e., contacts traced locally by a university and local public health team, rather than the national Track and Trace service) and enhanced support for students who were required to self-isolate. The processes were agreed with NHS public health partners and were in alignment with UK law. All students had access to usual university welfare support systems (e.g., welfare / disability advisors, a university Student Hardship Fund (standard provision) and Student Crisis Fund (additional provision during the COVID-19 pandemic) for students experiencing financial difficulties, counselling services, etc). Specific additional support for self-isolating students included attendance to food and medication needs, provision of telephone support for students, and telephone reassurance for close relatives (if requested by the student, to ensure confidentiality was observed). A dedicated email helpline was established for enquiries, with ATS service staff providing responses Mon-Fri 09:00–17:00. Any identified concerns relating to student welfare were raised with student welfare teams and/or academic tutors as appropriate. Students were required to wear face coverings when interacting with staff (e.g., in dining rooms), although face coverings were optional in areas of the residence where students were mixing only with other students (e.g., in student bedrooms or social areas). The identification of any individuals testing positive for SARS-CoV-2 RNA during this period would trigger residence surge testing, whereby all students living in the hall would then be required to test daily for 7 days. Additional surge testing was available for contacts of positive cases outside of the residences, if required (e.g., for academic cohorts if the student had been attending face-to-face teaching sessions).

Communications with students were focused on expectations (testing and social behaviour), testing processes and logistics. Communications were delivered primarily by email, supported by face-to-face communications from staff with student-facing roles, and three student ambassadors known as ‘testing champions’ offering peer-to-peer support and encouragement to participate. The testing service provider was the host University’s flagship asymptomatic testing service (ATS), which at the time of writing was one of only eight laboratories specifically recommended for, or accredited for, SARS-CoV-2, in the UK. The views of university staff and students towards the ATS have been published elsewhere [16, 17]. Tests were non-invasive self-administered saliva tests that were collected and analysed in the University’s laboratories. Specifically, they were reverse transcription polymerase chain reaction (RT-qPCR) tests that included the assay controls for the qualitative detection of viral RNA from SARS-CoV-2 in saliva specimens (since SARS-CoV-2 is an RNA virus, the genetic material for SARS-CoV-2 is encoded in ribonucleic acid (RNA)). During the study period, a second confirmatory National Health Service (NHS) Pillar 2 test was required for all SARS-CoV-2 RNA positive individuals identified via asymptomatic tests (no longer required from July 2021). The testing service set up deployments in the dining areas of the two sites, which were staffed on Tuesdays and Fridays for the students to drop off their samples. Opening hours were initially 10:00–14:00, but hours were extended to give students greater flexibility in when they could drop off their samples – extending hours initially to 16:00, then to 19:00 after student feedback. All RB-TPP processes were overseen by the university COVID-19 Testing Operations group working in collaboration with local and national public health teams.

Data collection

We gathered quantitative measures of intervention activities (such as number of students participating in the residence-based testing participation scheme) [27], and qualitative exploration of the interaction between the programme, how students and staff experience it, and the contextual characteristics of the two sites in which it was delivered. This is detailed in Table 1 and Fig. 2.

Fig. 2

Process Evaluation Data Collection

Implementation fidelity

Reach

Program reach was evaluated through the number of students recruited into the RB-TPP compared to the number of potentially eligible students living in one of the two participating sites in April 2021 (identified via residence manager’s records). Recruitment data were collected by the COVID-19 Testing Operations Team and entered directly into a secure web-based database. Reasons for participating (or not), and characteristics of students opting in and out were collected in an end-of-programme survey.

Adherence/compliance

Participant compliance was defined as the proportion of students that completed two tests per week. Pre-defined compliance was therefore completion of 8 tests over the 4-week period. Objective data on uptake were recorded by the COVID-19 Testing Operations Team and student self-reports of reasons for compliance and non-compliance with testing, and adherence and non-adherence to pre-defined behavioural expectations were collected in an end-of-programme survey.

Dose and timeliness of intervention delivery

Data related to the dose delivered (number of tests offered, total duration of intervention) and timing of intervention delivery (when tests were available, when and how results were received, additional surge testing, contact testing). Data were recorded by the COVID-19 Testing Operations Team with any reasons and challenges raised explored in the end-of-programme survey.

Online survey

Students opting in or out of the intervention received an email at programme end, sent from the ATS, containing a direct link to an information sheet and online survey (Additional file 4), hosted on Jisc Online Survey platform with automatic capture of responses. Items were adapted from a prior study [16] and contained a mixture of closed and open-ended free-text questions. The survey contained approximately 2–4 items per page, over 19 pages. Adaptive questioning (certain items, or only conditionally displayed based on responses to other items) was used to reduce number and complexity of questions. Non-response options were included, participants could amend their answers up to the final page, and completeness checks occurred prior to submission using JAVAScript. Usability and technical functionality of the electronic questionnaire were tested before fielding. Those who volunteered to take part in the survey received a £5 gift voucher as compensation for their time. The survey contained 55 items in 4 sections. Section 1 included 16 categorical items relating to socio-demographics (age, gender, ethnicity, international/home student, work status, stage of study, accommodation type), and prior experiences of COVID-19 (self/others). Section 2 included 25 items relating to confirmation of participation (or not) in the RB-TPP, reasons for taking part, number of tests taken (uptake/engagement) and adherence (testing according to protocol), views towards the intervention (testing and follow-up, logistics, communication, plans for surge testing, adherence to social behaviour regulations during the intervention), and experiences of self-isolation and support. These items were a mix of categorical and open-ended questions. Section 3 included 7 items relating to COVID-19 risk perception before and after the intervention (rated on a scale of 1 = ‘I didn’t think I would get it’ to 10 = ‘I knew I would most certainly get it’), views towards COVID-19 protective behaviours (social distancing, face coverings, handwashing, self-isolation - rated from 1 = not at all important, to 10 = extremely important). Worry about self or others contracting COVID-19 over the past 2-weeks were assessed on 4-point scales (‘do not worry...’, ‘occasionally worry...’, ‘spend much of my time worrying...’, ‘spend most of my time worrying...’). Anxiety was measured by the Generalised Anxiety Disorders Scale (GAD-7 [28, 29];. The GAD-7 is commonly used 7-item measure of general anxiety symptoms across various settings and populations, with established reliability and validity in clinical and non-clinical populations [28, 30], and university student samples [31,32,33]. The scale has shown high internal consistency (Cronbach’s alpha 0.83) [33]. The GAD-7 total score is calculated by assigning scores of 0, 1, 2, and 3, to the response categories of ‘not at all’, ‘several days’, ‘more than half the days’, and ‘nearly every day’, respectively, and summing item scores. Scores of 5, 10, and 15 are taken as the cut-off points for mild, moderate, and severe anxiety, respectively. Using the threshold score of 10, the GAD-7 has a sensitivity of 89% and a specificity of 82% [29].

Student focus groups and staff interviews

The qualitative arm of the study aimed to explore the impact, consequences and experiences of the staff and students involved in the RB-TPP and establish their views on all aspects of the programme and the implementation fidelity components detailed above. Students’ perspectives were examined through seven focus groups planned to take place at the end of week 2 (mid-point, May 2021). This included six groups for students (opting in or out of the RB-TPP), and a single group for student ambassadors (‘testing champions’). Group size ranged from 3 to 6 attendees. All students received £20 as compensation for attendance at a focus group. Staff perspectives were examined through individual interviews planned to take place in week 4–5 (end point, June 2021). Eligible participants were purposively sampled according to role (testing operations, residence management, student support). They were contacted by email, provided with a study information sheet and invited to take part in a focus group or interview. Student focus groups were conducted by two researchers (SC, LF) and lasted between 42 and 61 minutes (mean 52 minutes). Staff interviews were conducted by one researcher (LF) and lasted between 15 and 35 minutes (mean 22 minutes). Researchers collecting and analysing data had no involvement with the testing service or university residences. All participants in focus groups or interviews received an information sheet and provided written informed consent online, prior to the interview (additional to consent provided for participation in the RB-TPP). Interviewers used question guides (Additional file 5) developed by the lead author, a health psychologist, in consultation with the process evaluation team and members of a patient and public involvement and engagement (PPIE) group, and field notes were taken. The guide included prompts to discuss opinions about programme content, dose, delivery style, and delivery mode, as well as perceived benefits of and barriers and facilitators to participation. All interviews and focus groups were held via a video-conferencing platform, audio-recorded and transcribed.

Barriers, facilitators and acceptability

Barriers and facilitators to implementing RB-TPP, and acceptability of the programme to students and staff were identified through the student focus groups and survey, and staff interviews as detailed above.

Participant and public involvement

Student and staff views informed the study design and interview questioning guides at the point of study conception, via a Participant and Public Involvement and Engagement (PPIE) group. Students expressed a preference for small (n < =6) focus groups, and staff preferred to participate in individual interviews. Study findings will be disseminated to all participants through this publication and lay summaries disseminated via the participating university.

Data analysis

Quantitative survey data were analysed using descriptive analysis and non-parametric tests of association (Spearman’s Rho, Kendall’s Tau, and Chi-Square). Qualitative data from the semi-structured interviews and focus groups were analysed by two researchers (SC, LF) using deductive and inductive coding [34]. First, coding was guided by the assumptions of the RB-TPP programme logic and key components of the intervention. This included: impact of the RB-TPP on containing the spread of COVID-19, impact on students (e.g., personal risk and wellbeing, satisfaction, social behaviours, testing processes, identifying cases, wellbeing etc), impacts on staff (e.g., personal risk and wellbeing, satisfaction, resources etc), views towards key components of the pilot (e.g., communications, ambassador role, testing process, test type, logistics, social aspects, surge testing, enhanced contact tracing, managing positive cases, isolation support), barriers and facilitators to implementation and outcome (e.g., social factors, government policy, new guidance, incidents) and future recommendations. Then an inductive approach was used to code relevant features of the data beyond the pre-defined categories. Coding was undertaken using NVivo 12 software (released March 2020) [35]. Discrepancies were discussed until consensus was reached. Themes were identified from the codes and mapped to the Capability Opportunity Motivation – Behaviour (COM-B) model [36]. This model categorises behaviour (B) as the result of an individual’s capability (C); opportunity (O); and motivation (M), to perform the behaviour. The behaviours of interest for this evaluation were: (i) participation in the RB-TPP programme (student participants); and (ii) delivery of the RB-TPP (staff participants).

Results

Data were all collected from programme end. Student focus groups were conducted within 1 week, and staff interviews within 3 weeks. The survey closed after 3 weeks. Monitoring data relating to uptake and reach were collected concurrently. Due to a national escalation of positive cases of the COVID-19 Delta variant at the time, the primary behavioural element of the RB-TPP (relaxed social distancing rules) was terminated (after 10 days, broadly the mid-point). Due to a need to focus on the institutional response to the changing national picture, testing compliance data for Site 1 and Site 2 were therefore collected only from Thursday 6th May to Sunday 16th May 2021, although the testing provision was retained. Therefore, for these purposes, any student who had tested three times or more over this period was therefore classed as fully compliant.

In this process evaluation, 152 students completed the online survey (88 women, 63 men, mean age 19.24 years; SD = 1.34) of whom 145 (95.4% of survey respondents, 31% of RB-TPP participants) had participated in the RB-TPP. All responses were included in analysis. Total survey participation rate was 34% of RB-TPP participants. Survey participants were broadly representative of RB-TPP participants. There was a total of 30 students (14 women, 16 men, mean age 19.9 years; SD = 2.31) attending one of 7 focus groups, and 13 staff (7 women, 6 men) were interviewed. Staff job roles were related to the testing service (strategic and operations), hospitality, accommodation or other student support. The staff interviewed were student facing (e.g., welfare support, hall managers, domestic staff with direct student contact) (n = 6) and non-student facing (e.g., service operations staff without direct student contact) (n = 7).

Implementation fidelity

Reach

A total of 464 students chose to participate in RB-TPP (site 1: 306; site 2: 158). This represented 98% of students who were resident in the two sites at initiation of the RB-TPP (and 79% of all students listed as occupants, including those who were not present on campus at the time of the study). Survey participant characteristics are presented in Table 2, reasons for participation are shown in Table 3. There was no significant difference in the proportion of participants consenting to the RB-TPP at site 1 and site 2, or in sociodemographic characteristics of those who opted in, or out. Further details of student health and prior experience of COVID-19 are provided in Additional file 6.

Table 2 Characteristics of RB-TPP participants

Table 3 Reasons for participation in the RB-TPP (n = 145)

Of survey respondents, 24 students undertaking paid or voluntary work, of whom 12 identified themselves as key workers (e.g., health or social care, food chain supplies, public service), regularly coming into close contact (< 2 m) with others outside of the residence. Although reasons for participation were diverse, the three most common reasons for participating were positive: to contribute to the national efforts to contain COVID-19, helping to keep campus safe for everyone, and getting to know other students better (Table 3). A minority participated to avoid negative consequences (e.g., perceived pressure, not wishing to relocate during the intervention). Seven survey respondents had opted out of the programme (5 M, 2F;1 keyworker). Over one third of survey respondents had tested positive for COVID-19 at some point during the pandemic and 83.2% had been required to self-isolate at least once before. Of those who opted out, only two had previous experience of self-isolating and none had tested positive for COVID-19 previously. All 7 students reporting an existing physical health issue had participated in the pilot. One fifth of the sample (n = 29, 19.3%) reported a prior history of mental health issues. All the survey respondents reported symptoms of anxiety on GAD-7, and this was moderate to severe in 45.9% (n = 68) students who met the screening threshold for general anxiety disorders (score > =10 on the GAD-7 [28, 29];).

Adherence/compliance

Saliva testing

Intervention monitoring data showed that 409 of the 464 intervention participants (88%; site 1: 278, site 2: 131) completed at least one test during the data collection period. A total of 213 (site 1: 134, site 2: 79; 46% of all participants; 52% of those who tested) were classed as fully compliant. There were no reactive tests identified. Of the 145 respondents, 64.1% (n = 93) were extremely and 33.8% (n = 49) somewhat confident in the results of their asymptomatic saliva tests. Almost all students were satisfied with the physical process of taking a saliva test (95.9%, n = 139/145). Non-compliance was largely due to students being away, or missing drop-off times due to academic commitments (prior to the timings being extended to address the raised issue).

Correlates of testing frequency

A larger number of tests completed was associated with increased satisfaction with their ability to interact with others in their hall (r = −.180, p = .031, n = 145), although the magnitude of this association is small. Students who were more satisfied with the test drop-off and pick-up processes were more likely than those dissatisfied to report full test compliance during the pilot (X² (1, n = 143) = 4.917, p = .027, effect size (Cramer’s V) = .185). Students who reported higher levels of worry about the risk of getting COVID-19 completed a greater number of tests during the pilot (ie., were more adherent to testing) (r = −.151, p = .043, n = 138). A higher number of tests completed during the pilot was significantly associated with increased positive perceptions towards social distancing (r = .178, p = .033, n = 144), face coverings (r = .227, p = .006, n = 144), and hand washing (r = .165, p = .047, n = 144) as essential controlling measures for COVID-19.

Social behaviours: social distancing, self-isolation, and face coverings

Of respondents, 88.3% (n = 128/145) were somewhat or extremely satisfied with level they were able to interact with other people in their hall of residence during this time. Almost all (97.9%, n = 141) felt that relaxed social distancing in halls was acceptable; many indicated that social contact was happening regardless and better to be sanctioned: ‘we already have unavoidable contact’. Some indicated that the relaxation of social distancing and being able to socialise more freely was a reason for their participation. There was an overwhelming perception that the benefits of social interaction to mental health outweighed the risk of virus transmission which was perceived to be low. Three-quarters (75.5%, n = 108) of students reported always maintaining social distancing (two metres) when interacting with staff, out of ‘courtesy’ and ‘respect’. An estimated 92.4% (n = 133) indicated that they had adhered to social distancing and all other COVID-19 security rules outside the hall environment ‘to protect others’. All (100%) students reported they did not have to self-isolate during the pilot (n = 144). Three-quarters (73.4%, n = 80) of students indicated they were either ‘extremely’ or ‘somewhat satisfied’ with the University’s support offer for students required to self-isolate. Over half (7.3%, n = 82) reported always wearing a face covering in communal areas in their hall. Non-compliance with behavioural regulations (e.g., face coverings, social distancing) was largely due to misunderstanding (of students and some staff) as to what was allowed and where, and frequent changes or inconsistency in messaging. Complacent attitudes and misbehaviour of some students caused frustration among those conforming to the rules.

Student satisfaction, barriers and facilitators

Of respondents, 88.1% (n = 126) would take part in a similar surveillance testing participation scheme in future. Eight out of ten respondents (82.5%, n = 118) would recommend it to their peers. Dissatisfaction in the minority largely stemmed from communication issues (with regards conflicting information, perceived pressure or hostility), and a mismatch between some students’ expectations of the pilot and the reality of delivery (with regards incentives, and social behaviour regulations). Barriers and facilitators to participation in, and delivery of, RB-TPP are mapped to the COM-B framework [36] (Table 4).

Table 4 Barriers and facilitators to participation in and delivery of RB-TPP, mapped to the COM-B Framework

Dose and timeliness

Tests were provided to students each week (2 per week) in line with planned delivery timing (scheduled days/times). Weekly ‘delivered’ dose and timeliness therefore aligned with the pre-determined plan (8 tests in total). However, it was not possible to assess whether overall ‘received’ dose was per protocol due to early termination of data collection around test compliance. Evaluation is therefore based on a reduced dose (3 tests) across a shorter intervention period (10 days) during which uptake and compliance data were collected.

The majority of students were satisfied with test pick-up and drop-off processes (84%, n = 121/144) and the communication around test results (96.6%, n = 140/144). Any dissatisfaction was generally due to a perception of poor communication around changed procedures, or perceived inappropriateness of drop-off timings due to academic commitments (this issue was identified early in the pilot and addressed on receipt of student feedback, with drop-off times extended). If a positive case was identified in a student’s hall during the pilot, all students in hall were required to take an additional test that week (surge testing). The vast majority, 97.1% (n = 136) of students participating in the pilot thought this plan was acceptable, although surge testing did not occur since there were no positive cases identified during the intervention period.

“I thought it was a good thing because it spread so much in the October, November time so I felt like it definitely would have halted it.” [FG5, S3].

Students who were identified as a close contact of a person who tested positive were required to test every day for 7 days. An estimated 96.4% (n = 134) felt that the 7-day contact testing frequency was acceptable: “small price to pay for protecting others from covid.” Students who were identified as a close contact of a person who tested positive did not need to isolate if their tests over the next 7 days were negative, and this was viewed to be acceptable (93.5%, n = 130/142). Only one (.7%) individual reported experiencing this process as they were identified as a close contact of a positive case. A further case (.7%) indicated they preferred ‘not to say’.

Discussion

To our knowledge, this is the first process evaluation of a residence-based SARS-CoV-2 surveillance testing participation intervention in a university campus setting. The RB-TPP had high uptake and fidelity, resulted in a dramatic increase in the proportion of students engaging in surveillance testing (for the detection of SARS-CoV-2 RNA), and played a significant role in students’ mental wellbeing.

Uptake of the RB-TPP was high across both sites (98% participation of in-house students, 79% of listed occupants) with high engagement in testing (88% of programme participants testing, compared with 5% pre-intervention), albeit only half (46% of participants, 52% of testers) were fully compliant with the twice-weekly testing frequency during the data collection period. Based on the increase in testing uptake, staff and students viewed the RB-TPP to be successful despite not meeting the 90% target, and this pre-determined target was perceived by interviewees to be too high. Most of the student participants indicated they would take part again and would recommend the initiative to others. Students found the saliva test itself to be acceptable for regular testing; saliva testing for SARS-CoV-2 has been found to be a useful and acceptable tool for use in a mass screening context [37,38,39]. Although we did not explicitly assess students’ perceptions towards the reliability or safety of asymptomatic testing in this process evaluation, our finding that the tests were highly acceptable concurs with prior evidence showing that university students find saliva testing acceptable, with ease of donation and minimal invasiveness, and they are confident in the results [10, 16, 17].

The RB-TPP was largely implemented as planned. At the time of intervention delivery, there was a national surge in cases of the B.1.617.2 (Delta) variant of SARS-CoV-2. The Delta variant is associated with more severe disease than the previously dominant Alpha (B.1.1.7) variant as determined by twice the relative risk of hospital admission [40]; positive cases had been identified in the region. Due to this, NHS Test and Trace (national public health team) requested early termination of the relaxed social distancing rules in the RB-TPP, with students and staff to revert to adherence to the national behavioural rules applied at that critical time. Testing continued throughout the intervention period, albeit with a reduced data collection period relating to testing adherence in the RB-TPP. Since there were no reactive tests during the intervention period, surge testing and local contact tracing for positive cases were not required and so evaluation of these approaches ‘in action’ was not possible. However, students and staff held positive views towards the planned approaches.

University staff highlighted that using an accredited asymptomatic test would reduce the lag in isolating positive cases, through removal of the need to undertake a confirmatory test. This was subsequently resolved, since the university ATS used here was recommended for accreditation by UKAS (the National Accreditation Body for the UK) in July 2021, accelerating the containment process and removing the burden of confirmatory tests for national testing services. The original (pre-accreditation) process was that a positive PCR test with the ATS would lead to a request (with no legal grounds) for the individual to isolate and take a government Pillar 2 PCR confirmatory test. Therefore, the time from the initial saliva test in the RB-TPP to the result of the confirmatory test could be many days or even a week. After the recommendation for accreditation of the service was in place, the new approach considerably speeded up the process. Going forwards, a positive PCR test result from the ATS resulted in immediate notification of the individual who was then required to isolate by law (as the result was then equivalent to that of a government PCR test). In parallel, Public Health England (PHE) was notified of the positive result in order that further actions could be taken (i.e., identification and notification of close contacts), which happened the day after ATS sample provision.

Students were personally motivated to take part in the RB-TPP by the perceived safety of regular testing, societal responsibility to protect others, and a strong desire to socialise. These factors have previously been identified as important in SARS-CoV-2 testing uptake in higher education settings [16, 17]. Engagement in the programme was primarily facilitated by the positive impact of RB-TPP on students’ mental wellbeing, stemming from a reduced fear of self-isolation (regular testing of close contacts of positive cases, instead of self-isolation), social contact within their accommodation and a perceived return to normal university life. Sanctioning social contact was well received by the vast majority, despite perceptions from some students that the RB-TPP did not meet all their expectations with regards residence-wide social events. Motivation to participate was further enhanced through the involvement of student ambassadors to assist with communications and provide peer-to-peer support. Student ambassadors have been used successfully to raise health-awareness or advocate health screening programmes in educational settings (e.g., [41, 42]: COVID-19 vaccination [43];: COVID-19 communication and mitigation behaviours [44];: HPV vaccination [45];: flu vaccination). Our findings contribute to an emerging evidence-base advocating the role and impact of peer-to-peer student health ambassadors on campus to mitigate the spread of COVID-19 [46]. Views towards the provision of small incentives to maximise engagement in the programme were mixed. Incentives were perceived as either ‘motivating’, or ‘not enough’ (by students), ‘lacking’ or ‘inappropriate’ (by staff).

Adherence to testing was satisfactory. In the current study, conducted in 2021, we found that most students engaged in asymptomatic saliva testing (88%); this is comparable with a prior study conducted at the same institution in 2020, in which 89.2% of first year students completed one or more saliva samples during an intervention period [16]. In the current study, we found that 46% of participants were fully compliant with a pre-determined testing frequency; in 2020 we found that 47.7% of students completed a pre-determined testing protocol (albeit much longer - 12 weeks) [16]. There are few published reports of student ‘uptake’ and ‘compliance’ with SARS-CoV-2 testing, although this level of compliance appears to be higher than that reported in other university settings (albeit with variations in testing protocols). For example, lack of full compliance with a saliva-based SARS-CoV-2 testing protocol on a university campus in the United States of America was observed in 82.3% of participants (with reasons not assessed) [12]. Further, individual adherence was likely to be higher than documented, since students who left the residence during the study period or could not locate a test kit due to logistic issues were not excluded from test uptake figures. Adherence was facilitated by attitudes and views (about COVID-19 and the importance or protective behaviours), satisfaction with the level of social contact, and practical issues (the ease of the saliva test, the efficiency of the testing service and the convenience of testing within accommodation).

Testing adherence was hindered by changes in procedures or logistics, such as timings for sample drop-off (which were subsequently revised), and inconsistency in messages around rules and regulations delivered by different staff groups and student representatives. One barrier to intervention success was complacent attitudes and non-compliance with the rules in a minority, which was challenging for staff and frustrating for adhering students. Complacent attitudes and misbehaviour of some students caused frustration among those conforming to the rules. It was not possible to determine whether (and how) students were mixing with others outside of the residences, or in the general community and so this cannot be ruled out. However, it should be borne in mind that the purpose of the RB-TPP was to provide an asymptomatic testing service which would allow for the identification of people who were asymptomatic (including pre-symptomatic) or who had the symptoms that were not being promoted in national guidance at the time (e.g., headache, sore throat etc). The sensitivity of Quantitative Reverse Transcription PCR (RT-qPCR) meant the aim was for early detection, and hence removal of people testing positive from the circulating population, thereby reducing onward transmission.

Non-compliance with behavioural regulations (e.g., face coverings, social distancing) was largely due to misunderstanding (of students and some staff) about what was allowed and where, and frequent changes or perceived inconsistency in messaging. Although central briefings to students were delivered by email and contained accurate and up-to-date information, the rapidly changing external context of the pandemic meant that communications and updates were very frequent, and written communication was often perceived to be lengthy and complex in nature. Feedback from students indicated that not all had accessed and read the written communications in a timely way. Concurrently, staff in student-facing roles were challenged to continually communicate updates, verbally, in a regularly changing context and among students with varying levels of familiarity with new processes and procedures. Studies of health messaging during previous pandemics (e.g., [47]: H1N1) have identified the challenges of communicating effectively to staff and students about the spread of viruses without inciting unnecessary fear or promoting complacency. Further, communications need to account for known variability in health literacy in student populations [48], and empathy in messaging is critical but often overlooked in a pandemic situation [49].

The prevalence of mental health concerns among students at this point in the pandemic should not be underestimated. Our qualitative findings highlight the widespread belief that relaxed social distancing within the residence was beneficial for students’ mental well-being; some students viewed this social contact as essential for their mental health. Nevertheless, in our survey, all students reported signs of anxiety and almost half our sample (46%) had clinically relevant anxiety levels (moderate to severe score on the GAD-7). The negative impact of the COVID-19 pandemic on university students’ mental well-being is already established [17, 50,51,52,53], is associated with increased social isolation [54] and is likely to have long-term consequences on students’ health and education [55]. This highlights the importance of initiatives that create opportunities for safe social contact during the pandemic. Social interaction was deemed to be exceptionally important for mental health by this student group and there was an overwhelming perception that the benefits of social interaction to mental health outweighed the risk of virus transmission, which they perceived to be low. Therefore, efforts to engage students in COVID-19 mitigation initiatives that provide socialising opportunities and a perceived return to a more ‘normal university experience’ may be more successful than those focusing on testing uptake alone.

Our results will help to inform whether, and how, asymptomatic testing could be implemented in residences at other campus-based university settings as part of COVID-19 outbreak prevention and management approaches in higher education environments. Although the intervention was shorter than planned, findings support the premise that that residence-based high-frequency repeated testing may be an effective strategy for COVID-19 mitigation. The RB-TPP approach was perceived by students and staff to be acceptable, increased perceptions of safety on campus and assisted in normalisation of university life with benefits for mental well-being during an extended pandemic. This process evaluation supports the implementation of such schemes, but future success relies on the necessary infrastructure or funding for implementation, expectation checks with students, and consistency of messaging relating to changes in processes and behavioural expectations. Key findings and recommendations are shown in Table 5.

Table 5 Key findings and recommendations

Study limitations

Log file analysis for identification of multiple survey entries was not used; Internet Protocol (IP) addresses were not available to protect confidentiality. The inclusion of monetary incentive for completion of the survey may have introduced bias into the sample. However, this is unlikely since the financial value of the incentive was very low (£5), and prior studies have not found any significant differences on response completeness between those who received an incentive offer and those who did not [56]. The survey response rate was low, although respondents were broadly representative of the wider pool of students registered as living at the two participating sites. The views of non-participants in the programme were invited but are under-represented.

Reflection on rapid process evaluation approaches

The process evaluation was designed in alignment with a pre-determined logic model, which assumes linear and predictive pathways. However, our findings demonstrate that even over a 4-week period, participants (students, service providers) may adapt (intentionally or unintentionally) as they respond to feedback (e.g., from university students or staff, and local public health teams) and contextual changes (e.g., escalation of the Delta variant, changes in national guidelines). The context of rapidly changing global circumstances required immediate responsiveness in local outbreak prevention and management approaches. Our study went beyond adherence or non-adherence to the implementation plan and recognised the reasons for in-situ changes as they occurred, allowing for process evaluation to feedback into the operationalisation of the service. As such, for a process evaluation conducted alongside the implementation of an intervention within a complex organisational system, in the context of a pandemic we would advocate for a more developmental approach requiring an emergent perspective [57, 58].

Conclusions

To our knowledge, this study is the first mixed-methods process evaluation conducted alongside a university residence-based asymptomatic SARS-CoV-2 testing intervention, during the global COVID-19 pandemic. The RB-TPP intervention for students in university residences increased testing behaviour and improved students’ mental wellbeing. It was adequately delivered, well-received and could be implemented more widely with some modifications to optimise future delivery.