Reintroducing face-to-face support alongside remote support to form a hybrid stop smoking service in England: a formative mixed methods evaluation

Background During the COVID-19 pandemic, United Kingdom (UK) stop smoking services had to shift to remote delivery models due to social distancing regulations, later reintroducing face-to-face provision. The “Living Well Smokefree” service in North Yorkshire County Council adopted a hybrid model offering face-to-face, remote, or a mix of both. This evaluation aimed to assess the hybrid approach’s strengths and weaknesses and explore potential improvements. Methods Conducted from September 2022 to February 2023, the evaluation consisted of three components. First, qualitative interviews involved 11 staff and 16 service users, analysed thematically. Second, quantitative data from the QuitManager system that monitored the numbers and proportions of individuals selecting and successfully completing a 4-week quit via each service option. Third, face-to-face service expenses data was used to estimate the value for money of additional face-to-face provision. The qualitative findings were used to give context to the quantitative data via an “expansion” approach and complementary analysis. Results Overall, a hybrid model was seen to provide convenience and flexible options for support. In the evaluation, 733 individuals accessed the service, with 91.3% selecting remote support, 6.1% face-to-face, and 2.6% mixed provision. Remote support was valued by service users and staff for promoting openness, privacy, and reducing stigma, and was noted as removing access barriers and improving service availability. However, the absence of carbon monoxide monitoring in remote support raised accountability concerns. The trade-off in “quantity vs. quality” of quits was debated, as remote support reached more users but produced fewer carbon monoxide-validated quits. Primarily offering remote support could lead to substantial workloads, as staff often extend their roles to include social/mental health support, which was sometimes emotionally challenging. Offering service users a choice of support options was considered more important than the “cost-per-quit”. Improved dissemination of information to support service users in understanding their options for support was suggested. Conclusions The hybrid approach allows smoking cessation services to evaluate which groups benefit from remote, face-to-face, or mixed options and allocate resources accordingly. Providing choice, flexible provision, non-judgmental support, and clear information about available options could improve engagement and match support to individual needs, enhancing outcomes. Supplementary Information The online version contains supplementary material available at 10.1186/s12889-024-18235-0.


Background to the PHIRST scheme & LWSF study
This evaluation initiative is funded by the National Institute for Health and Care Research (NIHR) and undertaken by the Public Health Intervention Responsive Studies Team (PHIRST) Fusion team.The PHIRST Fusion team's approach to evaluation follows a 5-step process: brokerage, work allocation, research, reporting & knowledge mobilisation, and continuous improvement, which includes evaluability assessment methodology and embedded research with local government practitioners.The Living Well Smoke Free Project (LWSF) was allocated to PHIRST Fusion in August 2021.This report describes an Evaluability Assessment of LWSF conducted between December 2021 and May 2022.
LWSF is an intervention offered in North Yorkshire since October 2021.It is a hybrid service programme comprising online and face-to-face elements.LWSF aims to offer a more flexible service for service users resulting in higher quit numbers/rates and improved rural access; ability to resume face-to-face interventions and CO monitoring with a focus on priority groups and key points of quit journeys; a more cost-effective service model; and a flexible way of working.

Programme context
Community stop smoking services are at the forefront of efforts to reduce premature death and disability due to smoking in the UK (ASH, 2022).'Living Well Smokefree' (LWSF) is North Yorkshire County Council's programme to tackle smoking in the region.There are approximately 70,000 smokers within North Yorkshire, with higher rates being found in areas of greater deprivation, and worryingly, above national average rates of smoking in priority risk groups (such as during pregnancy and in populations with mental health conditions) (North Yorkshire tobacco control strategy, 2015).Working with people over the age of 12, the LWSF service offers one-to-one support over a 6-12 week period (one session per week) in an effort to achieve a Russell Standard quit (West, 2005).Support is provided to help service users set a quit date, and to manage cravings/withdrawal symptoms.Support includes pharmacotherapy (nicotine replacement therapy (NRT) medication vouchers provided by prescription), CO (carbon monoxide) monitoring, and every client has a named key worker (smoking cessation adviser) who employs Motivational Interviewing principles and techniques to increase self-efficacy to reduce smoking.

Impact of Covid-19
Due to the COVID-19 pandemic, to continue to support individuals wishing to quit smoking, the North Yorkshire Living Well Smokefree (LWSF) service moved to remote delivery with support being delivered in virtual formats.LWSF have seen remote delivery as having benefits (increased accessibility for rural communities), as well as challenges (difficulty in achieving/recording CO validated quits).Therefore, as COVID-19 restrictions have eased LWSF has developed a hybrid approach, combining elements of face-to-face and virtual delivery to better target different priority groups.This hybrid approach included three service delivery methods (face-to-face only, online-only, and a blend of face-to-face and online).
This study aims to explore the implementation of this hybrid smoking cessation model, and to assess the effectiveness and acceptability of the different delivery methods, in order to inform future service delivery

Rapid Review
We undertook a rapid literature scope to explore feasibility studies of smoking cessation services (including, blended/hybrid/flexible/tailored services, and online/phone/face-toface approaches).Data was initially searched through google scholar.Initial search terms around 'smoking cessation services', 'feasibility' and 'delivery models' (e.g., blended/hybrid/flexible/tailored) were used.Relevant data was extracted from the returned results: • 'Blended' = usually a specific intervention when some specific content is delivered face to face and other specific content is delivered online • 'Hybrid' = the same service can be delivered in different formats/locations • 'Flexible' = where the client has a choice of intervention delivery mode • 'Tailored' = any service designed for and delivered to a specific patient group Service user perspectives Generally, work exploring service users' perceptions of remote smoking cessation provision found positive perspectives, experiences and high levels of acceptability, as well as preferences for engagement in remote provision modalities in the future (Mahoney et al., 2021).There were comparable and high levels of acceptance for both voice and video call approaches from service users (Kim et al., 2016;Richter et al., 2015).Kim et al. (2018) reported a slightly greater engagement in treatment (and higher maintenance of smoking abstinence) from video-call interventions over voice-call interventions.They also found similar, and high, levels of acceptability from participants between voice and video call approaches, but issues in the feasibility of video calls were noted (e.g., older and more deprived populations did not have technological access or knowledge for this approach).For some groups, remote provision may increase access to support, and thus be a feasible option (e.g., engaging rural communities (Byaruhanga et al., 2020;Carlson et al., 2012)), helping overcome issues such as travel time and costs for face-to-face support (Tall et al., 2015;Byaruhanga et al., 2020).Indeed, remote provision was perceived as acceptable by rural communities (Carlson et al., 2012), with video call models being a feasible option for smoking cessation services for rural populations (Byaruhanga et al., 2020).However, remote provision may not be feasible for all types of support and may not be acceptable to all service user groups (Appleton et al., 2021).Remote provision may be less suited to/effective with populations with more complex health conditions and exacerbating health issues (e.g., pregnant women) and comorbidities (substance use and mental health issues) (Siemer et al., 2020;Appleton et al., 2021).Indeed, Ashton et al. (2015) reported much lower cessation rates for people with mental health issues who received phone support only vs face-to-face and phone support.However, if provision is appropriate and designed for specific populations, remote provision can be acceptable, feasible and useful (e.g., Joyce et al., 2021;Bennet et al., 2016;Ashton et al., 2015).For example, Joyce et al. (2021) reported high levels of feasibility and acceptability of several components of a 'technology-supported, incentive-based smoking cessation program' for pregnant women, including the feasibility and strong acceptability of video chat observed cotinine testing.The importance of flexible approaches to engage different populations was evident (e.g., those in deprived areas (Ormston et al., 2015), pregnant women (Bennett et al., 2014;Joyce et al., 2021), people with severe mental illness (Ashton et al., 2015)).More important than a specific population, maybe the need for, where possible, a flexible and patient-centred approach, where the modality of care is designed, chosen and delivered in conjunction with service users (Vera San Juan et al., 2020;Ritchie et al., 2007).Having a mix of approaches to draw on may be beneficial in engaging and providing support for priority groups (Mahoney et al., 2021;Appleton et al., 2021).Blended approaches have been noted to have a generally positive experience (Siemer et al. 2020).
-Benefits of remote provision for participants included convenience (time/cost/effort), comfort and safety (Mahoney et al., 2021;Vera San Juan et al., 2021;Vinci et al., 2022).Video-call over voice-calls were noted as being more personal (Mahoney et al., 2021).Remote provision was seen as potentially beneficial for more 'functional' appointments (e.g., check-ins (Vera San Juan et al., 2020;Appleton et al., 2021)).-Challenges of remote provision (compared to in-person support) for participants included reduced interaction/relationship building, issues with technology, and less accountability to complete work (Mahoney et al., 2021).The service provider's ability and level of comfort around the use of technology/remote provision was important in service user experience and perceptions of accessibility (Vera San Juan et al., 2021).The ability of providers to troubleshoot support issues, and clear guides around how to access/engage with remote sessions was noted as important (Vera San Juan et al., 2021).

Service provider perspectives
There was little work looking at provider/practitioner perspectives, but the existing work, combined with the qualitative work exploring service users' perspectives, highlights overall general positive perspectives of remote provision (Bennett et al., 2016;Appleton et al., 2021;Kim et al., 2018).
For smoking cessation staff, important trade-offs between remote and face-to-face approaches (Appleton et al., 2021) were noted.Whilst remote provision was described as more flexible, more efficient and convenient (Vera San Juan et al., 2020;Mahoney et al., 2021), and being more 'cost effective' (in terms of staff time), it was also seen as impacting upon rapport building, reducing the development of therapeutic relationships (Appleton et al., 2021;Vera San Juan et al., 2020;Mahoney et al., 2021).There were contradictory findings around remote provision reducing engagement levels, as well as reducing the number of missed appointments, suggesting greater convenience may be improving accessibility (Appleton et al., 2021).Thus, this may be dependent upon different populations/individuals.
Within remote approaches, video calls were seen to offer more 'personal' and 'interactive' experiences than voice calls (Mahoney et al., 2021), resulting in higher rates of service user engagement/retention (Kim et al., 2018).Video calls, compared to voice calls, were also seen to enable service providers to identify and respond to (verbal and non-verbal) cues.
Potential learning -Remote smoking cessation provision may provide an alternative acceptable and feasible treatment delivery modality, which has benefits in terms of access, costs and convenience for both service providers and service users.However, there are potential issues/challenges for both service users and providers.-It appears that cessation rates are higher for those with higher rates of service engagement (e.g., attending more sessions (Ashton et al., 2015;Ormston et al., 2015;Zwar et al., 2015)).Therefore, being able to offer a range of approaches to best suit and engage service users appears crucial.-Modality of care should be discussed on an individual basis, to find the best fit for service users' preferences and circumstances.This requires flexible and personalised hybrid models of care.

Evaluability Assessment process
We used evaluability assessment methods to develop the evaluation design (Leviton, Khan et al. 2010, Craig andCampbell 2015).Evaluability assessment (EA) is a rapid, systematic, and collaborative way of deciding whether and how a programme or policy can be evaluated, and at what potential cost.We conducted three workshops with LWSF stakeholders to ascertain their understanding of how the LWSF hybrid model will deliver a better service for people wishing to quit smoking.
Workshop participants included people with lived experience of using employment services, local government, NHS and voluntary sector stakeholders, and observers from the Department of Work and Pensions and the Scottish Government.We allowed the workshop format to evolve to take account of feedback from preceding workshops, and to enable stakeholders to shape the approach to evaluation.The second EA workshop focused on getting the Theory of Change signed off (Figure 1), as well as considering evaluation questions and data issues (like sources and collection methods) in relation to the evaluation design.This was considered from both the service user as well as the service provider perspectives.Delivery and Service User-level outcomes were developed.

Workshop 3
The final EA workshop discussed a recalibration of the evaluation.There was now agreement that a feasibility study was preferred in the light of ongoing uncertainties around the delivery of LWSF.With a change in focus, stakeholders agreed evaluation questions should be framed around different aspects of the feasibility of the LWSF hybrid model.

Theory of Change
The LWSF Theory of Change (ToC) was co-developed with workshops participants and signed off in workshop 2. See Figure 1 below.

Figure 1 Living Well Smokefree Theory of Change 4 Evaluation design considerations
At the end of the EA workshops, LWSF stakeholders, together with PHIRST team members reached an understanding that the LWSF evaluation is a formative feasibility study.This is in line with current MRC guidance advocating the need to better understand the feasibility of the intervention design around optimal and acceptable intervention content and delivery; adherence to the intervention; likelihood of cost-effectiveness; and capacity of those providing the intervention to deliver as planned (Skivington, Matthews et al. 2021).

Aim & evaluation questions
The overall aim of the North Yorkshire County Council LWSF evaluation is to assess the feasibility of its implementation.The evaluation questions are: 1. What is the acceptability of LWSF to participants (non-target groups)?
• What is the drop-off/LTFU (Lost To Follow Up)?
• What is the reach and uptake?
• What are the facilitators and barriers to participating in LWSF?
• What are the experiences & perspectives of participants regarding face-toface/blended/remote smoking cessation provision approaches? 2. What is the acceptability of LWSF to the target groups?
• Which groups benefit the most/least?
• What is the drop-off/LTFU (Lost To Follow Up)?
• What is the reach and uptake within the target groups?
• What are the facilitators and barriers to participating in LWSF?
• What are the experiences & perspectives of participants regarding face-toface/blended/remote smoking cessation provision approaches? 3. What is the acceptability of LWSF to delivery staff?
• What are the facilitators and barriers to delivering LWSF?
• What are the experiences & perspectives of service providers regarding faceto-face/blended/remote smoking cessation provision approaches?• Who are seen as key priority groups?4. To what extent does existing monitoring & evaluation support understanding of how LWSF would reach and be effective for target groups?
• What is the rationale/ underpinning assumptions behind the application of different modalities for different groups?• How is service provided/offered to service users 5. What can be learnt at mid-point (formative design)?
• What are the strengths/weaknesses of LWSF for each pathway?
• How can LWSF be adapted and improved?

Data
Data is currently collected on the QuitManager system and meets the reporting requirements of NHS Digital.To inform the LWSF KPIs, further investigation of service outputs and quitting outcomes is done by exporting data to Excel and cross-tabulating by client characteristics, source of referral, pharmacotherapy use etc.The LWSF service has an online dashboard that presents this information graphically and tabularly based on an established service performance framework.

Evaluation options
Four options are presented in this chapter for starting with the cheapest and most basic, and building on additional evaluation activity.See Table 1 for an overview: • Option 0 involves no change or additional evaluation approach and relies on relevant and existing Monitoring & Evaluation (M&E) systems.We include Option 0 as a reference option.As discussed in the workshops, existing M&E systems do not address meet LWSF evaluation requirements.• Option 1 supplements Option 0 with a summative process evaluation.A mixed method summative process evaluation of intervention and data collection system can provide end of evaluation findings for programme learning & adaptation.• Option 2 is similar to Option 1 but is a formative evaluation approach with one check-in time point around the mid-point of the evaluation to provide timely data for programme learning & adaptation.• Option 3 is similar to Option 2 but has more (quarterly) check-in time points to maximise opportunities for programme learning & adaptation.The rest of this chapter describes each option in detail.
Option 0: Existing M&E This is the reference option where no additional evaluation is done.This option instead relies on routinely collected data for NHS Digital requirements but will not support the aims of the LWSF evaluation which is met by the other options.We include Option 0 as a reference option.As discussed in the workshops, existing M&E systems do not address meet LWSF evaluation requirements.
Option 1: Summative Process Evaluation The quantitative and qualitative data are designed to inform how to increase participation rates in the LWSF service and how to refine the service to increase the likelihood of quitting success.Both types of data collection will therefore have an emphasis on understanding variation in service use, experiences and outcomes among priority client groups.The qualitative data collection will go further to understand staff perspectives on the new hybrid service offering.The perspectives of staff will help to understand the facilitators and barriers to staff offering an efficient service to different client groups.

Quantitative investigation
The overall aim of quantitative data collection and use in the evaluation is to describe the demand for the different modes of contact offered.Descriptive statistics will be produced that show how demand and quitting outcomes differs among routes of referral and priority client groups.

Existing data collection and uses
Data is currently collected on the QuitManager system and meets the reporting requirements of NHS Digital.To inform the LWSF KPIs, further investigation of service outputs and quitting outcomes is done by exporting data to Excel and cross-tabulating by client characteristics, source of referral, pharmacotherapy use etc.The LWSF service has an online dashboard that presents this information graphically and tabularly based on an established service performance framework.

Setting
The data collected refers to clients seen in the community setting only (excluding clients seen in pharmacy or GP settings).How does the hybrid model of LWSF fit alongside the options for getting support to stop smoking from the GP or pharmacy -do these also offer in-person and phone options?

Addition of a new mode of contact field
Aims: To assign clients to one of three categories that then be used to stratify the existing tables of service use and outcomes.These reflect the three pathways defined in the logic models during the EA workshops.
Pathway 1: Clients who chose a hybrid option to begin with, or clients who chose to switch to a hybrid option later by starting with phone consultations only and then requesting an inperson consultation, or vice versa.Clients would be provided with an electronic medication voucher.Clients would then have a flexible mix of in person and phone consultations with smoking advisors.If CO monitoring is introduced into the service, then the timing of in-person consultations could be planned to fit with the key points for CO monitoring.If a client is on a phone only or in-person only option and some unplanned life event means that they need to temporarily switch to the other mode of contact, will they be classed as having a hybrid service?
Pathway 2: Clients who chose in-person consultations to begin with and then stayed with in-person consultations only.Clients would be provided with an electronic medication voucher.
Pathway 3: Clients who chose phone consultations to begin with and then stayed with phone consultations only.Clients would be provided with an electronic medication voucher.
If a client is on a phone only option and they come for an in-person consultation to have a CO check, will they be classed as having a hybrid service?Will the service move to more single use CO monitors for remote clients to allow them to do their own CO monitoring?
One suggestion is that a client should only be recorded as having switched to a hybrid mode of contact if they formally choose the hybrid option as part of a conversation with their advisor, e.g., on the basis of the hybrid option being a better option for their smoking cessation support.

Item Description
Options / decisions / questions Mode of contact with the service Mode of contact -there is a preexisting field in QuitManager ("service provided") that records whether each client contact was faceto-face or remote.This field will not be used for service evaluation.Instead, a new field will be added with staff training on how to use it with the new service.
Proposed edits to QuitManager to create a new way of recording mode of contact: At the first consultation with the client [on referral], the client will be asked to choose one of three options as their preference: Phone only, In person only, hybrid.On subsequent consultations, a client who has previously chosen either phone or in-person only might then choose the hybrid option.Their original choice will remain in the system and the 'hybrid' option will also be ticked.
The LWSF team are currently exploring the cost and feasibility of two alternative options of implementing this with the QUITmanager team.

Numbers and percentages Same as existing reporting
Use existing specifications within the QuitManager system Expect that the client's preferred mode of contact under the new service is not yet known at the point in the service pathway where these data fields are recorded These data fields would therefore not be stratified by mode of contact Important to understand whether the 'hybrid model' improves the numbers of people participating in the service i.e. not just referral numbers but the number of people who go on to access the service following triage (at present a large proportion of referred clients do not go on to start their programme).Does the new hybrid offer improve the number of referrals that go on to start their quit journey?
Is the client's preference for mode of contact (phone, in person, hybrid) expressed at the point of referral and before the 1 st appointment?Number of clients had 1 st appointment The first consultation with a tobacco treatment advisor.This could be by phone or in person.
Split by the client's first choice mode of contact (phone, in person, hybrid) Expect that the mode of contact of the first appointment will be the same as the client's first choice preference.
If the client chooses a hybrid preference, then the data collected for the evaluation would not be able to say whether this 1 st appointment was in person or by phone Provision of electronic medication vouchers to service users Identified from EA workshop logic model What are options for getting data on this from the QuitManager system?Need to know relative use of different forms of pharmacotherapy / ecigarettes between pathway options?-potentially important for calculating cost per quit differences between pathways if the option to do so is chosen

Number of quit dates set
Indicator that a client is making an attempt to quit smoking

Qualitative investigation
The overall aim of the qualitative data collection is to understand staff and service users' perspectives on the service's offering of different modes of contact.This will provide data to inform refinements to how the service is implemented.In particular, the service is interested in how a hybrid mode of contact might enable them to tailor the service offering to priority groups, e.g., to achieve a better reach and better outcomes.

Timing
Service setup: The main activities will be focus groups with staff.This will further develop a theory of change for the new service.Staff perspectives on the service will be anonymised and summarised thematically to be presented back to service staff to support reflection service implementation so far.
Service implementation: This will involve further focus groups with staff to understand how perspectives change through the service implementation.It will also involve interviews with service users to understand their perspectives on the different modes of contact offered and how that affected their experience of the service.

Link between quantitative and qualitative data collection
Qualitative data collection will then aim to understand more details about client preferences for different modes of contact can be focused on sampling patients from each of the three categories of client above.Particular effort in qualitative data collection could be put into understanding the views of patients in the 'hybrid' category.This category will be a heterogenous mix of clients who chose hybrid initially, switched to hybrid and who had different mixes of phone and in-patient consultations.

Potential use of questionnaires
There is a pre-existing feedback form that is given to clients when they complete the service.However, few forms are returned.Contact with clients is made mainly by email.The service is considering a phone text option, and a financial incentive (voucher) to complete the feedback.The service is interested in using the evaluation process to understand how to their method of gaining feedback.The qualitative interviews with clients could look to understand the facilitators and barriers to clients providing feedback in this way.

Potential questions:
-Would you recommend this service to your friend/family member/colleague?-What are key factors that helped you quit? -If you participate again, would you choose the same mode of contact?
Everyone has a final meeting with an advisor so might be easy to collect questionnaire data during that meeting.
Questionnaires might be given to people when they enter the service too -to help understand the characteristics of patients lost to follow-up.
A questionnaire could be an easy way for the evaluation to collect data from a wider range of staff and service users.
Analyse surveys quantitatively to assess a change in key responses from baseline?Option 2: Formative Process Evaluation (Mid) Option 2 is similar to Option 1 but with the inclusion of an interim report/presentation with LWSF project partners where emerging findings are discussed and initial learning shared to help shape continued development of the service.
Option 3: Formative Process Evaluation (Quarterly) Option 3 is similar to Option 2 but for the number of interim reporting to be increased to every three months.

Recommendation
Option 2 is the preferred option because it will help to define and then refine the service as the evaluation progresses.Option 2 would provide enough time for substantial data collection before the interim workshop (about 6 months in), and between the interim workshop and the evaluation final report.
Time period of data to be used for the evaluation Start: When the new QuitManager fields that describe the new hybrid service are in operation (possibly with some time for the service to 'bed in').Possibly use data from a period (3 months?) before the new service was implemented to get an impression of the impact of the new hybrid offer on overall service use?Interim stop: In enough time to process the data for an interim report that would feed into an interim workshop with the service team to reflect on service implementation so far.

Final stop:
In enough time to produce the final evaluation report.Anticipate a 12 to 18 month evaluation.

Development and qualitative description of the theory of change
The theory of change developed during the EA workshops will be further developed.Focus groups and interviews with staff and service users will add a thematic understanding of the key factors affecting service reach to priority groups, staff experience, client experience and quitting outcomes.

Statistical descriptions of service outputs and outcomes
Descriptive summary statistics will be produced that show: • Numbers and percentages of clients choosing each pathway option at their first consultation • Numbers and percentages of clients who subsequently switch from phone only or inperson only to hybrid • How these numbers are split by priority client groups Reflective meeting on evaluation findings together with the service It is important to fit the outputs of the evaluation to the key time points for service commissioning and planning.It is recommended to have a workshop half way through the evaluation into which the evaluation could feed outputs to support reflection on service implementation so far by the service team.The outcomes of this workshop might be plans for refinement of the service implementation.

Modelling and the calculation of cost per quit
To calculate a monthly cost per 4-week quit of the service, we will estimate the monthly service cost (with and without including the sunk costs) and divide this by the monthly number of 4-week quits recorded by the service.This method is designed to be based on the method already used to compute the cost per quit of the service.We will not attempt to compute the cost per quit separately for each of the three pathways of the new hybrid service.

Public involvement and engagement
Involving past clients in advising on how to best collect data from current clients Planning for the qualitative data collection from clients would benefit from advice from at least one past client on the best way to approach clients.This advice could be especially important for understanding how to best approach clients from certain priority groups e.g., with long term health conditions.The evaluation and service teams should discuss this further.

Involvement and engagement of LWSF staff in the evaluation
Service implementation and data recording LWSF staff will need to be engaged from the beginning in understanding the new setup of the service and how to use the new features of the QUITManager recording system.LWSF staff will also need training to understand when a client should be marked as having had the hybrid service vs. another reason why a client might temporarily have a different mode of contact.There will need to be a consistent approach to triaging new clients (different advisors may lean towards promoting a specific pathway depending on their own views).Service staff need a clear briefing as to what the new service is and how to administer it and record service activity on QUITmanager.This training should happen at the start of the new service -in the subsequent qualitative data collection among staff, the usefulness of this training might be referred back to in relation to the realities of delivering the service.
Readiness to reflect on and adapt service implementation LWSF staff engagement is particularly important for Option 2. The success of option 2 relies on LWSF staff being engaged in the service and being ready to reflect on findings and having the capacity to adapt.
Supporting the qualitative investigation LWSF staff understanding of and involvement in the evaluation will be essential to help encourage them to participate in the evaluation by giving their own views in focus groups, and to gain their support in accessing other people in the organisation to participate in the staff-focused qualitative investigation.Engagement of smoking cessation advisors will further help to recruit service users to participate in interviews for the evaluation.Ideally, the advisors would know enough to be able to explain to clients why their involvement is important.This could be done by holding a meeting with staff at the start of service implementation.
Key themes are likely to be: -Data systems setup -Staff training -Client experience and journey through the new hybrid model -Issues encountered by staff -Resource implications of changing the service pathway

-
Whilst rapport has been noted as being possible to develop using remote means only(Cruvinel et al., 2019), providers' skills and confidence in delivering remote support is important(Vera San Juan et al., 2020).
(Mahoney et al., 2021;Vera SAppleton et al., 2021) enhanced by the development of rapport through an initial (i.e., the first session) or ongoing face-to-face meetings(Mahoney et al., 2021;Vera San Juan et al., 2021).-Remoteapproachesmaybe a beneficial way to deliver some kinds of support (e.g., brief interventions and check-ins(Vera San Juan et al., 2020;Appleton et al., 2021)), with more complex support being provided in-person.

Table 1
Summary of workshop dates & agendaThe In particular, it was observed that LWSF has done well to adapt in this difficult time.There is an urgent need to try and understand the (changed) needs of service users and follow evidence best as we can.It was also noted that the service monitoring guidance is out of date (last updated in 2014).The hope is this EA process can help to update the guidance first LWSF EA workshop aimed to provide the evaluation team with an overview of the LWSF programme and aim, how LWSF was expected to bring about change, and what success would look like.A Theory of Change approach was used to clarify the intervention aims and desired outcomes.LWSF stakeholders shared their experiences of delivering a LWSF hybrid model during the Covid-19 lockdowns.Main themes discussed include:

Table 2
Overview of LWSF evaluation options

Table 4
Qualitative data to be collected