- Study protocol
- Open Access
A pragmatic randomised controlled trial of the Welsh National Exercise Referral Scheme: protocol for trial and integrated economic and process evaluation
BMC Public Health volume 10, Article number: 352 (2010)
The benefits to health of a physically active lifestyle are well established and there is evidence that a sedentary lifestyle plays a significant role in the onset and progression of chronic disease. Despite a recognised need for effective public health interventions encouraging sedentary people with a medical condition to become more active, there are few rigorous evaluations of their effectiveness. Following NICE guidance, the Welsh national exercise referral scheme was implemented within the context of a pragmatic randomised controlled trial.
The randomised controlled trial, with nested economic and process evaluations, recruited 2,104 inactive men and women aged 16+ with coronary heart disease (CHD) risk factors and/or mild to moderate depression, anxiety or stress. Participants were recruited from 12 local health boards in Wales and referred directly by health professionals working in a range of health care settings. Consenting participants were randomised to either a 16 week tailored exercise programme run by qualified exercise professionals at community sports centres (intervention), or received an information booklet on physical activity (control). A range of validated measures assessing physical activity, mental health, psycho-social processes and health economics were administered at 6 and 12 months, with the primary 12 month outcome measure being 7 day Physical Activity Recall. The process evaluation explored factors determining the effectiveness or otherwise of the scheme, whilst the economic evaluation determined the relative cost-effectiveness of the scheme in terms of public spending.
Evaluation of such a large scale national public health intervention presents methodological challenges in terms of trial design and implementation. This study was facilitated by early collaboration with social research and policy colleagues to develop a rigorous design which included an innovative approach to patient referral and trial recruitment, a comprehensive process evaluation examining intervention delivery and an integrated economic evaluation. This will allow a unique insight into the feasibility, effectiveness and cost effectiveness of a national exercise referral scheme for participants with CHD risk factors or mild to moderate anxiety, depression, or stress and provides a potential model for future policy evaluations.
Current Controlled Trials ISRCTN47680448
It is widely recognised that regular physical activity is beneficial to both physical and mental health ; it is associated with reduced risk of chronic diseases including coronary heart disease (CHD)  and improved mental health [3, 4]. Despite recommendations that adults should undertake 30 minutes of moderate intensity exercise at least five times per week, only 30 per cent of adults in Wales are active to this level . Whilst a number of population-based approaches to promoting physical activity have been identified, exercise referral schemes (ERS) represent a more targeted approach for specific patient or population subgroups . This provides an opportunity for patients to have direct contact with and receive advice from qualified health and exercise professionals and to access a range of tailored activities to support and enable lifestyle changes . However, despite the growing numbers of ERS around the UK over the past decade, the evidence base for their effectiveness is weak [6, 8].
Previous studies of ERS have evaluated interventions of variable content and intensity, recruited different samples with varying inclusion and exclusion criteria, utilised different outcome measures and adopted variable follow up points. A systematic review of ERS identified six randomised controlled trials (RCTs,) from the UK . Three trials compared 10-12 week long courses of gym-based exercise with exercise information leaflets [10–12]; exercise classes in community halls with no intervention ; walking scheme with exercise advice ; and gym-based exercise with a walking scheme or exercise advice . Five of these reported on the proportion of participants who were moderately active at the end of the intervention and there was a modest but statistically significant improvement in activity with a combined risk ratio of 1.2 (95% CI = 1.06 to 1.35). The pooled number needed to treat (NNT) indicated that 17 sedentary people need to be treated for one to become moderately active . This modest effect was partly explained by poor rates of uptake and adherence to the schemes and a lack of intervention relapse prevention strategies. Despite only modest improvement in activity levels, one RCT found a relative improvement in depression in the exercise group  and another found improvement in health-related quality of life .
This review was consistent with previous systematic reviews [16, 17] in finding that ERS promote exercise in the short-term, but only in certain populations, that more intensive and longer interventions appear to be more effective; and that they may be more effective in improving mental health or quality of life than in changing long-term exercise habits. Only one RCT of an exercise referral scheme has incorporated a thorough health economic evaluation, which concluded that the exercise intervention was more costly but only slightly more effective than advice alone and was unlikely to be cost-effective . However, there was a large degree of cross-arm contamination. Indeed, methodological concerns over the rigour of many of the above studies have led the National Institute for Health and Clinical Excellence (NICE) to conclude that 'Further research, using a controlled research design, is required to determine the impact that exercise referral scheme may have on reducing health inequalities and their effectiveness in increasing physical activity levels in adult populations" .
The Welsh National Exercise Referral Scheme
In Wales, in the early half of the last decade, most local health board (LHB) areas operated ERS, each following different protocols . The development of high quality schemes was identified as a key action area in a number of Welsh Assembly Government documents [19, 20] and in 2006, existing good practice was assessed and standardised in Wales-wide protocols . The National Exercise Referral Scheme (NERS) subsequently replaced local schemes and was rolled out in three phases from 2007. Supported by the Welsh Assembly Government working in partnership with Local Authorities, Public Health Wales (formerly the National Public Health Service) and LHBs, the NERS provided funds for a dedicated Exercise Co-ordinator (EC) and a number Exercise Professionals (EP) in each local health board area.
NERS consists of a series of motivational interviewing (MI)  consultations with an EP based in a community sports centre and access to a tailored, subsidised 16 week activity programme. To be eligible for NERS, participants must be sedentary (defined as not moderately active for 3 or more times per week or deconditioned through age or inactivity), and have at least one medical condition, covering CHD risk factors, mental health, musculoskeletal, respiratory/pulmonary and neurological conditions (see Table 1 for scheme inclusion criteria). The primary aim of NERS is for participants to achieve 30 minutes of moderate physical activity on at least 5 days per week. Common features of the scheme are detailed below.
Delivery of the Welsh national exercise referral scheme:
16 week programme of exercise supervised by a qualified EP
Initial face to face consultation with EP on entry - lifestyle questionnaire, health check (resting heart rate, blood pressure, BMI, and waist circumference), introduction to facilities, MI and goal setting
Access to one to one exercise instruction and/or group exercise classes
Discounted rate for exercise activities £1 per session.
Four week telephone consultation with EP - review of goals and MI
Sixteen week face to face consultation with EP - review of goals, MI, health check, lifestyle questionnaire, service evaluation questionnaire  and signposted to exit routes
Post 16 week activities
Range and cost of exit routes dependent on area
8 months contact by phone to check progress
12 months face to face review including Chester fitness step test .
Consultations occur at entry, 4 weeks (by phone) and 16 weeks. Following this, participants are contacted by telephone at 8 months to monitor progress and at 12 months they are invited to attend a review session. Routine programme monitoring systems are maintained by EPs and capture the dates of and records from initial, 4 and 16 week and 8 and 12 month consultations.
The Welsh Assembly Government commissioned an independent evaluation of the scheme as it was implemented in 13 of the 22 LHBs in Wales during phase one. This evaluation was to focus on the effectiveness of the scheme among two priority patient groups, those referred for mental health reasons (anxiety and depression) or for CHD risk factors This paper reports upon the design of the national evaluation, which utilised a randomised controlled trial design with nested process and economic evaluations, and reflects on issues in the development and implementation of the evaluation design for current challenges in policy evaluation [25, 26]. Ethical approval for the study was obtained from the Thames Valley Multi-centre Research Ethics Committee (Ref: 06/MRE12/85). Approval from medical directors within each LHB was also obtained.
Methods and Design
The evaluation study comprised three key components. In order to evaluate the overall effectiveness of the intervention, a randomised controlled design was employed. Mixed methodology RCTs have the potential to provide unbiased estimates of the effectiveness of interventions as well as identifying contextual influences on intervention delivery . A nested process evaluation therefore examined how the initiative was implemented, gained a more in-depth understanding of the views of providers and users, and facilitated interpretation of outcome effects. A nested economic evaluation was also undertaken in the form of a cost utility analysis which measured costs and benefits from a public sector multi-agency perspective, with results expressed as an incremental cost per quality adjusted life year (QALY) gained. In this way, the evaluation not only addresses the question, 'Does it work?', but also considers 'What works?', 'For whom?' 'Under what circumstances?' and 'At what cost?'. A summary of the study design is presented in Figure 1.
Participants were recruited using opportunistic referral by a range of health professionals. NERS coordinators in each area undertook briefings with primary care teams to explain the trial and to provide referral forms and trial information materials. Health professionals who chose to participate in trial recruitment identified suitable sedentary patients who met scheme inclusion criteria (see Table 1) as part of their normal clinical practice. Patients were provided with basic trial information and a referral form was forwarded to the evaluation team, who assessed trial eligibility criteria (see Table 1). For patients referred for mental health and/or CHD reasons, NERS was only available to those who were willing to participate in the trial. Patients referred for reasons other than mental health and/or CHD were able to access the scheme without participating in the trial and their referral from were forwarded to ECs for processing.
Referred patients who were eligible for the trial were sent full informed consent materials and a brief baseline questionnaire for them to return by post. Recruitment If patients did not want to participate in the trial, they could gain access to the scheme by joining a 12 month waiting list.
Those consenting to participate in the trial were randomly assigned to an intervention (exercise referral scheme) or control trial arm using a random number generator, with gender and LHB as stratification variables. Randomisation of forwarded referral forms occurred every 2 weeks, with treatment allocation blind and remote from participants and practitioners. The control group received usual health care and a leaflet highlighting the benefits of exercise and including a website address listing local leisure facilities. Control participants were offered priority access to NERS at 12 months.
Pre randomisation baseline measures
At baseline, participants completed a short postal questionnaire assessing age, gender, marital status, ethnicity, employment status, education, car ownership, number in household, willingness to pay for the scheme and visits to the GP in last 6 months. It also included the General Practice Physical Activity Questionnaire (GPPAQ) , which provides a patient activity index which indicates whether the respondent is inactive, moderately inactive, moderately active, or active. Participant post codes were also used to obtain the Welsh Index of Multiple Deprivation  of the lower layer super output area in which they lived, with higher scores representing greater deprivation.
Outcomes at 12 months
- Minutes of weekly activity
The primary outcome is total minutes of weekly activity at 12 months, using the seven day physical activity recall questionnaire (7-DAY PAR) . This interview-based measure has been validated in both community and experimental studies  and utilised in several randomised controlled trials to evaluate the effectiveness of physical activity promotion strategies [32, 33], including several high-quality exercise referral scheme evaluations [10–12, 14]. The 7-DAY PAR was administered by telephone interview. For those respondents who were contacted by telephone but refused to complete the time-consuming 7-DAY PAR telephone questionnaire, the GPPAQ was administered to give an indication of activity level at 12 month follow up.
The Baecke Questionnaire of Habitual Physical Activity (Baecke)  was designed for the measurement of habitual physical activity in epidemiological studies  and provides work, leisure, sport, and total activity indices. Each score can vary from 1-5, and the total score from 3-15, where a higher number indicates more physical activity. Baecke was included in the 12-month postal questionnaire.
- Anxiety and depression
The Hospital Anxiety and Depression Scale (HADS)  was used to assess depression and anxiety at 12 months. Scores on each of the two scales can range from 0-21, with higher scores indicating greater levels of anxiety or depression. HADS has been used extensively and has performed well in a variety of populations: A review of 747 studies concluded that it performed well in assessing the severity and presence of anxiety disorders and depression in both somatic and psychiatric cases in primary care patients and the general population .
Cost utility at 6 and 12 months
The primary outcome measure for the economic analysis will be the quality adjusted life year (QALY). This is a measure of health utility. This is calculated by 'weighting' each period of follow-up time by the value corresponding to the health-related quality of life reported by study participants [38, 39]. Health related quality of life was assessed using the EQ-5D questionnaire collected at 6 and 12 months . The EQ-5D is a validated generic health-related preference-based measure made up of five items covering mobility, self-care, usual activity, pain, anxiety and depression, each with three levels of severity (no problems, some problems, a lot of problems). EQ-5D is supported by National Institute for Health and Clinical Excellence (NICE). The Short Form 12 Health Survey (SF-12) assessed general health status at 6 and 12 months in terms of physical and mental functioning .
6 month intermediate measures for exploratory analysis
It has been argued that evaluations of exercise interventions could usefully assess individual differences in responses to identify why particular intervention content may or may not be successful with particular groups or individuals and to generate theories of causation . A number of measures were collected at 6 months for exploratory analysis.
- Motivation to exercise
This was assessed via the Behavioural regulation in exercise questionnaire-2 (BREQ-2) . Based on self-determination theory  it posits that as motivation becomes more internalized the more it becomes the basis for autonomous behaviour. Thus, it might be expected that participants who start by being externally motivated but who become more internally motivated will have higher rates of adherence than those who, for instance, remain externally motivated. Previous studies have found it was predictive of adherence to exercise on prescription schemes . The measure provides indices for levels of motivation, external, introjected, indentified, and intrinsic motivation, as well as a relative autonomy index (RAI). Scores on each index can range from 0-4, and between -24 - 20 for the RAI, where higher scores indicate a higher degree of motivation.
The self-efficacy and exercise habits survey  was used to measure perceived self-efficacy in terms of exercise behaviours. The two indices are 'sticking to it' and 'making time for exercise'. Each can vary between 1 - 5, where a higher number indicates greater self-efficacy. It has been shown that high perceived self-efficacy facilitates goal-setting, effort investment, persistence in face of barriers and recovery from setbacks . In a review of physical activity studies, Lewis et al.  suggest there is some support for self-efficacy as a mediator of physical activity.
- Social support
The social support and exercise survey 1987 , provides three indices for family social support, friends social support, and family rewards and punishments. Scores on the first two can vary from 5-50, where higher numbers indicate better quantity, structure, and or content of social relationships which facilitate exercise behaviour. Scores on the final area can vary between 2-15, where higher numbers indicate more punishment for exercising. Social support has been linked to a number of health outcomes including adherence to medical regimens, success in smoking cessation, adherence to exercise and enhanced weight loss treatment, although findings have not always been consistent [48, 49].
Variables from programme database
- Programme attendance
A variable measuring programme attendance for each participant was defined using programme attendance data recorded by the EP. This variable has three levels: those who have received no intervention, a partial intervention (between one and 15 weeks), or the full intervention of 16 weeks.
- Programme implementation fidelity
Scores for fidelity to protocols are calculated for each of the 12 areas by allocating a score of 1 each time for; an instance of 4 week contact being made with patients who have stopped attending at this point, for follow up consultations being conducted at 8 months, and for consultations at 12 months with those completing the programme. A score of 0 was allocated for each instance of non contact, providing a range of 0 to 3 and allowing calculation of percentage fidelity overall and by area/exercise professional. An additional measure of fidelity is the quality of goal setting, with goal records coded as measurable and time bound (1) or as not measurable and time bound (0), allowing calculation of percentage fidelity overall and by area/exercise professional.
The research team were responsible for distributing, collecting and processing postal questionnaires at baseline, 6 months and 12 months. Respondents completed an initial informed consent form and subsequent mailings highlighted that they were free to withdraw from the study at any point. Questionnaires were anonymised, featured a unique ID and were accompanied by a freepost return envelope. Questionnaire non-responders were sent a repeat mailing two weeks after the first. For the primary outcome at 12 months, a specialist health research call centre team based at Cardiff University were employed to conduct telephone interviews using a standardised protocol  and blind to condition. Researchers again highlighted that respondents were free to withdraw from the study at any point. All telephone interviewers were trained and monitored to maximise standardisation of interviews. A strict protocol was adhered to in all cases, with each participant dialled on at least 15 occasions at different times and days of the week before being categorised as a non-respondent. Responders to 6 and 12 month questionnaires and 12 month telephone interviews were offered the opportunity to enter separate free prize draws for prizes of £200, £100 and £50 each time.
The trial sample size was determined to detect a difference in the primary outcome, total minutes of weekly exercise at 12 months. The planned sample size of 1052 participants in each group, has 90% power to detect an effect size of 0.15 with no loss to follow-up, and, more realistically, 84% and 87% power to detect an effect size of 0.15 if 25% and 20% respectively of randomised participants are lost to follow up.
The primary analysis of the primary outcome is a regression model with the stratification variables (gender/LHB area), age group (16-44, 45-59, 60+) and baseline activity level (GPPAQ) included as covariates. In the event that the primary outcome has a highly skewed or bimodal distribution, it will be recoded as a five level ordinal variable, and ordinal regression used in the primary and secondary analyses. These analyses will be conducted on an intention-to-treat basis, in which each participant is coded according to the treatment condition to which they were randomised. A secondary analysis of the primary outcome repeats the primary analysis, but excluding baseline activity level as a covariate. For the primary outcome only, the primary and secondary analyses will be conducted (i) for all participants completing the 7-DAY PAR and (ii) with the addition of multiply imputed values of the primary outcome for those who did not complete the 7-DAY PAR but who did complete either the Baecke or GPPAQ instruments at 12-month follow-up.
The primary analysis for the primary outcome, and for imputed values of 7-DAY PAR, is repeated to conduct sub-group analyses for each of the following variables: gender, age group (16-44, 45-59, 60+), referral reason (mental health only, CHD only, or combination of CHD and mental health), Welsh Index of Multiple Deprivation tertile, fidelity of programme implementation in LHB area (high/low). In each case, the statistical significance of sub-group effects is assessed by including an interaction term in separate models for each respective sub-group variable, with each model also including the main effect for the respective sub-group variable. An analysis to identify whether outcomes vary in terms of exposure to the programme replaces the binary intervention group variable with the three level programme attendance variable.
The analyses described in the preceding two paragraphs are also conducted for the secondary outcome. However, the primary analysis for the secondary outcome (HADS anxiety and depression scales) is conducted only among participants referred for mental health only or a combination of mental health and CHD reasons. The analyses of the secondary outcome among all participants is secondary to this analysis.
Other analyses to be conducted, but not associated with formal hypothesis testing, will include an analysis of baseline demographic characteristics and their relationship with outcomes and the relationship between 6-month intermediate variables and the primary and secondary outcomes. This analysis will require complex multivariable models and mediation analyses estimated within a multilevel modelling framework to allow for independence in measures within individuals (over time) and within exercise specialist.
Nested process evaluation
A nested process evaluation examined programme theory, programme implementation and how NERS was received, facilitating an in-depth understanding of the views of providers and users, as well as interpretation of outcome effects . Guided by the framework proposed by Steckler and Linnan , the process evaluation explored diffusion of the national policy across local contexts, fidelity and dose of implementation, patient experiences, programme reach and recruitment. The process evaluation adopted a pluralistic multi method approach involving triangulation  of multiple perspectives and methods (Table 2). All analysis will be conducted prior to any knowledge of outcome effects to guard against interpretation bias
In order to elicit programme theory and provide a framework for implementation checks, a programme logic model was developed through examination of scheme protocols and telephone and email contact with policy representatives. Quantitative assessments of programme implementation in terms of fidelity (i.e. consistency with programme theory) and dose delivered, draws on a combination of self reports from professionals and coordinator telephone interviews, as well as routine monitoring data and structured observation of tape recorded consultations. Exercise professionals were supplied with recording equipment and asked to provide recordings for at least one initial consultation, rated by two independent coders for fidelity to motivational interviewing principles using the Behaviour Change Counselling Index (BECCI); . Programme uptake and reach are quantitatively assessed using routine monitoring data.
Experiences of diffusing the national policy into local contexts were explored through qualitative semi-structured telephone interviews with policy representatives and all local area coordinators. All exercise professionals were invited to participate in one to one telephone interviews exploring issues including experiences of implementing the scheme and perceived patient responses to the programme. In order to incorporate patients' perspectives on the programme, six case study centres were sampled purposively to represent a range of geographical areas and levels of deprivation and two members of the research team visited each centre. Patients attending the centre on the day of the visit were informed in advance of the researchers' attendance, and were invited to participate in group or one to one semi-structured interviews immediately after classes finished. In addition, GPs, practice nurses and practice managers referring patients to the scheme were interviewed, exploring influences on trial and scheme recruitment. Nine group interviews were conducted across areas, practice size and level of referral.
Nested economic evaluation
The nested economic evaluation [55–57] took a public sector, multi agency perspective, spanning the Welsh Assembly Government (WAG), local government and the NHS. It costed the national exercise referral programme using WAG and local authority exercise programme budgets. In addition, telephone interviews with all principal leisure centre managers ascertained any additional expenditure incurred by local authorities in establishing and running NERS. Similarly telephone interviews ascertained the costs to WAG of establishing and coordinating the Scheme. The questionnaire data recorded trial participant primary and secondary care health service use, costed using national unit costs [58, 59]. With this, we will conduct a primary cost utility analysis to calculate the cost per QALY of NERS. We will undertake sensitivity analysis to explore what happens when we add in the cost of a specific GP consultation to facilitate referral onto NERS, and the effect of participants paying £1 or £2 per exercise session. Sensitivity analysis will be particularly important in the application of economic evaluation methods to a large pragmatic public health trial . We will undertake subgroup analysis to investigate how age, gender, adherence and main reason for referral e.g. CHD risk or mental health problems, affects our estimates of the cost per QALY of NERS. We will use EQ-5D as our measure of utility to generate a cost per QALY and Cost Effectiveness Acceptability Curve (CEAC) for comparison with the NICE ceiling of £30,000 .
Trial recruitment occurred in 12 of the 13 areas, with one area failing to start a scheme during the trial period. Between July 2007 and October 2008, 4,779 health professional referrals were received by the evaluation team. Of these, 1,493 were not eligible for the trial and their referral forms were forwarded to the ECs for processing directly onto the scheme. The remaining 3,286 were sent full informed consent information and a baseline questionnaire. In total 2,160 patients were recruited to the trial, with 890 not responding and 236 not consenting to participate in the trial.
Respondents ranged between 16 and 88 years old, with a mean age of 52 (SD 14.7) and the vast majority classed themselves as white (96%). Whilst, measures of area deprivation ranged from 2.3 to 81.0 with a mean of 22.6 (SD 14.6). Table 3 shows that the majority were referred for CHD risk factors on their own (72%) or in combination with mental health conditions (24%), with only 4% referred solely for mental health reasons. Those recruited were most likely to be female (66%), to be married or with a partner (61%) and there was a fairly equal split between those employed (31%) and retired (32%). Finally, whilst the majority referred onto the scheme classified themselves as either inactive or moderately inactive (74%), 24% defined themselves as either active or moderately active.
Previous attempts to facilitate rigorous evaluations of exercise referral schemes have been hampered by policy and practice constraints [25, 26]. Existing studies meanwhile, have suffered from a number of methodological shortcomings, not least significant barriers to recruitment  and a lack of understanding of the causal and implementation process of ERS means that we still do not know what types of ERS have worked, why, for whom, in what circumstances and at what cost . To address these shortcomings, this study included early and sustained collaboration with policy and practice partners, adopted an innovative approach to trial recruitment and included comprehensive nested process and economic evaluations. Such strategies will not only facilitate rigorous evidence on the effectiveness of the national exercise referral scheme but provide examples of good practice for the design and conduct of trials of similar policy interventions.
Collaboration with policy and practice partners
The difficulties in facilitating rigorous evaluations of policy interventions is well recognised, particularly for ERS [25, 26]. This evaluation included a substantial pre trial development phase. Early engagement between evaluators, the national policy co-coordinator (EM) and local implementers at the intervention development and implementation stage meant that a rigorous design could be negotiated that met both scientific and practical considerations. Regular meetings with policy makers and practitioners also ensured that the research design was understood and maintained. The involvement of government social researchers (JH, CR) who promote RCTs of policy evaluations with ministers and support policy colleagues when they are implemented was key to the success of this process. Such an approach is enshrined in the work of the Public Health Improvement Research Network (PHIRN) in Wales which develops and supports multidisciplinary research groups of academics, policy makers and practitioners to identify and develop opportunities for rigorous policy evaluations.
Collaboration with national policy leads and local practitioners was also key in developing and implementing programme monitoring systems. The provision of a national, rather than a number of local schemes, offered the opportunity to introduce a standardised national monitoring system. This included the provision of a common database and protocols for data collection and assessment. High quality programme monitoring data can provide important information, such as referral and completion rates and useful data on programme fidelity. The evaluation will use this information to assess the impact of adherence and fidelity on effectiveness Robust programme monitoring systems also provide the opportunity for long term self evaluation once trials such as this are completed.
Establishing effective recruitment strategies
In Wales, the NERS was introduced within a RCT design in the context of the termination of pre-existing local schemes. This was particularly challenging, as a significant barrier to facilitating rigorous designs has been professional reluctance to refer into trials when there is a perception of change or withdrawal of service, regardless of the evidence base for the effectiveness of that service . Establishing effective recruitment strategies was further complicated by a reliance on primary care clinicians as the main source of referral into the scheme. It has been argued that such clinicians are well placed to promote increased physical activity [62, 63]. However, previous studies have found that physical activity promotion was not a priority during routine consultations [42, 64], that there is a lack of consensus in how clinicians perceive their role in changing patients' behaviour and that patients are frequently referred in an unsystematic way [42, 65].
Despite this, the trial achieved referrals from 12 of the 13 implementation areas (with one area failing to start the scheme) and a high referral rate from clinicians. This may be in part a function of the size of the scheme being evaluated, but was also facilitated by a number of strategies employed in the conduct of this study. This included briefing sessions for professional stakeholders by the evaluation and national policy team at an early stage to address concerns and promote the evaluation. At the local level, the provision of Exercise Coordinators who were able to promote the trial with clinicians also facilitated professional participation. Particularly important was the fact that trial recruitment was undertaken by the evaluation team after clinicians had provided basic information to patients. This placed a low research burden on health professionals who promoted but did not recruit to the trial.
Examining characteristics of those recruited to the trial highlights the greatest proportion were referred for CHD risk, that there were a larger number of females and a significant minority who classified themselves as active or moderately active. Although the scheme was targeted at sedentary individuals, this latter finding reflects the pragmatic nature of the trial which placed minimal control on the implementation of the scheme other than randomisation. Results are therefore likely to be highly reproducible and have high external validity. This is particularly important given previous criticisms of RCTs of ERS that what is gained in internal validity is often at the expense of external validity . Influences on NERS referral processes and the acceptability of trial recruitment strategies will be assessed within the nested process evaluation which will explore influences on clinician referral such as lack of time during consultations; knowledge about ERS, views on evaluation; and medico- legal responsibility [42, 67–69].
A comprehensive understanding of the intervention
Good quality evaluative research not only quantifies outcomes, but also helps us to understand how an intervention produces the outcomes that it does . This is challenging for the evaluation of ERS which are typically heterogeneous and driven by differing latent programme theories, which can only realistically be compared with other similarly conceived schemes . However, no previous evaluations have clearly articulated ERS programme theories and attempted to establish congruence between the conceived and the delivered intervention. In this study, comprehensive nested process and economic evaluations will attempt to address these shortcomings. An intervention logic model will be developed through discussions with implementers which will identify key proposed programme inputs and the anticipated intermediate outcomes associated with these inputs. This will provide the basis for a comprehensive assessment of intervention implementation and will inform attempts to map causal pathways.
Programme implementation will be assessed initially by case studies of community sports centres, purposively sampled to reflect a range of area level socioeconomic deprivation and geographic regions. These will examine intervention delivery across contexts  and explore participant experiences of NERS [72–74]. Data will also inform the development of structured interview schedules for national and local implementers to assess aspects of implementation such as diffusion, reach and fidelity. Aspects of fidelity will also be assessed via routine monitoring data records and systematic checks of treatment integrity . This follows recommends that the delivery of motivational interviewing should be assessed through coding of tape recorded consultations, rather than relying upon self reports of the delivering party . Finally, complex interventions also need to be understood as leading to outcomes through the activation of mechanisms of change, with the activation of these mechanisms varying across contexts and between subgroups . This study has therefore includes a number of variables assessed at 6 months that will be used to model causal processes amongst a number of sub groups. Taken together with the nested economic evaluation these elements provide the opportunity not only to judge the effectiveness of the NERS, but to understand why it may be effective, for whom, under what circumstances and at what cost .
Seven Day Physical Activity Recall interview
Behaviour Change Counselling Index
Behavioural Regulation in Exercise Questionnaire-2
Cost Effectiveness Acceptability Curve
Coronary Heart Disease
Exercise Referral Schemes
General Practice Physical Activity Questionnaire
Hospital Anxiety and Depression Scales
Local Health Board
National Exercise Referral Scheme
Physical Activity Recall
Quality Adjusted Life Year
Randomised Controlled Trial
Short Form 12 Health Survey
Welsh Assembly Government
Department of Health: At least five a week: Evidence on the impact of physical activity and its relationship to health. 2004, London, Department of Health
Paffenbarger RS, Blair SN, Lee IM: A history of physical activity, cardiovascular health and longevity: the scientific contributions of Jeremy N Morris, DSc, DPH, FRCP. International Journal of Epidemiology. 2001, 30 (5): 1184-1192. 10.1093/ije/30.5.1184.
Callaghan P: Exercise: a neglected intervention in mental health care?. Journal of Psychiatric & Mental Health Nursing. 2004, 11 (4): 476-83.
Stathopoulou G, Powers MB, Berry AC, Smits JA, Otto MW: Exercise interventions for mental health: A quantitative and qualitative review. Clinical Psychology-Science and Practice. 2006, 13 (2): 179-193. 10.1111/j.1468-2850.2006.00021.x.
Welsh Assembly Government: Welsh Health Survey. 2008, Cardiff, Welsh Assembly Government
Williams NH: "The wise, for cure, on exercise depend": physical activity interventions in primary care in Wales. Br J Sports Med. 2009, 43 (2): 106-108. 10.1136/bjsm.2008.051458.
Graham RC, Dugdill L, Cable NT: Health professionals' perspectives in exercise referral: Implications for the referral process. Ergonomics. 2005, 48 (11-14): 1411-1422. 10.1080/00140130500101064.
Iliffe S, Tai SS, Gould M, Thorogood M, Hillsdon M: Prescribing Exercise in General-Practice. British Medical Journal. 1994, 309 (6953): 494-495.
Williams NH, Hendry M, France B, Lewis R, Wilkinson C: Effectiveness of exercise-referral schemes to promote physical activity in adults: systematic review. British Journal of General Practice. 2007, 57: 979-986. 10.3399/096016407782604866.
Stevens W, Hillsdon M, Thorogood M, McArdle D: Cost-effectiveness of a primary care based physical activity intervention in 45-74 year old men and women: A randomised controlled trial. British Journal of Sports Medicine. 1998, 32 (3): 236-241. 10.1136/bjsm.32.3.236.
Taylor AH, Doust J, Webborn N: Randomised controlled trial to examine the effects of a GP exercise referral programme in Hailsham, East Sussex, on modifiable coronary heart disease risk factors. Journal of Epidemiology & Community Health. 1998, 52 (9): 595-601.
Harrison RA, Roberts C, Elton P: Does primary care referral to an exercise programme increase physical activity one year later? A randomized controlled trial. Journal of Public Health. 2005, 27 (1): 25-32. 10.1093/pubmed/fdh197.
Munro JF, Nicholl JP, Brazier JE, Davey R, Cochrane T: Cost effectiveness of a community based exercise programme in over 65 year olds: Cluster randomised trial. Journal of Epidemiology & Community Health. 2004, 58 (12): 1004-1010.
Lamb SE, Bartlett HP, Ashely A, Bird W: Can lay-led walking programmes increase physical activity in middle aged adults? A randomised controlled trial. Journal of Epidemiology & Community Health. 2002, 56 (4): 246-252.
Isaacs AJ, Critchley JA, Tai SS, Buckingham K, Westley D, Harridge SD, Smith C, Gottlieb JM: Exercise evaluation randomised trial (EXERT): a randomised trial comparing GP referral for leisure centre-based exercise, community-based walking and advice only. Health Technol Assess. 2007, 11 (10): 1-185.
Gidlow C, Johnston LH, Crone D, James D: Attendance of exercise referral schemes in the UK: A systematic review. Health Education Journal. 2005, 64 (2): 168-186. 10.1177/001789690506400208.
Morgan O: Approaches to increase physical activity: Reviewing the evidence for exercise-referral schemes. Public Health. 2005, 119 (5): 361-370. 10.1016/j.puhe.2004.06.008.
National Institute for Clinical Excellence: Four commonly used methods to increase physical activity: brief interventions in primary care, exercise referral schemes, pedometers and community-based exercise programmes for walking and cycling. 2006, London, NICE
Welsh Assembly Government: Healthy and Active Lifestyles in Wales: a Framework for Action. 2003, Cardiff, Welsh Assembly Government
Welsh Assembly Government: Climbing Higher: the Welsh Assembly Strategy for Sport and Active Recreation in Wales. 2005, Cardiff, Welsh Assembly Government
Welsh Assembly Government: Exercise referral - a guide to developing high quality schemes. 2006, Cardiff, Welsh Assembly Government
Miller WR, Rollnick S: Motivational interviewing: Preparing people for change. 2002, New York: Guilford Press
Cock D, Adams IC, Ibbetson AB, Baugh P: REFERQUAL: a pilot study of a new service quality assessment instrument in the GP exercise referral scheme setting. BMC Health Services Research. 2006, 6: 10.1186/1472-6963-6-61.
Sykes K, Roberts A: The Chester step test-a simple yet effective tool for the prediction of aerobic capacity. Physiotherapy. 2004, 90 (4): 183-188. 10.1016/j.physio.2004.03.008.
Sowden SL, Raine R: Running along parallel lines: how political reality impedes the evaluation of public health interventions. A case study of exercise referral schemes in England. Journal of Epidemiology and Community Health. 2008, 62: 835-841. 10.1136/jech.2007.069781.
Simmons R, Ogilvie D, Griffin S, Sargeant L: Applied public health research - falling through the cracks?. BMC Public Health. 2009, 9: 362-10.1186/1471-2458-9-362.
Moore L, Moore G, Tapper K, Lynch R, Desousa C, Hale J, Roberts C, Murphy S: Free breakfasts in schools: design and conduct of a cluster randomised controlled trial of the Primary School Free Breakfast Initiative in Wales [ISRCTN 1833 6527]. BMC Public Health. 2007, 7 (258):
Pawson R, Tilley N: Realistic Evaluation. 1997, London: Sage Publications
Department of Health: The General Practice Physical Activity Questionnaire. 2006, London, Department of Health
Welsh Assembly Government: Welsh Index of Multiple Deprivation. 2008, Cardiff, Welsh Assembly Government, (URL accessed 01/02/10), [http://wales.gov.uk/docs/statistics/2009/090930wimd08laintroe.pdf]
Blair SN, Haskell WL, Ho P, Paffenbarger RS, Vranizan KM, Farquhar JW, Wood PD: Assessment of habitual physical activity by a seven-day recall in a community survey and controlled experiments. American Journal of Epidemiology. 1985, 122 (5): 794-804.
Donnelly JE, Jacobsen DJ, Jakicic JM, Whatley JE: Very low calorie diet with concurrent versus delayed and sequential exercise. International Journal of Obesity. 1994, 18 (7): 469-475.
Dunn AL, Marcus BH, Kampert JB, Garcia ME, Kohl HW, Blair SN: Comparison of lifestyle and structured interventions to increase physical activity and cardiorespiratory fitness: a randomized trial. JAMA. 1999, 281 (4): 327-34. 10.1001/jama.281.4.327.
Baecke JA, Burema HJ, Frijters JE: A short questionnaire for the measurement of habitual physical activity in epidemiological studies. American Journal of Clinical Nutrition. 1982, 36 (5): 936-942.
Philippaerts RM, Westerterp KR, Lefevre J: Doubly labelled water validation of three physical activity questionnaires. International Journal of Sports Medicine. 1999, 20 (5): 284-289. 10.1055/s-2007-971132.
Snaith RP, Zigmond AS: HADS: Hospital Anxiety and Depression Scale. 1994, Windsor, NFER Nelson
Bjelland I, Dahl AA, Haug TT, Necklemann D: The validity of the Hospital Anxiety and Depression Scale: An updated literature review. Journal of Psychosomatic Research. 2002, 52 (2): 69-77. 10.1016/S0022-3999(01)00296-3.
National Institute for Clinical Excellence: Guide to Methods of Technology Appraisals. 2008, URL accessed 25th March 2010, [http://www.nice.org.uk/media/B52/A7/TAMethodsGuideUpdatedJune2008.pdf]
Richardson G, Bloor K, Williams J, Russell I, Durai D, Cheung WY, Farrin A, Coulton S: Cost-effectiveness of nurse-delivered endoscopy: findings from a randomised Multi-Institution Nurse Endoscopy Trial (MINuET). British Medical Journal. 2009, 338 (519-530): URL accessed 13th May 2010 BMJ 2009;338:b270 doi:10.1136/bmj.b270
Rabin R, De Charro F: EQ-5D: A measure of health status from the EuroQol Group. Annals of Medicine. 2001, 33 (5): 337-343. 10.3109/07853890109002087.
Ware J, Kosinski M, Keller SD: A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Medical Care. 1996, 34 (3): 220-33. 10.1097/00005650-199603000-00003.
Dugdill LR, Graham C, McNair F: Exercise referral: the public health panacea for physical activity promotion? A critical perspective of exercise referral schemes; their development and evaluation. Ergonomics. 2005, 48 (11-14): 1390-410. 10.1080/00140130500101544.
Markland D, Tobin V: A modification of the Behavioral Regulation in Exercise Questionnaire to include an assessment of motivation. Journal of Sport and Exercise Psychology. 2004, 26: 191-196.
Deci EL, Ryan RM: Intrinsic motivation and self-determination in human behavior. 1985, New York: Plenum
Edmunds J, Ntoumanis N, Duda J: Adherence and well-being in overweight and obese patients referred to an exercise on prescription scheme: A self-determination theory perspective. Psychology of Sport and.
Sallis JF, Pinski RB, Grossman RM, Patterson TL, Nader PR: The development of self-efficacy scales for health-related diet and exercise behaviors. Health Education Research. 1988, 3 (3): 283-292. 10.1093/her/3.3.283.
Bandura A: Self-efficacy: toward a unifying theory of behavioral change. Psychological Review. 1977, 84 (2): 191-215. 10.1037/0033-295X.84.2.191.
Lewis BA, Marcus BH, Pate RR, Dunn AL: Psychosocial mediators of physical activity behavior among adults and children. American Journal of Preventive Medicine. 2002, 23 (2, Supplement 1): 26-35. 10.1016/S0749-3797(02)00471-3.
Sallis JF, Grossman RM, Pinski RB, Patterson TL, Nader PR: The development of scales to measure social support for diet and exercise behaviors. Preventive Medicine. 1987, 16 (6): 825-836. 10.1016/0091-7435(87)90022-3.
Sarkin J, Campbell J, Gross L, Roby J, Brazzo S, Sallis J, Calfas K: Project GRAD seven day physical activity recall interviewer's manual. Medicine & Science in Sports & Exercise. 1997, 29: S91-S102.
Springett J: Appropriate approaches to the evaluation of health promotion. Critical Public Health. 2001, 11 (2): 139-151. 10.1080/09581590110039856.
Steckler A, Linnan L: Process evaluation for public health interventions and research. 2004, San Fransico, California: Jossey-Bass
Beattie A, (ed): Health and Wellbeing: A Reader. The changing boundaries of health. 1993, Oxford: Oxford University Press
Lane C, Huws-Thomas M, Hood K, Rollnick S, Edwards K, Robling M: Measuring adaptations of motivational interviewing: the development and validation of the behavior change counseling index (BECCI). Patient Education and Counseling. 2005, 56 (2): 166-173. 10.1016/j.pec.2004.01.003.
Drummond M, Sculpher MJ, Torrance GW, O'Brien BJ, Stoddart GL: Methods for the Economic Evaluation of Health Care Programmes. 2005, Oxford: Oxford University Press, 3-
Glick HA, Doshi JA, Sonnad SS, Polsky D: Economic Evaluation in Clinical Trials (Handbooks for Health Economic Evaluation). 2007, Oxford: Oxford University Press
Edwards R, Hounsome B, Linck P, Russell I: Economic evaluation alongside pragmatic randomised trials: developing a standard operating procedure for clinical trials units. Trials. 2008, 9 (1): 64-10.1186/1745-6215-9-64.
Department of Health: NHS reference costs 2008. 2008, London; Department of Health, [http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_098945]
Curtis L: Unit costs of health and social care. 2008, Canterbury, Kent: Personal Social Services Research Unit, [http://www.pssru.ac.uk/uc/uc2008contents.htm]Accessed 8th July 2009
Craig P, Dieppe P, Macintyre S, Macintyre S, Mitchie S, Nazareth I, Petticrew M: Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008, 337: a1655-10.1136/bmj.a1655.
Edwards RT, Ceilleachair AJO, Bywater T, Hughes DA, Hutchings J: Parenting programme for children at risk of developing conduct disorder: cost-effectiveness analysis. BMJ. 2007, 334: 682-687. 10.1136/bmj.39126.699421.55.
Stott NCH, Davis RH: The exceptional potential in each primary care consultation. J R Coll Gen Pract. 1979, 29: 201-205.
Wyke S: Reasons for consultation. Oxford Textbook of Primary Medical Care: Volume 1: Principles and concepts. Edited by: Jones R. 2004, Oxford, Oxford University Press, 95-98.
Coulter A, Schofield T: Prevention in general practice: the views of doctors in the Oxford region. British Journal of General Practice. 1991, 41 (345): 140-3.
Lawlor DA, Keen S, Neal RD: Increasing population levels of physical activity through primary care: GPs' knowledge, attitudes and self-reported practice. Family Practice. 1999, 16: 250-254. 10.1093/fampra/16.3.250.
Gidlow CL, Johnston H, Crone D, Morris C, Smith A, Foster C, James DV: Socio-demographic patterning of referral, uptake and attendance in Physical Activity Referral Schemes. J Public Health. 2007, 29: 107-113. 10.1093/pubmed/fdm002.
Gould MM, Thorogood M, LLiffe S, Morris JN: Promoting physical activity in primary care: measuring the knowledge gap. Health Education Journal. 1995, 54 (3): 304-311. 10.1177/001789699505400305.
McKenna J, Naylor PJ, McDowell N: Barriers to physical activity promotion by general practitioners and practice nurses. Br J Sports Med. 1998, 32 (3): 242-247. 10.1136/bjsm.32.3.242.
Douglas F, Torrance N, van Teijlingen E, Meloni S, Kerr A: Primary care staff's views and experiences related to routinely advising patients about physical activity. A questionnaire survey. BMC Public Health. 2006, 6 (1): 138-10.1186/1471-2458-6-138.
Rychetnik L, Frommer M, Hawe P, Shiell A: Criteria for evaluating evidence on public health interventions. Journal of Epidemiology and Community Health. 2002, 56 (2): 119-127. 10.1136/jech.56.2.119.
Department of Health: Exercise referral systems: A national quality assurance framework. 2001, London
Stathi A, McKenna J, Fox K: The experiences of older people participating in exercise referral schemes. Journal of The Royal Society for the Promotion of Health. 2004, 124: 18-23. 10.1177/146642400312400108.
Wormald H, Ingle L: GP exercise referral schemes: Improving the patient's experience. Health Education Journal. 2004, 63 (4): 362-373. 10.1177/001789690406300407.
Hardcastle S, Taylor AH: Finding an exercise identity in an older body: "It's redefining yourself and working out who you are". Psychology of Sport and Exercise. 2005, 6 (2): 173-188. 10.1016/j.psychsport.2003.12.002.
Moyers TB, Martin T, Manuel JK, Hendrickson SML, Miller WR: Assessing competence in the use of motivational interviewing. Journal of Substance Abuse Treatment. 2005, 28 (1): 19-26. 10.1016/j.jsat.2004.11.001.
Miller WR: When is it motivational interviewing? Commentary. Addiction. 2001, 96 (12): 1770-1772.
Clark AM, MacIntyre PD, Cruickshank J: A critical realist approach to understanding and evaluating heart health programmes. Health. 2007, 11 (4): 513-539. 10.1177/1363459307080876.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/10/352/prepub
We thank Kim Sheppard and Chris Warlow for administrative assistance. We also thank our advisory group, particularly Melvyn Hillsdon and JH and EM for support and assistance throughout the project. In addition, we gratefully acknowledge the co-operation of all those participating in this study. The Thames Valley Multi-centre Research Ethics Committee (MREC) approved the evaluation of the Welsh NERS on Feb. 8, 2007 (Ref: 06/MRE12/85). Due to changes in COREC procedures, an additional Site-Specific Information (SSI) form (Ref: C/92139/137055/1) was approved by the MREC. Approval from medical directors within each local health board (LHB) was also obtained. The study is funded by the Welsh Assembly Government (tender contract number: 146/2005).
The authors declare that they have no competing interests.
Principal responsibility for the main study design is assumed by SM. LR is the trial manager and responsible for the day to day running of the study. GM is responsible for the development and day-to-day running of the mixed-method process evaluation. RTE is responsible for the economic evaluation study design. PL is responsible for the day to day running of the economic evaluation. NW contributed to the study design and aspects of the process evaluation. JH and CR were responsible for developing the initial research specification and for input into the study design and EH for facilitating the implementation of the research design. LM contributed to the study design, conducted sample size calculations and designed the analysis plan. LR produced a first draft of the manuscript and SM was responsible for the final revised version developing and integrating contributions from GM, RTE, PL, NW and LM. All authors read and commented on drafts and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
Rights and permissions
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Murphy, S., Raisanen, L., Moore, G. et al. A pragmatic randomised controlled trial of the Welsh National Exercise Referral Scheme: protocol for trial and integrated economic and process evaluation. BMC Public Health 10, 352 (2010). https://doi.org/10.1186/1471-2458-10-352
- Motivational Interview
- Quality Adjusted Life Year
- Coronary Heart Disease Risk Factor
- Trial Recruitment
- Programme Attendance