Interpreting population reach of a large, successful physical activity trial delivered through primary care

Abstract

Background

Failure to include socio-economically deprived or ethnic minority groups in physical activity (PA) trials may limit representativeness and could lead to implementation of interventions that then increase health inequalities. Randomised intervention trials often have low recruitment rates and rarely assess recruitment bias. A previous trial by the same team using similar methods recruited 30% of the eligible population but was in an affluent setting with few non-white residents and was limited to those over 60 years of age.

Methods

PACE-UP is a large, effective, population-based walking trial in inactive 45-75 year-olds that recruited through seven London general practices. Anonymised practice demographic data were available for all those invited, enabling investigation of inequalities in trial recruitment. Non-participants were invited to complete a questionnaire.

Results

From 10,927 postal invitations, 1150 (10.5%) completed baseline assessment. Participation rate ratios (95% CI), adjusted for age and gender as appropriate, were lower in men 0.59 (0.52, 0.67) than women, in those under 55 compared with those ≥65, 0.60 (0.51, 0.71), in the most deprived quintile compared with the least deprived 0.52 (0.39, 0.70) and in Asian individuals compared with whites 0.62 (0.50, 0.76). Black individuals were equally likely to participate as white individuals. Participation was also associated with having a co-morbidity or some degree of health limitation. The most common reasons for non-participation were considering themselves as being too active or lack of time.

Conclusions

Conducting the trial in this diverse setting reduced overall response, with lower response in socio-economically deprived and Asian sub-groups. Trials with greater reach are likely to be more expensive in terms of recruitment and gains in generalizability need to be balanced with greater costs. Differential uptake of successful trial interventions may increase inequalities in PA levels and should be monitored.

Trial registration

ISRCTN.com ISRCTN98538934. Registered 2nd March 2012.

Background

Failure to include groups with lower physical activity (PA) levels in trials assessing the effectiveness of PA interventions, may increase health inequalities. The health benefits of PA are well established, but the numbers achieving recommended PA levels are generally low [1], with women, older people, those from socio-economically deprived areas and those of Asian ethnicity [1, 2] reporting lower levels. Although a number of walking intervention trials have been conducted [3], and recruitment rates of 30-40% [4,5,6,7], reported, some trials have much lower rates [8,9,10] and many population level PA interventions use advertising to recruit and cannot therefore estimate recruitment rates [11, 12]. The extent to which the results of such trials could be applied to those who might be offered the intervention in routine practice, sometimes referred to as an intervention’s ‘reach’ is unclear [13]. Reach is affected by the setting, participation rates and participant representativeness. Where participation rates are low [8,9,10], there may be systematic differences between those who participate and those who do not participate but who may represent a sizeable proportion of the population to whom the intervention could be appropriately offered.

Identifying trial recruitment inequalities is important to understand the limitations of the evidence, in terms of generalisability, and also to aid in planning studies. Failure to recruit is a major concern for research funders. Forty-five percent of trials funded by two UK funding agencies between 2002 and 2008 failed to recruit to target and required extensions [14]. The review found community and primary care trial recruitment similar to other settings, but did not investigate whether increased reach was associated with lower recruitment levels. Trials carried out in more ethnically diverse and socioeconomically deprived settings may be less likely to achieve recruitment targets.

The PACE-UP pedometer-based walking intervention trial recruited from primary care registers in seven south-west London practices [15]. The trial was successful as both intervention arms increased objectively measured physical activity levels at 12 months [16]. The population-based sampling frame provided an opportunity to assess differences in terms of age, gender, ethnicity and area-level deprivation between general practice (GP) patients who agreed to participate in the trial compared with those who did not, and between those who replied to the invitation letter compared with those who did not. We also compared health, lifestyle, education and social factors of those who agreed to participate with those who agreed to complete a questionnaire, but did not wish to participate.

Methods

The PACE-UP trial

The PACE-UP trial [15] recruited inactive 45-75 year olds registered at seven south-west London general practices and randomised 1023 people to one of three arms: an intervention arm designed to increase walking using pedometers, personalised walking plans and nurse consultations; a postal intervention arm (without nurse consultations); and a standard practice arm. Participants in the trial had to be able to walk outside and have no contraindications to increasing their PA levels.

Potentially eligible patients were identified using Read codes for medical conditions and local knowledge about care homes to exclude ineligible patients. The trial randomised households (one person living alone or two people with an age difference of less than 15 years if more than one eligible person in the household). Batches of approximately 400 potentially eligible individuals from randomly selected households were screened for exclusion criteria by general practitioners or practice nurses to avoid inviting patients with conditions that would exclude them from being offered the intervention in routine care, but which are not well recorded on Read codes (e.g., acute systemic illness such as pneumonia, unstable heart failure, unable to move about independently, psychotic illness). Some individuals will have already been screened out through READ codes. The remainder were invited to participate in the trial by letter, which included the trial information leaflet, a reply slip and stamped addressed envelope. Those willing to participate but who reported that they achieved, or were not sure if they achieved, recommended levels of at least 150 min weekly of at least moderate intensity physical activity [17], were telephoned to check their eligibility. If confirmed to be active they were excluded. One reminder letter was sent to those who did not reply. All trial information was written in English. For further details see the trial protocol [15].

Data collection

The gender, age and Index of Multiple Deprivation (IMD) of all those invited were collected from GP records. IMD is an anonymised post-code deprivation measure [18]. To avoid the possibility of individuals being identified, aggregated practice-recorded ethnicity was exported in 10 year age-bands for all batches where everyone was mailed, less exclusions. We classified the practice recorded ethnicity into 4 categories, White (including ‘British or Mixed British’), Asian (including ‘Asian British’), Black (including ‘Carribbean’, ‘African’ and ‘Black British’) and Other.

Those not wanting to participate in the trial were asked if they would complete a shortened trial baseline questionnaire, including demographics, health, a primary care PA questionnaire (General Practice Physical Activity Questionnaire GPPAQ [19]), EQ-5D [20] and a question on reasons for not participating.

Comparison groups

Individuals whose invitation letters were ‘returned-to-sender’ were excluded from analyses before calculating response rates. ‘Responders’ are defined as those who replied to the invitation letter, regardless of whether they wanted to take part or not. Individuals could respond by post, email or phone.

‘Participants’ are those who completed baseline assessment, although not all were randomised as some provided inadequate objectively measured physical activity data. ‘Non- participants’ are those who completed a questionnaire but did not wish to participate in the trial (Fig. 1).

Fig. 1

Flow chart to show the recruitment process in the PACE-UP trial. All percentages are out of all those whose age and gender were matched with GP records (10927)

Since PACE-UP targeted inactive adults, participants who attended a baseline appointment were selected on the basis of their low PA levels. Non-participants were not selected in this way. In order to minimise selection bias, analysis of participants and non-participants was therefore restricted to those categorized as ‘not active’ according to GPPAQ, which was the only physical activity measure available on both groups.

Statistical analysis

Age and gender standardised rates were used to compare IMD quintiles for responders. Similarly, gender standardised rates were used to compare age groups and age standardised rates to compare genders. The full population of invitees was used as a standard population throughout. No further analysis on non-responders was possible because they did not provide any questionnaire data on ethnicity or other factors.

Practice ethnicity data were extracted in 10-year age bands for 10,155 invitees from batches where everyone was mailed, effectively a random sample of the 11,015 invited. The proportion of patients belonging to each ethnicity category within age band and within practice was calculated and the number of invitees in each ethnicity in each practice and age-band was estimated. Overall, 1903 invitees had ethnicity recorded as ‘unknown’. These are assumed to be missing at random in the main results but sensitivity analyses were performed, assuming these were all white or all non-white. Age standardised participation rates for not active participants and non-participants completing questionnaires were calculated assuming that invitees gave the same ethnicity on the questionnaire as was recorded in their practice records. Participation rates by age, gender and IMD were calculated for not active participants versus not active non participants completing questionnaires, as in the analysis of responders.

Not active participants and non-participants completing the questionnaire were compared for additional demographic and social characteristics and health and lifestyle factors using logistic regression. All data came from questionnaires. Models were adjusted for clustering by practice and household by including fixed effects for practice and using robust standard errors for household.

Results

Of 12,625 individuals selected for screening (see Fig. 1), 1421 (11.3%) were excluded by practice staff and 189 (1.5%) had invitation letters that were returned, as they had moved away; both of these groups were classified as ‘not invited’. In 44 households where one person refused the invitation and the other did not respond, it was impossible to match the response to individual invitees within household, so age and gender are unknown. These 88 people have been excluded from all further analyses. Of the remaining 10,927, 4572 (42%) responded to the invitation letter, mainly by post, and 1150 (11%) completed baseline assessments.

Of all invitees, 5229 (48%) were aged 45 to 54. Although all quintiles of deprivation were represented, only 7% were in the most deprived quintile. Response rates were higher in older people, women and those living in less deprived areas (Table 1). Since individual ethnicity was available only for the participants and non-participants who completed a questionnaire, it was not possible to estimate response rates by ethnicity for all responders.

Table 1 Responders to invitation letter by age, gender and Index of Multiple Deprivation (IMD)

Although GPPAQ was not used to assess PA levels for trial inclusion, it was the only PA measure available for both participants and non-participants. 118 participants and 388 non-participants were classified as active by GPPAQ and 134 did not complete GPPAQ. These people were excluded from further analysis, leaving 924 participants and 715 non-participants.

Similar to response rates, participation rates were higher in older people, women and those living in less deprived areas (Table 2). Ethnicity was extracted from the practice for 10,155 invitees. Of these, 5991 were recorded as White (59%), 893 (9%) as Asian or British Asian and 915 (9%) as Black Caribbean, Black British or Black African. 1903 (18.7%) were recorded as ‘unknown’. The percentage ‘unknown’ varied by practice from 3% to 48%.

Table 2 Completion of baseline assessment and questionnaires in participants and non-participants who are not active on GPPAQ by age, gender, Index of Multiple Deprivation (IMD) and ethnicity

Of the White invitees 709 (8.7%) agreed to participate in the trial and were not active and a further 638 (7.9%) completed a non-participant questionnaire and were not active. Both Asian and Black invitees had very low non-participant questionnaire completion (2.4% and 1.9%) but black invitees were as willing to participate as white invitees (8.5% v 8.7%), while only 5.4% of Asians participated. Sensitivity analyses assuming that all ethnicities recorded as ‘unknown’ were white or non-white showed similar results, and the same patterns were also seen in practices with nearly complete ethnicity coding.

Compared with non-participants providing questionnaire data, participants were more likely to be working part-time, to be married or living with a partner, and to have finished their education between 17 and 18 years (Table 3). Participation was associated with recent primary care contact and with some degree of health problems (general health, long standing illness and co-morbidities), although those more severely affected were less likely to participate (Table 3). This is consistent with EQ-5D (health-related quality of life) domains, where participants were more likely to have problems with pain and mobility but less likely to have problems with self-care, which is likely to indicate greater disability.

Table 3 Participants and non-participants who completed questionnaires and were not active on GPPAQ: demographics, and health and lifestyle factors

Participants were less likely to walk fast, but there was no statistically significant association between participation and having someone to walk with (Table 3), having balance problems or falling.

Insufficient time was given by 45% (n = 327) of all invitees and 60% of 45-54 year olds as a reason for non-participation (Table 4). Even though those classified on GPPAQ as active were excluded from this analysis, 45% of invitees gave being sufficiently active as a reason, more commonly cited by men and those in less deprived areas. There were no clear trends with ethnicity but the number of non-participants in ethnic minority groups was small. Less commonly, 152 (21%) answered they could not or were not interested in (122, 17%) increasing their PA. Randomisation was only cited as a reason for non-participation by 88 (12%) of respondents.

Table 4 Reasons for non-participation by age, gender, Index of Multiple Deprivation (IMD) and ethnicity for not active non participants

Discussion

Summary of findings

The PACE-UP trial recruited 11% of patients aged 45 to 75 invited by post by their practice. Those replying were older, more likely to be female and from less deprived postcodes. Participants in the trial who completed a baseline assessment and were not classified as active by questionnaire, were also more likely to be older, female, and from less deprived postcodes compared with non-active non-participants. Asian patients were less likely to participate. Participation was associated with having some comorbidity or some degree of health impairment, and having had recent primary care contact. Insufficient time and perceiving themselves as being already physically active were common reasons for non-participation, even though we only included those who were classified by GPPAQ as not active.

Comparison with previous work

A systematic review of 47 walking intervention studies [3] showed recruitment methods and participation rates were poorly reported. Participation rates could only be calculated for 5/25 randomised controlled trials. We recruited by post to reduce practice staff burden and to obtain response rate data. Primary care uses postal invitations for other preventive activities, making this a pragmatic approach [21]. Other primary care walking interventions using postal invitations [8, 22,23,24] had similar response rates of 10 to 20%. Dubbert [4, 5] had higher rates (37% and 39%) but recruited additionally through routine primary care visits and patients were over 60 with on average 3.8 co-morbidities. We have shown that older age, having some comorbidity and recent health professional contact were associated with increased trial participation. Our previous trial [7] used similar recruitment strategies to PACE-UP and had a 30% recruitment rate, but was limited to over 60 year olds, was conducted in an affluent setting with few non-white residents and did not exclude those reporting they were active.

Non-responders were followed up with one reminder letter, but due to data protection constraints we could not telephone non-responders. Although only 1% of invitation letters were returned to sender, this may underestimate those not receiving the letter, as we did not used registered post. A previous London study using registered post found 26% of letters were not delivered [25]. Warner [24] found active refusal rates to be low but about 30% of eligible patients could not be contacted.

Most recruitment studies focus on recruitment methods [9, 26, 27]. In this study we compared participation rates for different groups within one primary care study using postal invitations. We have previously explored this in the PACE-Lift trial [28] setting, but PACE-UP is larger, with greater diversity, allowing ethnicity and deprivation effects to be explored comprehensively. We have already published findings from interviews to explore reasons for non-participation [29] in PACE-UP.

Our finding of greater participation in women, older people and those in affluent areas are supported by other studies [28, 29]. Although Attwood [30] found no association with deprivation or ethnicity this was in a highly deprived area with few non-white patients.

Among Asian patients, our response rate was similar to postal invitations in the PODOSA trial (5.2%) where community based approaches [31], through partnership with local South Asian groups were found to be more effective. Wilbur found social networking the most effective method for recruiting African American women from low income areas [27].

Strengths and limitations

PACE-UP is a large trial recruiting from a clearly defined invited population, based on GP lists, enabling us to assess the potential reach of the intervention in terms of age, gender and deprivation. Our estimate of 11% participation may be an underestimate of the true rate, particularly in areas of high mobility.

Although based on limited data, the PACE-UP trial offers a rare opportunity to examine demographic differences between participants and non-participants. We were able to estimate participation within different ethnicities using pooled data from the practices. However, we were not able to match at an individual level and some participants may have categorised themselves in a different ethnic group to that on the GP register. Ethnicity was also poorly recorded in some practices and we needed to make assumptions about whether those with ‘unknown’ ethnicity were similar to those with recorded ethnicity. In a sensitivity analysis, even under extreme assumptions, the same ethnic variations persisted. It is possible we have underestimated the response rates in White people compared to other groups because in our estimation of the number of white invitees we classified as ‘White’ those classified as ‘British or Mixed British’; the practice data did not distinguish between ‘British’ and ‘Mixed British’. However comparing with the area census data (Wandsworth in 2011) [32] we believe any effect would be small.

The trial excluded individuals who self-reported being active, but the non-participants were not selected in this way. Our analysis attempted to mitigate this difference by restricting analysis to all those who self-reported as not being active using the same question. However, some residual bias may remain.

One possible limitation is that we did not attempt to compare different recruitment methods. Our main focus was to reach our recruitment target of 993 randomised patients in 1 year and we had insufficient resources to design a study within a trial [33] to compare different recruitment strategies. We chose to use postal recruitment which had been successful in our previous trial and we planned conservatively so that we had a large enough pool of invitees to achieve our target even with a low response rate. Copeland et al. [9] tried a number of strategies to boost recruitment in addition to postal invitation. These included asking GPs to recruit during consultations, and asking community leaders, health trainers and champions to distribute recruitment packs but these were all unsuccessful and all participants were recruited postally. Recruitment strategies based on advertising or community groups could be compared in terms of research effort required but would not allow characteristics of participants and non-participants to be compared; one of the key strengths of this study.

Implications

NICE guidelines [34] conclude that more research is needed to determine which interventions are effective and cost-effective in increasing activity levels among lower socio-economic and high risk groups, and that there is little evidence on differential effects of interventions. In our trial those groups for which more evidence is required tended to be those with the lowest recruitment rates, such as Asians and those in more deprived areas [2]. It has been suggested [35] that specific cultural groups may respond better to interventions directly targeted at their needs, rather than to universal interventions. Reasons for non-participation often related to individuals not wanting to increase activity or feeling that they were sufficiently active. It is likely such resistance will similarly apply to any intervention roll out and may apply more widely to other public health interventions. Low participation rates mean policy makers should be cautious about the intervention’s potential reach and the possibility that it could increase activity inequalities, but is not a reason not to implement an intervention shown to be effective in 11% [16] of the population. We were successful in recruiting older people, women and those with co-morbidities or some degree of health limitation. These groups have lower PA levels and are likely to benefit more from increased physical activity. However, those with more severe disability, fallers, and those with a fear of falling were not over-represented, indicating a rational choice by individuals.

Only 12% of non-participants cited randomisation as a factor for not participating, while 45% cited time constraints. The nurse intervention required three additional visits to the practice on top of the three data collection visits, which may deter working people or those with childcare and other commitments. However, PACE-UP showed that both the nurse and postal groups performed similarly at the main 12 month outcome [16]. An intervention offering pedometers with brief advice, without the need to provide research data, may be more acceptable.

Both PACE-Lift and PACE-UP recruited to target, achieved follow up rates of over 90% and demonstrated the interventions were effective in increasing physical activity [16, 24]. However, considerably more research effort was required per randomised participant in PACE-UP compared with PACE-Lift, due to lower uptake. In spite of the effort, we still had limited power to investigate ethnic and socio-economic subgroups.

Conclusions

Participation in an effective physical activity trial among adults and older adults in a socially and ethnically diverse population was only 11% with lower rates in more deprived and Asian subgroups, limiting the trial’s ability to investigate differential effects in these important subgroups. Trials with greater reach are likely to be more expensive in terms of recruitment and gains in generalizability need to be balanced with greater costs. Differential uptake of interventions found to be successful in trials may increase inequalities in PA levels and should be monitored.