The acceptability and impact of a randomised controlled trial of welfare rights advice accessed via primary health care: qualitative study
© Moffatt et al; licensee BioMed Central Ltd. 2006
Received: 01 February 2006
Accepted: 21 June 2006
Published: 21 June 2006
Qualitative research is increasingly used alongside randomised controlled trials (RCTs) to study a range of factors including participants' experiences of a trial. The need for a sound evidence base within public health will increase the need for RCTs of non-clinical interventions. Welfare rights advice has been proposed as an intervention with potential to reduce health inequalities. This qualitative study, nested within an RCT of the impact of welfare rights advice, examined the acceptability of the intervention, the acceptability of the research process and the perceived impact of the intervention.
25 men and women aged 60 years or over were recruited from four general practices in Newcastle upon Tyne (UK), a sub-sample of those who consented to be contacted (n = 96) during the RCT baseline interview. Semi-structured interviews were undertaken and analysed using the Framework Method.
Participants viewed the trial positively although, despite agreeing that the information leaflet was clear, some had agreed to participate without being fully aware of what was involved. Some participants were unaware of the implications of randomisation. Most thought it fair, but a few concerns were raised about the control condition. The intervention was acceptable and made participants feel confident about applying for benefit entitlements. 14 out of 25 participants received some financial award; median weekly income gain was £57 (€84, $101). The perceived impact of additional finances was considerable and included: increased affordability of necessities and occasional expenses; increased capacity to deal with emergencies; and a reduction in stress related to financial worries. Overall, perceived independence and ability to participate in society increased. Most participants perceived benefits to their mental well-being, but no-one reported an improvement in physical health. The RCT showed little or no effect on a wide range of outcome measures.
Participation in the trial and the intervention was acceptable to participants. Welfare rights advice targeted at people aged 60 years or over and accessed via primary care had a positive impact on quality of life and resulted in increased social participation. The divergence of qualitative and quantitative findings suggests that both methods make important contributions to the evaluation of complex social interventions.
Qualitative research is increasingly used alongside randomised controlled trials (RCTs) to study factors such as recruitment, participants' experiences, lay understandings of randomisation and informed consent, and to illuminate reasons why the intervention may or may not have worked. Much of this work has focused on participant experiences in clinical trials, but the need for a sound evidence base for public health will increase the need for RCTs of non-clinical interventions. While evidence of outcomes can be derived from RCTs, 'evidence of the process by which those outcomes were achieved, the quality of implementation of the intervention, and the context in which it occurred is likely to come from qualitative data' (p528).
Welfare rights advice can lead to significant financial and non-financial gains [8–10], and it has been proposed as an intervention with significant potential to reduce inequalities in health. Previous qualitative research has indicated that welfare rights advice accessed via primary care is viewed positively and is perceived to address both social and health needs. A number of studies of welfare rights advice accessed via primary care indicate the usefulness of such services and their perceived acceptability among recipients, primary care and welfare rights staff [13–17]. However, no robust qualitative work has examined the acceptability and health impact of this social intervention within the context of an RCT among those aged 60 years or over.
In this qualitative study, undertaken within the context of a pilot RCT reported in an accompanying paper, we aimed to assess: the acceptability of a domiciliary welfare rights advice service accessed via primary care; the acceptability of the research process; and the perceived impact of the intervention, in order to inform future studies.
The study took place in parallel with a single blind pilot randomised controlled trial with allocation of individuals to intervention (receipt of welfare rights advice immediately) or control condition (receipt of intervention after a six month delay), the methods of which are described in detail in the accompanying paper. The intervention comprised a domiciliary assessment of current welfare status and benefits entitlement, active assistance in making claims where appropriate, and follow-up for unresolved claims as required.
Extracts from participant information leaflet
What is this study about?
A lot of people do not get the state welfare benefits that they should. We are testing out a new way of giving advice about welfare rights in your doctor's surgery. We are trying to find out how this advice may help and whether there are any health benefits. We are inviting you to take part in this study.
What does this involve?
Mainly, we need you to give up a little of your time. You will be asked some questions about your health and circumstances in a private interview. This will probably take about an hour. After this you will be given an appointment to see a Welfare Rights Officer from Newcastle Social Services, who will give you confidential advice about your rights to state and other benefits, and help with making claims if appropriate. If you have difficulty filling out forms, help will be provided. The Welfare Officer can come to your home or meet with you at your general practice.
In this study, one group of people will be given an appointment with the welfare advice service straight away and another group of people will be given an appointment a few months later. The group you are put in will be decided by chance, like tossing a coin. However, everyone will get an appointment within 6 months.
Appointments were arranged by telephone and semi-structured interviews were undertaken by SM between April and December 2003 in participants' homes. The average time between the welfare rights consultation and research interview was ten months for the intervention group and five months for the control group because the control group received their welfare rights assessment six months after the intervention group. The topic guide covered perceptions of the impact of material and/or financial benefits on physical/mental health, health related behaviours, social benefits and the acceptability of the intervention and research process. Interviews were audio-recorded and transcribed in full.
We undertook thematic analysis following the Framework method, with constant comparison and deviant case analysis to enhance internal validity. Resulting typologies were derived and higher level descriptive and explanatory categories developed.
The protocol was approved by the Newcastle upon Tyne joint universities and NHS research ethics committee.
Participants and benefits received
Twenty-five semi-structured, in-depth interviews were undertaken (14 intervention, 11 control). Ten participants were interviewed with partners who made active contributions. Fourteen participants received some financial award. The median weekly income gain was £57 (€84, $101) (range £10 (€15, $18) to £100 (€148, $178)) representing a 4–55% increase in household income. Eighteen participants were in receipt of benefits, either as a result of the current intervention or because of claims made prior to the study. Although 96 of the total sample (126) agreed to participate, data saturation was reached after interviewing 25 respondents.
Acceptability and understanding of the research process
Participation in the RCT required considerable commitment. All participants were interviewed four times over 24 months with the same structured interview schedule and the qualitative sub-sample participated in a further in-depth interview. Participation rates for the RCT remained high (87%) after two years (five deaths; one moved, 11 declined follow-up).
Experiences of the research process
I feel that I'm enjoying it, I mean I've quite enjoyed listening and as I say, it's benefited us in more ways than one, because it's given me ideas and possibly it's given you ideas as well. (Case 8, male, 77)
I was apprehensive about going out, I thought maybe I had to go to a hospital meeting, and I found it wonderful, it's all in my own home. (Case 3, female, 72)
You look at the stature of the organisation involved and the people that come to see you. (Case 20, male, 72)
Some of the questions have been personal but there has been no sort of awkwardness and everything they have asked me I have answered ... and they have been very friendly ... there has been no sort of dictatorial attitude or anything. (Case 1, male, 75)
Basically, when someone comes and says, well it's totally confidential, I believe them that it is totally confidential. I mean I understand the Data Protection Act ... so you know, no problem whatsoever. (Case 18, male, 65)
Motivation for taking part
I feel it's sort of putting something back. (Case 22, female, 78, increased Attendance Allowance from low to high rate and Staywarm)
I didn't realise I was claiming for money ... it wasn't put to me like that. It was just a study, people were coming to ask you questions and you would answer them ... I didn't realise at all about getting a grant. (Case 1, male, 75, gained higher rate Attendance Allowance)
Well, if we don't take part in research we're never going to learn ... research produces information that you can properly base your next decision on. (Case 20, male, 72, no gain)
Well, there's no harm in anybody coming to the house and anybody asking questions. That's the way I look at it and if it's going to benefit you, why shouldn't you accept them. (Case 4, male, 74, gained Attendance Allowance (lower rate) and Invalid Care Allowance)
Well they are reliable, sensible, you know, educated people, and they know what they are talking about. And if they go to the trouble of writing me a letter, well it must be for some good. (Case 25, female, 82, no gain)
Coming from your doctor, you know ... obviously they're not going to ask you if it's not important. (Case 10, male, 65, pension credit)
Views on randomisation
I probably wouldn't know whether I was in the first group or the second group, so I mean you wouldn't be worried about it ... . Well I don't think it would bother me really because you got there in the end you know. It's not like it's a race ... it's just one of those things isn't it. (Case 19, female, 82, intervention)
If they couldn't see everyone all at once ... if you say it was randomly done, so they didn't even assess the pros and cons of the person's case so that was fair, wasn't it? (Case 7, male, 75, control)
I mean, if people sign up and they know that they're on course to be seen six months later, and if they're unfortunate to be in the second group, hard cheese. (Case 8, male, 77, intervention)
I thought, well that would have been lovely if I'd had that six months ago or a year ago. I mean I didn't know that you'd done what you'd said you'd done. (Case 22, female, 78, control)
Well, I just accepted that really. I thought, well if I wasn't in the first group that would be it, I would just come into the second group. But as time goes on, maybe I might have thought it was unfair you know that I could have had that money and all this help sooner. (Case 2, female, 74, intervention)
Views on the intervention
Views on the intervention
I found someone was interested in me and come and seen me ... that makes a big difference. It's nice to get someone to discuss things with and ... assure me that I was entitled to this ... I couldn't believe it. (Case 17, female, 84, weekly income increased by 52%)
You talk to someone like [the advice worker] you felt that in two or three minutes she knew her business ... she would give you a very well-informed and reliable answer immediately. (Case 19, female, 82, weekly income increased through another welfare rights service prior to this study)
I wouldn't have felt any animosity ... I was already prepared for the fact that I wouldn't be entitled to anything. (Case 8, male, 77, no gain)
When [welfare rights officer] asked about finances and everything, I've nothing to hide, so I just told her the truth. (Case 23, female, 74, no gain)
... other than saying I wasn't prepared to divulge what finances we had, apart from that she didn't press us. She said excess £30,000, I said yes, you could say that. (Case 12, male, 79, weekly income increased by 18%)
I'm used to taking all my papers up to the Council Office for the rent benefit and that, so, I'm not stupid, it didn't worry me. I know it does worry a lot of people, but I'm not stupid. (Case 14, female, 82, weekly income increased by 51%)
The process of the intervention required the disclosure of a considerable amount of information about personal finances. No-one objected to this in the context of the welfare rights assessment, although one participant did not wish to disclose actual amounts. Most participants saw this as a necessary part of the process, and many were already required to disclose finances for other matters, such as qualifying for council tax rebates. Participants highlighted the fact that the service made them feel relaxed, at ease and confident about applying for benefits.
Impact of the intervention
Impact of additional resources
I couldn't bear it [not having a car] ... just to have the car there is such a godsend and its gets you out anyway because my walking isn't so good. (Case 22, female, 78, weekly income increase £19 (€28, $34))
I am eating more fruit and things that you couldn't of bought before ... you don't need to think about, can I afford it? You can, and as I say, I buy more fruit. (Case 3, female, 72, weekly income increase £57 (€84, $101))
We have been quite depressed last month ... now we can afford petrol to go up the coast in the car. Tank full of petrol if we feel like it ... we cheered ourselves up ... whereas it would have been another miserable day ... it did us good. (Case 15, female, 64, weekly income increase £72 (€106, $128))
Capacity to cope with crises
We have always been in the position where we have never had an awful lot of cash in the bank ... if we got an emergency we have got something there for it now. (Case 7, male, 75, weekly income increase £39 (€58, $69))
Peace of mind
[it makes] all the difference in the world. From being worried about bills to not being worried, and to be able to keep the heating on and not bother when the bill mounts up a bit ... but now I needn't worry ... it's the best help of all really, the lack of worry about finance. The worst thing for me that depressed me and got me down more than anything was worrying about not being able to pay bills. (Case 19, female, 82, income increased through another welfare rights service prior to this study)
Main findings and interpretation
Participation in this RCT was regarded as a positive experience by most interviewees; the intervention was highly regarded and had wide-ranging impacts, particularly for those who gained new benefits.
Participants' perspectives in RCTs have generally been measured by using attitude questionnaires with a view to improving participation rates. Although limited by the questionnaire format, such studies show that the main reasons given for anticipated participation in trials were altruismand personal benefit, and major disadvantages identified were time involved and problems with travel[25, 26]. Altruism, generating new knowledge through research and personal benefit were the main reasons given for participation in this study. Because of its domiciliary design, the study did not require participants to travel, something that many, particularly those with limited mobility or sensory impairments, appreciated and which may have increased recruitment and reduced attrition. Our study also suggests that the involvement of a trusted health professional in the recruitment can increase participation rates.
Although every effort was made to ensure that consent was fully informed, it became clear that a number of people did not understand the nature of the intervention or were not fully aware that they were being randomised. This occurred despite ethical committee approved information leaflets that were regarded as clear and a good reflection of participants' experiences in the trial. This has also been found in clinical research. With the exception of two participants, no-one was unhappy about randomisation to the control condition which involved waiting six months for welfare rights advice. Participants rationalised this in three main ways: firstly that it was a fair way to 'ration' a service that was otherwise not usually available; secondly, that people must face the consequences of their informed consent; and, thirdly, a degree of fatalism and/or ignorance about group allocation. Participants in this study appeared to have a more relaxed attitude to randomisation than has been reported in clinical trials. This could be explained by differences in the nature and context of clinical and social interventions, the former concerning treatment for an existing condition with anticipated health benefits; the latter, as in this case, concerning access to a potentially valuable service, but with unknown health effects. This study demonstrates that it was acceptable to randomise the intervention with a delay of six months to the control group. However, the pilot RCT findings suggest that this is unlikely to be adequate time for any measurable health benefits to become evident . A longer delay between the study and control group's receipt of the intervention raises ethical issues that require further consideration .
Strengths and limitations
The qualitative data were derived from a sub-sample of the RCT participants and it is possible that only those willing to be interviewed further held such positive views. Comparison between the quantitative and qualitative samples on a number of social and economic variables indicated similarities according to age, proportion of those living alone/as a couple, council tax band and long term limiting illness, suggesting that both samples were not qualitatively different [18, 28].
The effects of the intervention were wide-ranging and positively regarded by participants. However, this was not mirrored by the outcome measures used in the pilot RCT. Despite measuring a wide range of physical, psychological and social outcomes, the pilot RCT found little or no differences between intervention and control groups, or between those who did and did not receive additional resources suggesting that the intervention had no impact on these outcome measures [18, 29]. It is not uncommon for qualitative and quantitative studies to produce divergent findings [2, 30, 31] and it is likely that each method, with its different epistemological underpinnings, captured different aspects of phenomena under investigation. The qualitative approach enabled participants to give an account of the various ways in which the intervention impacted on their lives, such as increased independence and improved quality of life, which were not explicitly measured in the pilot RCT and are challenging to capture quantitatively. This qualitative study has thus not only provided valuable information about outcome measures for a future trial, but helped with the overall interpretation of the pilot RCT. The study also suggests that inclusion of a qualitative component will help to illuminate the process and outcome of a future trial.
We thank the study participants, Rosemary Bell, Nick Whitton and Jenny Dover of Newcastle City Council Welfare Rights Service, Laura Stokoe for administrative support, Astrid MacIntyre and Paula Carolyn for transcribing interviews, Joe Kai for advice with the topic guide, Graham Scambler for helpful advice throughout, and Elaine McColl, Robbie Foy, Alicia O'Cathain and Katie Brittain for helpful comments on the paper.
Funding. The research was funded by the Department of Health Policy Research Programme and supported by the University of Newcastle upon Tyne, Newcastle Primary Care Trust and Newcastle City Council Social Services Directorate. The views expressed in this paper are those of the authors and not necessarily those of the Department of Health.
- Donovan J, Mills N, Smith M, Brindle L, Jacoby A, Peters T, Frankel S, Neal D, Hamdy F: Improving design and conduct of randomised trials by embedding them in qualitative research: (ProtecT) study. BMJ. 2002, 325: 766-769. 10.1136/bmj.325.7367.766.View ArticlePubMedPubMed CentralGoogle Scholar
- Cox K: Assessing the quality of life of patients in phase I and II anti-cancer drug trials: interviews versus questionnaires. Soc Sci Med. 2003, 56: 921-934. 10.1016/S0277-9536(02)00100-4.View ArticlePubMedGoogle Scholar
- Featherstone K, Donovan JL: "Why don't they just tell me straight, why allocate it?" The struggle to make sense of participating in a randomised controlled trial. Soc Sci Med. 2002, 55: 709-719. 10.1016/S0277-9536(01)00197-6.View ArticlePubMedGoogle Scholar
- Harth SC, Thong YH: Parental perceptions and attitudes about informed consent in clinical research. Soc Sci Med. 1995, 41: 1647-1651. 10.1016/0277-9536(95)00058-F.View ArticlePubMedGoogle Scholar
- Rousseau N, McColl E, Newton J, Grimshaw J, Eccles M: Practice based, longitudinal, qualitative interview study of computerised evidence based guidelines in primary care. BMJ. 2003, 326: 314-322. 10.1136/bmj.326.7384.314.View ArticlePubMedPubMed CentralGoogle Scholar
- Wanless D: Securing Good Health for the Whole Population. 2004, London, HMSOGoogle Scholar
- Petticrew M, Roberts H: Evidence, hierarchies, and typologies: horses for courses. J Epidemiol Community Health. 2003, 57: 527-529. 10.1136/jech.57.7.527.View ArticlePubMedPubMed CentralGoogle Scholar
- Paris JAG, Player D: Citizens advice in general practice. BMJ. 1993, 306: 1518-1520.View ArticlePubMedPubMed CentralGoogle Scholar
- Hoskins RAJ, Smith LN: Nurse-led welfare benefits screening in a general practice located in a deprived area. Public Health. 2002, 116: 214-220.View ArticlePubMedGoogle Scholar
- Adams J, White M, Moffatt S, Howel D, Mackintosh J: A systematic review of the health, social and financial impacts of welfare rights advice delivered in health care settings. BMC Public Health. 2006, 6: 81-10.1186/1471-2458-6-81.View ArticlePubMedPubMed CentralGoogle Scholar
- Acheson D: Independent Inquiry into Inequalities in Health. 1998, London, HMSOGoogle Scholar
- Moffatt S, White M, Stacy R, Downey D, Hudson E: The impact of welfare advice in primary care: a qualitative study. Critical Public Health. 2004, 14: 295-309. 10.1080/09581590400007959.View ArticleGoogle Scholar
- Galvin K, Sharples A, Jackson D: Citizens Advice Bureaux in general practice: an illuminative evaluation. Health and Social Care in the Community. 2000, 8: 277-282. 10.1046/j.1365-2524.2000.00249.x.View ArticlePubMedGoogle Scholar
- Emanuel J, Begum S: What do you advise doc? A Citizen's Advice Bureau in Primary Care in the West Midlands. 2000, Manchester, University of ManchesterGoogle Scholar
- Sherratt M, Jones K, Middleton P: A citizen's advice service in primary care: improving patient access to benefits. Primary Care Research and Development. 2000, 1: 139-146. 10.1191/146342300672823063.View ArticleGoogle Scholar
- Moffatt S, White M, Stacy R, Hudson E, Downey D: "If we had not got referred and got the advice, I don't know where we'd be, it doesn't bear thinking about". The impact of welfare advice provided in general practice. A qualitative study. 1999, Newcastle upon Tyne, University of Newcastle upon TyneGoogle Scholar
- Fleming B, Golding L: Evaluation of 4 CAT-funded Citizens' Advice Bureaux units. 1997, Birmingham, Soundings ResearchGoogle Scholar
- Mackintosh J, White M, Howel D, Chadwick T, Moffatt S, Deverill M, Sandell A: Randomised controlled trial of welfare rights advice accessed via primary health care: pilot study. BMC Public Health.
- Denzin NK, Lincoln YS: Handbook of Qualitative Research. 2000, London, SageGoogle Scholar
- Ritchie J, Lewis J: Qualitative Research Practice. A Guide for Social Scientists. 2003, London, SageGoogle Scholar
- Silverman D: Doing qualitative research. 2000, London, SageGoogle Scholar
- Clayman SE, Maynard DW: Ethnomethodology and conversation analysis. Situated Order: Studies in the Social Organisation of Talk and Embodied Activities. Edited by: Have PT and Psathas G. 1994, Washington, D.C., University Press of AmericaGoogle Scholar
- Suchanek Hudmon K, Stolzfus C, Chamberlain RM, Lorimor RJ, Steinbach G, Winn RJ: Participants' perceptions of a phase I colon cancer chemoprevention trial. Control Clin Trials. 1996, 17: 494-508. 10.1016/S0197-2456(96)00063-3.View ArticleGoogle Scholar
- Daugherty C, Ratain MJ, Grochowski E, Stocking C, Kodish E, Mick R: Perceptions of cancer patients and their physicians involved in phase I trials. J Clin Oncol. 1995, 1: 1062-1072.Google Scholar
- Schron EB, Wassertheil-Smoller S, Pressel S: Clinical trial participant satisfaction: survey of SHEO enrollees. J Am Geriatr Soc. 1997, 45: 934-938.View ArticlePubMedGoogle Scholar
- Matson ME, Curb JD, McArdle R: Participation in a clinical trial: the patients' point of view. Control Clin Trials. 1985, 6: 156-167. 10.1016/0197-2456(85)90121-7.View ArticleGoogle Scholar
- Thomson H, Hoskins R, Petticrew M, Ogilvie D, Craig N, Quinn T, Lindsey G: Evaluating the health effects of social interventions. BMJ. 2004, 328: 282-285. 10.1136/bmj.328.7434.282.View ArticlePubMedPubMed CentralGoogle Scholar
- White M, Moffatt S, Mackintosh J, Howel D, Sandell A, Chadwick T, Deverill M: Randomised controlled trial to evaluate the health effects of welfare rights advice in primary health care: a pilot study. Report to the Department of Health, Policy Research Programme. 2006, Newcastle upon Tyne, University of Newcastle upon TyneGoogle Scholar
- Moffatt S, White M, Mackintosh J, Howel D: Using quantitative and qualitative data in health services research - what happens when mixed method findings conflict? [ISRCTN61522618]. BMC Health Services Research. 2006, 6:Google Scholar
- Brannen J: Mixing Methods: qualitative and quantitative research. 1992, Aldershot, AshgateGoogle Scholar
- Campbell R, Quilty B, Dieppe P: Discrepancies between patients' assessments of outcome: qualitative study nested within a randomised controlled trial. BMJ. 2003, 326: 252-253. 10.1136/bmj.326.7383.252.View ArticlePubMedPubMed CentralGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/6/163/prepub