Skip to main content

Table 3 Non-randomised controlled study options and key difficulties

From: Challenges in evaluating Welfare to Work policy interventions: would an RCT design have been the answer to all our problems?

Study design Disadvantages specific to study design Key difficulty Outcome
Option 1: Intervention group: engagea with PAS
Comparison group: active exposure but no engagement with PASb
Systematic difference between intervention and comparison group. IB recipients who engage with PAS are likely to be healthier and closer to a return to work than those who are aware of PAS but are not referred (by GP or self).   Selection bias leading to overestimate of health effects
Option 2:
Intervention group: engage with PASa
Comparison group: not exposed to PASd
Need to identify suitable comparison group with respect to eligibility for referral to PAS (only 20% of those in the comparison GP practices would be 'comparable' to the intervention group) How to determine suitable controls: option to ask GP in comparison practices to tell us who, in principle, they would refer to PAS. This requires a high level of involvement by GPs not offering PAS Possible selection bias depending on ability to match controls
  Cluster level influences need to be accounted for   
Option 3:
Intervention group: engage with PASa
Comparison groups:
(1) Not exposed to PASd
(2) passive exposure to PASc
Need to identify suitable comparison group with respect to eligibility for referral to PAS (only 20% of those in the comparison GP practices would be 'comparable' to the intervention group) How to determine suitable controls (this will be aided by information from the group who are exposed but do not engage) Possible selection bias depending on ability to match controls
  Cluster level influences need to be accounted for   
  1. a Engagement with PAS: actually met with a PAS advisor
  2. b Active exposure to PAS: GP refers patient to PAS but patient does not take up the referral
  3. c Passive exposure to PAS: Registered patient with primary care practice participating in PAS service - may or may not be aware of PAS
  4. d Not exposed to PAS: Registered patient with primary care practice not participating in PAS service