Skip to main content

Table 3 Non-randomised controlled study options and key difficulties

From: Challenges in evaluating Welfare to Work policy interventions: would an RCT design have been the answer to all our problems?

Study design

Disadvantages specific to study design

Key difficulty

Outcome

Option 1: Intervention group: engagea with PAS

Comparison group: active exposure but no engagement with PASb

Systematic difference between intervention and comparison group. IB recipients who engage with PAS are likely to be healthier and closer to a return to work than those who are aware of PAS but are not referred (by GP or self).

 

Selection bias leading to overestimate of health effects

Option 2:

Intervention group: engage with PASa

Comparison group: not exposed to PASd

Need to identify suitable comparison group with respect to eligibility for referral to PAS (only 20% of those in the comparison GP practices would be 'comparable' to the intervention group)

How to determine suitable controls: option to ask GP in comparison practices to tell us who, in principle, they would refer to PAS. This requires a high level of involvement by GPs not offering PAS

Possible selection bias depending on ability to match controls

 

Cluster level influences need to be accounted for

  

Option 3:

Intervention group: engage with PASa

Comparison groups:

(1) Not exposed to PASd

(2) passive exposure to PASc

Need to identify suitable comparison group with respect to eligibility for referral to PAS (only 20% of those in the comparison GP practices would be 'comparable' to the intervention group)

How to determine suitable controls (this will be aided by information from the group who are exposed but do not engage)

Possible selection bias depending on ability to match controls

 

Cluster level influences need to be accounted for

  
  1. a Engagement with PAS: actually met with a PAS advisor
  2. b Active exposure to PAS: GP refers patient to PAS but patient does not take up the referral
  3. c Passive exposure to PAS: Registered patient with primary care practice participating in PAS service - may or may not be aware of PAS
  4. d Not exposed to PAS: Registered patient with primary care practice not participating in PAS service