Skip to main content

Table 2 Randomised controlled study options and key difficulties

From: Challenges in evaluating Welfare to Work policy interventions: would an RCT design have been the answer to all our problems?

Study design Disadvantages specific to study design Key difficulty Outcome
Recruitment into study by GP followed by randomisation Self-referral to PAS increases risk of contamination of comparison group Service is available external to the study Dilution bias
Underestimate of effect
  GPs may refer those who they think most in need/most likely to benefit - rather than recruit to the study Group being evaluated not representative of those using the service  
  This would half the flow of patients being referred to PAS PAS may not be sustainable  
  Requires high levels of co-operation from GP and PAS Resource implications for GPs/PAS  
Cluster randomisation Need to identify IB recipients in comparison practices who would be eligible for referral to PAS: it is likely that this would only be around 20% of the total sample Non-specific criteria for referral to service limits our capacity to identify an appropriate comparison group Possible selection bias depending on ability to match controls
  Cluster level differences need to be accounted for Requires high levels of collaboration with policy makers well before implementation of pilot Not possible given that PAS had been rolled out by the time of this evaluation