Skip to main content

Table 1 Reworded external validity items and extent of reporting by item (n = 39 studies) *

From: External validity in healthy public policy: application of the RE-AIM tool to the field of housing improvement

   Large extent Some extent Unclear Not at all
A Population: Representativeness of target population, setting & reach of intervention
1 Are data presented on variations in participation rate in improved housing interventions by a) setting b) delivery staff/organisations c) residents (for intervention among general target population not study area) 0 0 0 39
2 Is the intended target audience for adoption clearly described 11 18 8 2
3 Is the intended target setting for adoption clearly described? 4 27 5 3
4 Is there analysis of the baseline socio-demographic and ‘condition tested’ (health status) of evaluation participants versus non-participants? (relating to evaluation population only) 0 0 2 37
B Intervention: Implementation & adaptation
5 Are data presented on consistency of implementation of intervention & its different components? 0 2 2 35
6 Are data presented on the level of training of experience required to deliver the programme or quality of implementation by different types of staff? 0 1 1 37
7 Is information reported on whether/how the intervention is modified to individuals/households within the study? 5 6 0 11
8 Are data presented on mediating factors or processes (mechanisms) through which the intervention had an impact? 2 12 4 21
C Outcomes for decision making
9 Are the reported health (even if only one measure of health is comparable) outcomes comparable to wider policy/other studies? 23 14 0 2
10 Have additional outcomes of potential adverse impacts been reported? e.g. socio-economic impacts 4 21 1 13
11 Have authors demonstrated consideration of variation in reported health outcomes (key outcome of interest) by population sub-groups, or intervention setting/delivery staff? 2 4 1 32
12 Is there sensitivity analysis of dose–response/threshold level required to observe health effect (effect on key outcome of interest not proxies)? 3 4 1 31
13 Are data on costs presented? Are standard economic/accounting methods used? 2 19 0 18
D Maintenance and institutionalisation of intervention
14 Are long term effects reported? (12 months or longer since exposure to the intervention) 10 13 4 11
15 Are data reported on the sustainability (or reinvention or evolution) of programme implementation and intervention, at least 12 months after the formal evaluation? 0 0 0 29
16 a Is the drop-out rate/attrition reported? 19 (Yes) 10 (N/A)
16 b Are data on attrition by baseline health status of dropouts reported and are analyses conducted of the representativeness of remaining sample at time of final follow-up (or main follow-up time point- as appropriate)? 0 0 0 29 (10 N/A)
  1. (adapted from Green LW, Glasgow RE. Evaluating the Relevance, Generalization, and Applicability of Research: Issues in External Validation and Translation Methodology. Eval Health Prof 2006;29(1):126–153.)
  2. * see Additional file 1 for full details of external validity assessment tool.