Downgrading criteria

1. Individual study riskofbias rating and reporting limitations
 
One point deducted for each criterion where conditions are met. Sensitivity analysis considered to have appreciable impact if range of values changed estimates by >20 % or changed significance of overall effect.

a) >50 % of trials had an unclear or high overall riskofbias rating

a) = −1 b) = −1

b) Key assumptions/imputed values due to reporting limitations had appreciable impact on results in sensitivity analysis

2. Inconsistency of direction and heterogeneity of findings among studies
 
Heterogeneity in the results was measured by I^{2}. Consistency considered when the individual study estimates in the metaanalysis all show the same direction of effect.

a) Consistent direction of effect, but significant heterogeneity

a) = −1 b) = −2

b) Inconsistent direction of effect and significant heterogeneity

3. Imprecision of effect estimates

a) = −1

Power calculations conducted assuming α = 0.05 and β = 0.2. For continuous outcomes, power calculated for a difference in means of 0.5 using a range of representative standard deviations from the metaanalysis subgroup. For dichotomous outcomes, power calculated using a relative risk reduction of 30 % and median control group risk from the metaanalysis subgroup.

a) The total number of participants in the metaanalysis subgroup is less than that required by a conventional sample size calculation for a single adequately powered controlled trial

4. Indirectness of individual study parameter as representative of target parameter
 
Indirectness indicates studies did not directly measure the target parameter of interest to the review question (e.g. food safety outcomes only reported as part of a combined score/scale with other constructs such as nutrition).

a) >50 % of trials indirectly measure the intervention, population, comparison, or outcome

a) = −1 b) = −2

a) >50 % of trials measure two or more of the above parameters indirectly

5. Publication bias

a) = −1

This criterion can only be evaluated if publication bias assessment is possible based on the nature of the data (i.e. ≥10 studies, nonsignificant heterogeneity, and at least some of the studies have significant results).

a) Detected or suspected in data subset

Upgrading criteria

1. Large magnitude of effect

a) = +1

Large effect considered at least a 2fold reduction in risk.

a) Large effect in the absence of plausible confounders and major threats to validity

2. Results may have been underestimated due to the study design (e.g. population sampled)

a) = +1

E.g. intervention was tested only on individuals with prior food safety knowledge/training, and it is likely that a stronger effect would have been found if the intervention was tested in the general consumer population.

a) Criterion present

3. Doseresponse gradient

a) = +1

Metaregression dose variable represents >1 training course/session or multifaceted messaging interventions vs. a single course/session or provision of messaging materials through a single medium or exposure type.

a) >50 % of trials identified a doseresponse relationship OR dose identified as significant in metaregression.
