Table 1 Modified GRADE approach for evaluating the quality of evidence of meta-analysis data subgroups

 1. Individual study risk-of-bias rating and reporting limitations

One point deducted for each criterion where conditions are met. Sensitivity analysis considered to have appreciable impact if range of values changed estimates by >20 % or changed significance of overall effect.

  a) >50 % of trials had an unclear or high overall risk-of-bias rating

a) = −1 b) = −1

  b) Key assumptions/imputed values due to reporting limitations had appreciable impact on results in sensitivity analysis

 2. Inconsistency of direction and heterogeneity of findings among studies

Heterogeneity in the results was measured by I². Consistency considered when the individual study estimates in the meta-analysis all show the same direction of effect.

  a) Consistent direction of effect, but significant heterogeneity

a) = −1 b) = −2

  b) Inconsistent direction of effect and significant heterogeneity

 3. Imprecision of effect estimates

a) = −1

Power calculations conducted assuming α = 0.05 and β = 0.2. For continuous outcomes, power calculated for a difference in means of 0.5 using a range of representative standard deviations from the meta-analysis subgroup. For dichotomous outcomes, power calculated using a relative risk reduction of 30 % and median control group risk from the meta-analysis subgroup.

  a) The total number of participants in the meta-analysis subgroup is less than that required by a conventional sample size calculation for a single adequately powered controlled trial

 4. Indirectness of individual study parameter as representative of target parameter

Indirectness indicates studies did not directly measure the target parameter of interest to the review question (e.g. food safety outcomes only reported as part of a combined score/scale with other constructs such as nutrition).

  a) >50 % of trials indirectly measure the intervention, population, comparison, or outcome

a) = −1 b) = −2

  a) >50 % of trials measure two or more of the above parameters indirectly

 5. Publication bias

a) = −1

This criterion can only be evaluated if publication bias assessment is possible based on the nature of the data (i.e. ≥10 studies, non-significant heterogeneity, and at least some of the studies have significant results).

  a) Detected or suspected in data subset

 1. Large magnitude of effect

a) = +1

Large effect considered at least a 2-fold reduction in risk.

  a) Large effect in the absence of plausible confounders and major threats to validity

 2. Results may have been underestimated due to the study design (e.g. population sampled)

a) = +1

E.g. intervention was tested only on individuals with prior food safety knowledge/training, and it is likely that a stronger effect would have been found if the intervention was tested in the general consumer population.

  a) Criterion present

 3. Dose-response gradient

a) = +1

Meta-regression dose variable represents >1 training course/session or multifaceted messaging interventions vs. a single course/session or provision of messaging materials through a single medium or exposure type.

  a) >50 % of trials identified a dose-response relationship OR dose identified as significant in meta-regression.