Skip to main content

Advertisement

Table 1 Criteria used to appraise study quality

From: The views of young children in the UK about obesity, body size, shape and weight: a systematic review

Criterion (with guidance for reviewers)
1 Were steps taken to increase rigour in the sampling?
Consider whether:
*the sampling strategy was appropriate to the questions posed in the study (e.g. was the strategy well reasoned and justified?);
*attempts were made to obtain a diverse sample of the population in question (think about who might have been excluded; who may have had a different perspective to offer);
*characteristics of the sample critical to the understanding of the study context and findings were presented (i.e. do we know who the participants were in terms of, for example, basic socio-demographics, characteristics relevant to the context of the study, etc.).
2 Were steps taken to increase rigour in the data collected?
Consider whether:
*data collection tools were piloted/(and if quantitative) validated;
*(if qualitative) data collection was comprehensive, flexible and/or sensitive enough to provide a complete and/or vivid and rich description of people's perspectives and experiences (e.g. did the researchers spend sufficient time at the site/with participants? Did they keep 'following up'? Was more than one method of data collection used?);
* steps were taken to ensure that all participants were able and willing to contribute (e.g. processes for consent, language barriers, power relations between adults and children/young people).
3 Were steps taken to increase rigour in the analysis of the data?
Consider whether:
* data analysis methods were systematic (e.g. was a method described/can a method be discerned?);
*diversity in perspective was explored;
* (if qualitative) the analysis was balanced in the extent to which it was guided by preconceptions or by the data);
*the analysis sought to rule out alternative explanations for findings (in qualitative research this could be done by, for example, searching for negative cases/exceptions, feeding back preliminary results to participants, asking a colleague to review the data, or reflexivity; in quantitative research this may be done by, for example, significance testing).
4 Were the findings of the study grounded in/supported by the data?
Consider whether:
*enough data are presented to show how the authors arrived at their findings;
*the data presented fit the interpretation/support claims about patterns in data;
*the data presented illuminate/illustrate the findings;
*(for qualitative studies) quotes are numbered or otherwise identified and the reader can see that they don't just come from one or two people.
5 Please rate the findings of the study in terms of their breadth and depth.
Consider whether:
(NB: it may be helpful to consider 'breadth' as the extent of description and 'depth' as the extent to which data has been transformed/analysed);
*a range of issues are covered;
* the perspectives of participants are fully explored in terms of breadth (contrast of two or more perspectives) and depth (insight into a single perspective);
*richness and complexity has been portrayed (e.g. variation explained, meanings illuminated);
*there has been theoretical/conceptual development.
6 To what extent does the study privilege the perspectives and experiences of children?
Consider:
* whether there was a balance between open-ended and fixed response options;
*whether children were involved in designing the research;
* whether there was a balance between the use of an a priori coding framework and induction in the analysis;
*the position of the researchers (did they consider it important to listen to the perspectives of children?);
* whether steps were taken to assure confidentiality and put young people at ease.
7 Overall, what weight would you assign to this study in terms of the reliability/trustworthiness of its findings?
Guidance:
Think (mainly) about the answers you have given to questions 1 to 4 above.
8 What weight would you assign to this study in terms of the usefulness of its findings for this review?
Guidance:
Think (mainly) about the answers you have given to questions 5 and 6 above and consider:
*the match between the study aims and findings and the aims and purpose of the synthesis;
*its conceptual depth/explanatory power.