Key: 0 = low clarity and quality as assessed by the reviewer 1 = reasonable clarity and quality as assessed by the reviewer 2 = reflects a finding of high clarity and quality as assessed by the reviewer NC = not clear or not available from the paper | Bernath 2013 [24] | Bhate-Deosthali 2012 [44] | Doucet 2012 [43] | GHD Pty Ltd. 2015 [25] | Human Rights Watch 2015 [26] | Keesbury 2012 [27] | Kohli 2013 [46] | Manneschmidt 2009 [42] | Morel-Seytoux 2010 [28] | PHD Group 2012 [29] | Wessel 1997 [45] |
---|---|---|---|---|---|---|---|---|---|---|---|
1) Worth or relevance | |||||||||||
 1.1) Was this piece of work worth doing at all? | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 |
 1.2) Has it contributed usefully to knowledge? | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 |
2) Clarity of research question | |||||||||||
 2.1) If not at the outset of the study, by the end of the research process, was the research question clear? | 2 | 1 | 2 | 2 | 1 | 2 | 2 | 2 | 2 | 2 | 1 |
3) Appropriateness of the design of the question | |||||||||||
 3.1) Was an appropriate method used? | 1 | NC | 1 | 1 | 1 | 2 | 2 | 1 | 1 | 1 | 1 |
4) Context | |||||||||||
 4.1) Is the context or setting adequately described so that the reader could relate the findings to other settings? | 2 | 2 | 2 | 1 | 2 | 2 | 2 | 2 | 2 | 2 | 2 |
5) Sampling | |||||||||||
 5.1) Did the sample include the full range of possible causes or settings? | 0 | NC | 0 | NC | NC | 1 | 1 | NC | NC | NC | 1 |
 5.2) If appropriate, were efforts made to obtain data that might contradict or modify the analysis extending or modifying the sample? | 1 | NC | 0 | NC | NC | NC | NC | NC | NC | NC | NC |
6) Data collection and analysis | |||||||||||
 6.1) Were the data collection and analysis procedures systematic? | 1 | NC | 2 | NC | NC | 1 | 2 | 1 | 0 | 0 | 1 |
 6.2) Was an ‘audit trail’ provided? | 1 | 0 | 2 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 0 |
 6.3) How well did the analysis succeed in incorporating all the observations? | NC | NC | NC | NC | NC | NC | NC | 2 | 2 | NC | NC |
 6.4) Did the analysis develop concepts and categories capable of explaining key processes? | 2 | 1 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | NC | 2 |
 6.5) Was it possible to follow iteration between data and theory? | 2 | 0 | 2 | 1 | NC | 2 | 2 | 2 | 1 | 0 | 2 |
 6.6) Did the researcher search for disconfirming cases? | 0 | NC | 0 | NC | NC | NC | NC | NC | NC | NC | NC |
7) Reflexivity of the account | |||||||||||
 7.1) Did the researcher assess the likely impact of the methods used on the data obtained? | 0 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 0 |
 7.2) Were sufficient data included in the reports to provide sufficient evidence for readers to assess whether analytical criteria were met? | 0 | 1 | 2 | 2 | 2 | 2 | 2 | 1 | 1 | 0 | 2 |