Table 4 Coding instrument
Applicability | 1 | Refers to unrelated age limits |
3 | Mixes discussion of 21 and other age limits | |
5 | Clearly refers to population (21 and under) | |
Opinions v. facts | 1 | Opinions offered as facts without qualification |
3 | Mix of citations and opinions offered as facts | |
5 | All factual claims either quoted or cited | |
Validity | 1 | Research misrepresented |
2 | No reference to research | |
3 | Refers to study(ies) but no further discussion | |
4 | Makes unqualified claim of validity “prestigious” or “irrefutable” | |
5 | Some discussion of why study(ies) good (systematic review, weight of evidence) | |
Magnitude | 1 | No mention of effects or effects misrepresented |
2 | Effects implied but not explicitly mentioned | |
3 | Refers to “reduction” or “increase” without specifics | |
4 | Mixes exact figures with general claims | |
5 | Exact percentages or lives saved estimates | |
Precision | 1 | No indication of whether results are due to chance |
3 | Some effort to link study design to credibility of results | |
5 | Discusses alternative explanations, sampling, or omitted variable bias, etc. | |
Consistency | 1 | Potentially misleading selection of studies (e.g. “studies show”) |
2 | One study discussed | |
3 | Two studies discussed | |
4 | Three or more studies discussed | |
5 | Reference to body of evidence or to a systematic review (IOM report) | |
Consequences | 0 | No reference to consequences |
(Count #) | 1 | Affects youth smoking rate |
2 | Affects adult smoking rate | |
3 | Affects deaths from tobacco use | |
4 | Affects health care costs | |
5 | Affects sales or government revenue (e.g. taxes lost from reduced sales) | |
Global | 1 | Misleading |
2 | Evidence treated equally with opinion | |
3 | Some opinion included but weight of article is on evidence | |
4 | Evidence is focus of article but not explained | |
5 | Major claims supported by evidence and explained | |
Writer’s conclusion | Supports | |
Opposes | ||
No opinion given |