- Research article
- Open Access
A content analysis of popular media reporting regarding increases in minimum ages of legal access for tobacco
BMC Public Health volume 18, Article number: 1129 (2018)
In the late 20th century, US localities began increasing the minimum age of legal access (MLA) for tobacco from 18 to 21 years by enacting “Tobacco 21” ordinances. Although these policies have a strong evidence base and broad popular support, popular media coverage of tobacco control laws has not always been accurate. This study sought to determine if contemporaneous popular media reporting accurately reflected the scientific findings regarding increased tobacco MLAs.
We searched LexisNexis for popular media reports that (1) addressed proposed or enacted Tobacco 21 ordinances and were (2) published in English, (3) drawn from a US news source, and (4) written after January 2004. We conducted a content analysis for quality based on a validated measure of accuracy of reporting, the Index of Scientific Quality (ISQ), which allows assessment of articles by assigning scores ranging from 1 (lowest) to 5 (highest).
Searches yielded 378 articles; after screening for relevance and duplicates, 98 were included in the review. All studies identified through the keyword searches addressed Tobacco 21 policies. The average global score identifying the scientific quality of the articles was 2.98 of 5. Over three-quarters of the popular media articles addressing Tobacco 21 laws were written after a systematic review of these policies was released by the Institute of Medicine and approximately 4 in 10 cited findings from that review.
Popular media reports on Tobacco 21 laws demonstrated average overall quality and relied on both anecdotal and scientific evidence, in contrast to previous studies found that popular media reports on tobacco issues demonstrated low overall quality and relied primarily on anecdotal evidence. The systematic review of increased MLAs for tobacco written by the Institute of Medicine diffused quickly into popular reporting, suggesting that this type of evidence might improve research translation.
Tobacco use is the leading preventable cause of death in United States and the negative health consequences of tobacco use have been well established for decades . In 2015, approximately 4.7 million middle and high school students in the US were current tobacco smokers . Nine out of ten of smokers begin smoking before age 18, and smoking behavior among young adults is predictive of smoking in later years [1, 3, 4]. Despite evidence of tobacco industry marketing toward youth and young adults,  policies to reduce access to tobacco for this group have been limited in scope .
In the late 20th century, localities in the United States instituted renewed efforts to increase the MLA for tobacco from 18 to 21 years, generally referred to as “Tobacco 21” laws . These efforts resulted in a nearly 50% decrease in cigarette smoking rates among high-school students (13% to 7%) and a comparable decrease in store purchases of cigarettes (18% to 13%) . In 2015 the Institute of Medicine projected that increasing the MLA for tobacco to 21 years would reduce adult smoking by 12% and prevent 223,000 premature deaths . Tobacco 21 policies are popular: 70% of adults support raising the MLA for tobacco to 21 years, including a majority of adults in all demographic and smoking status categories . However despite clinical evidence and popular support, as of 2016 only two states (California and Hawaii) had increased their MLA for tobacco to 21 years; as of 2017, an additional three states (Maine, New Jersey, and Oregon) had done so .
Existing studies of research translation detail the process from the generation of research to its use by policymakers [9, 10]. These studies have identified the importance of systematic reviews in translating evidence into policy, [11,12,13,14] and note that dissemination strategies that involve contact with policymakers are critical [15,16,17,18] because most policymakers are not trained to interpret scientific research or rewarded for doing so [19, 20]. The nature of reporting affects public opinion, influences individual behavior, and plays a central role in the process of public health policy formation [21, 22]. Although partisanship, ideology, and maintaining consistent voting records all factor into policymakers’ decisions, the extent of public support for proposed policies offers critical information in making decisions about whether to enact such changes .
Media misunderstanding of research findings is common . Multiple studies report that journalists translate research evidence poorly, particularly during novel events [25,26,27]. Past studies suggest that the accuracy of research translation by journalists covering tobacco issues has been inconsistent [28, 29]. The limited research on the scientific accuracy of popular reporting on tobacco has led to calls for additional research in this area [30,31,32,33].
To address this gap, we sought to assess the accuracy of popular media reporting on Tobacco 21 laws. The coverage of proposed increases in MLAs for tobacco offers particular insight in understanding research translation because it addresses two issues anticipated to affect the accuracy of popular media reports: novelty and systematic reviews. Tobacco 21 policies became relevant over a limited time period; the issue first became relevant in the 21st century after the passage of a Tobacco 21 ordinance in Needham, Massachusetts in 2005 . In March 2015, the US Institute of Medicine (IOM) published a systematic review of the effects of increasing MLAs for tobacco . Following the publication of this report Hawaii passed a Tobacco 21 law in June 2015, and California passed a similar law in June 2016 . Consistent with existing research, we hypothesized that (1) popular reporting on Tobacco 21 laws would rely heavily on anecdotal evidence; and (2) the publication of the IOM report would lead to higher quality popular media reports.
We conducted a content analysis of popular news articles that addressed increased MLAs for tobacco. We focused on articles in the public domain that were most likely to be easily found by individuals who were inexperienced with traditional academic research methods. To identify these reports, one of the authors (JH) searched the LexisNexis database for newspaper and magazine articles with the assistance of a university librarian. The search was conducted in May 2016 and relied on relevant keywords: “smoking” AND “tobacco” AND (“smoking age” OR “legal age” OR “minimum age” OR ((“teen age” OR “adolescent”) AND “tobacco control”) or “tobacco 21”). We included articles from newspapers and newswires that were (1) published in English, (2) drawn from a US news source, and (3) written after January 2004. This start date was chosen because it was one year prior to the first local US implementation of a Tobacco 21 policy in the 21st century. We excluded duplicate articles and articles that assessed smoking cessation and other clean air policies.
The following information was extracted from each article or website by one reviewer (JH):
Title of the article
Publication type (e.g., newspaper, magazine article, wire service stories)
We relied on a validated instrument created by Oxman et al.,  the Index of Scientific Quality (ISQ), to assess the quality of popular media reports. The ISQ index uses a five-point scale, with 1 corresponding to the lowest level of quality and 5 corresponding to the highest level of quality. A score of 4 or 5 indicates clear references to evidence, while a score of 2 or 3 represents partly or definitely unclear references to evidence. An ISQ score of 1 is assigned to criteria where the evidence base is potentially misleading. We modified the ISQ coding instrument to reflect outcomes relevant to Tobacco 21 laws. The applicability measure restricted the topic specifically to MLAs for tobacco; the validity measure relied both on specific terms (e.g. “prestigious” used as a marker for quality) and mention of systematic reviews; the magnitude measure included measures of health outcomes related to tobacco; the consequences considered health outcomes specific to tobacco such as smoking rates and costs. Details regarding the coding of each content area are provided in the Appendix. In addition, the instrument was expanded so that both coders made a judgment regarding whether the article, taken overall, claimed that increasing the tobacco sales age to 21 was effective, ineffective, or took no position. We used the IOM report as a gold standard for assessing reporting of relevant research.
The instrument covered the following content areas:
Applicability: Describes whether or not the author clearly refers to the affected population (21 and under)
Opinions versus Facts: Describes whether or not facts are clearly distinguished from opinions
Validity: Describes whether or not the assessment of the credibility (validity) of the evidence is clear and well-founded (not misleading)
Magnitude: Describes whether or not the strength or magnitude of the findings (effects on smoking rate, health, or costs) that are the main focus of the article are clearly reported
Precision: Describes whether or not the author provides a clear and well-founded (not misleading) assessment of the precision of any estimates that are reported or of the probability that any of the reported findings might be due to chance
Consistency: Describes whether or not the consistency of the evidence (between studies) is considered and whether the assessment is well-founded (not misleading)
Consequences: Describes whether or not all of the important consequences (youth and adult smoking rates, deaths from tobacco use, health care costs, sales and government revenue) of concern relative to the central topic of the report are identified
Global: Describes the overall scientific quality of the report
The analysis of article quality relied on the mean quality scores in each category identified by the ISQ, with subgroup analyses conducted for articles published before and after the release of the IOM report. In coding for content, both authors reviewed each article using the instrument, working independently. Cohen’s κ was run to assess interrater reliability for each ISQ quality criteria. κ was interpreted by the guidelines from Altman (1991) in which a κ score of 0.00–0.20 indicates poor agreement, 0.21–0.40 indicates fair agreement, 0.41–0.60 indicates moderate agreement, 0.61–0.80 indicates good agreement, and 0.81–1.00 indicates very good agreement . Agreement was good for applicability and consistency; moderate for consequences and global, fair for opinions versus facts, validity, and precision, and poor for magnitude. Coding discrepancies in all categories were discussed and resolved by consensus.
The initial database searches identified 378 popular media articles. One of the authors (JH) screened these articles for relevance. Eighty-five articles were identified as duplicates based on title, word count, and preview of the first three lines, and were excluded from the analysis. An additional 162 articles were removed from analysis because they did not meet the inclusion criteria based on title and preview of the first three lines. The remaining 134 articles were eligible for full-text review by both authors. After reading these articles in full, an additional 36 were identified to be either duplicates or to not meet the inclusion criteria by consensus of both authors, leaving 98 articles included in the final analysis. The screening process is outlined in Fig. 1.
Publication dates ranged from 2006 to 2016. Eighty percent of the articles did not take a position on Tobacco 21 laws; of the remaining articles, 16% supported the policies and 4% opposed them, as shown in Table 1. The majority of articles (82%) were published after 2015, with only 18 articles published before 2015. An increase in reporting on Tobacco 21 was correlated with the introduction of Hawaii Senate Bill 1030 in January 2015, which first proposed to raise the state’s MLA for tobacco to 21 years.
Quality scores by content area
Table 2 provides the mean values and SDs for each of the ISQ quality criteria.
Each article clearly stated that it considered Tobacco 21 policies so applicability received an average score of 5, the highest ranking for all criteria. Although the search strategy was designed to identify articles addressing Tobacco 21 laws, previous research has found that popular media articles do not always accurately reference policies that are putatively being reported or assessed.
Opinion v. facts
Distinction of opinion v. facts averaged 3.92, the second highest ranking for all criteria, indicating that articles were more evidence-based than opinion-based. Of the total 98 articles, 35 (36%) received a 5, indicating all factual claims were quoted or cited, and two received a 1, which meant opinions were offered as facts without qualification.
Validity represented the journalist’s assessment of the quality of evidence used the article. A score of 1 indicates that research was misrepresented, 2 that research was not referenced, 3 that studies were presented without discussion of their quality, 4 that the article made unqualified claims, and 5 that there was some discussion about why a study was “good” such as a reference to the weight of evidence. The articles scored an average of 2.83 for validity, a score representing average quality.
The magnitude of findings, which referred to the extent to which claims about effects were anchored with data averaged 3.84, the third highest quality ranking across criteria, suggesting that the articles made both general and specific claims about the potential effects of a Tobacco 21 policy. A score of 1 indicated that effects either were not mentioned or were misrepresented, 2 that effects were implied but not explicitly mentioned, 3 that effects were discussed in general terms, 4 that exact figures assessing outcomes were mixed in with general claims, and 5 that the article relied on exact percentages or estimates of the numbers of lives saved.
Assessment of the precision of results due to study design scored an average of 1.08, the lowest ranking for all criteria; a score of 1 indicated there was no indication of whether results were due to chance, 3 that there was some effort to link study design to credibility, and 5 that there was an explanation of study design.
Consistency of evidence between studies, referencing the number of studies discussed and the accuracy with which they represented the state of contemporaneous research, scored 2.79 for all articles, suggesting average quality. Articles that did not cite a specific study or that used a potentially misleading source of data were assigned a score of 1, while discussions of one, two, or three or more studies were scored 2, 3, and 4 respectively. Articles that referred to a systematic review, such as the IOM report, were scored 5.
We tallied the number of consequences relevant to Tobacco 21 that were mentioned in articles, specifically potential effects on smoking rates, deaths from tobacco use, health care costs, and sales of tobacco and/or tax receipts. On average, 2.26 consequences were listed, with 12 articles listing 4 or more potential effects of the policy and 19 articles listing either one potential effect or none.
The average global score identifying the scientific quality of the articles was 2.98 of a potential 5, representing average quality. Misleading articles scored were scored as 1, those that treated evidence equally with opinion scored 2, those that included some opinion but had more weight on evidence scored 3, those that presented claims that were evidence focused but not explained scored 4, and articles in which major claims were supported by evidence and explained scored 5.
Before and after the IOM report
The 2015 report by the Institute of Medicine found that increasing the MLA for tobacco products would prevent or delay use of such products by adolescents, improve population health, and reduce tobacco-related deaths. Table 2 also provides a comparison of the mean values and SDs prior to and after the IOM report. Our review found that 76 (78%) articles were written after the report was released; of these, 43% cited findings from the IOM’s report. After the release of the IOM report the quality scores for consistency, which represented the number of studies discussed and their representation of current research, improved from 1.95 to 3.04; this difference was statistically significant (p < 0.001). Scores also increased for magnitude and consequences but the differences were not statistically significant. Scores decreased for opinions v. facts, validity, and precision; these differences were also not statistically significant.
Nature of arguments
Proponents and opponents of Tobacco 21 policies included in the articles used different types of arguments, with proponents focused on outcomes and opponents focused on ideological claims, as shown in Table 3.
Supporters of Tobacco 21 policies primarily referenced scientific studies that focused on the prevalence of smoking and the health consequences of increasing the MLA. Among the five major consequences analyzed, the impact on youth smoking rate (80% of articles) and deaths due to smoking (51%) were the most frequently mentioned effects. Articles also referred to effects on adult smoking rate (43%), health care cost (35%), and revenue (16%), however these issues were discussed less frequently. Approximately 7% of articles cited statistics that demonstrated strong public support for the policies, particularly among current or former smokers. Supportive claims were typically made by public health professionals or legislators speaking on tobacco-related issues, rather than the general public.
Consistent with past arguments against stronger tobacco control policies, the concerns expressed by opponents of Tobacco 21 policies primarily focused on the individual rights to make decisions rather than on research findings regarding the effects of the policies or on tobacco industry marketing to youth. About 38% of articles claimed that Tobacco 21 laws would impede individual decision making; opponents argued that increasing the MLA was tantamount to creating a “nanny state” that interfered with the decisions of young adults. These claims often focused on extended analogy; 33% of articles stated that if people were old enough to vote and enlist in the military, they were old enough to smoke. In 15% of articles, opponents of the policies speculated that despite research showing that Tobacco 21 policies had resulted in reduced tobacco use, young people might circumvent the law by purchasing tobacco in neighboring jurisdictions with lower MLAs or obtain tobacco from family and friends. In 15% of the articles, critics of the policies attempted to shift focus from the potential of an increased MLA to save lives and reduce health care costs by making counterclaims that such policies would have a negative financial impact on small businesses and government by reducing tobacco sales and tobacco tax revenue.
This study provides the first assessment of popular media coverage addressing laws that increased tobacco MLAs to 21 years. Consistent with previous studies, we hypothesized that popular reporting would demonstrate low overall quality and rely on anecdotal evidence. Instead, we found that media reports on this topic were of average quality and relied on both anecdotal and scientific evidence. Our content analysis found that applicability, opinion v. facts, and magnitude were the highest scoring categories, indicating that articles were focused on Tobacco 21 policies and mostly reported facts and figures assessing their effects. The views of public health advocates were better represented than those of the tobacco industry. However, when reporting on claims made by opponents to the policies, articles disproportionately relied on their anecdotes and speculation, rather than research findings. We found that measures of precision were consistently weak, suggesting that the concept of statistical significance and the role of chance remains difficult to communicate through popular media reports. This finding is consistent with previous research; one study suggested that reporters preferentially cover medical research with weaker methodology .
We also hypothesized that publication of the IOM report would lead to higher quality popular media reports. The scores for consistency showed a statistically significant increase in quality, suggesting that journalists recognize the value of systematic reviews over individual studies. These findings also appeared in claims made by policy advocates; in contrast, policy opponents lacked comparable evidence and relied on ideological or anecdotal claims. Our results are consistent with previous research that attempted to train consumer advocates to better understand and communicate research. The majority of advocates believed that systematic reviews were more reliable than individual studies after training, even though less than half stated that they were comfortable with analyzing research methods and designs . The differences in scores for other measured categories of scientific quality were not statistically significant.
This research has limitations. We may not have identified all published articles in LexisNexis, given that our inclusion criteria limited our selection to articles published from January 2004 to March 2016. Data collection stopped shortly after the passage of California law, making it possible that later articles were missed. Coverage may increase again if additional states propose and enact Tobacco 21 laws. In addition, we focused on written media, and did not assess reporting in television, radio, or social media. Finally, our findings with respect to Tobacco 21 laws may not be generalizable to other aspects of tobacco control.
Our findings provide new evidence about translation of clinical research into community settings, and help fill a gap in understanding the accuracy of media reports on tobacco issues. Consistent with the continued concern about the quality of popular media reporting on scientific research, we found that reporting on Tobacco 21 policies was of average quality and inconsistently cited data from scientific studies. Our results also show that while a systematic review addressing this topic diffused relatively quickly into popular reporting, it was not always referenced. Nonetheless these findings suggest that systematic reviews appear to improve popular media reporting with respect to communicating the overall state of research evidence. Development of policy-relevant systematic reviews may be a useful strategy to help reduce tobacco-related disease by communicating information about research evidence to policymakers and the public.
Institute of Medicine
Index of Scientific Quality
Minimal age of legal access
Centers for Disease Control and Prevention. The health consequences of smoking—50 years of progress: a report of the Surgeon General. Atlanta: US Department of Health and Human Services; 2014.
Singh T. Tobacco use among middle and high school students—United States, 2011–2015. MMWR Morb Mortal Wkly Rep. 2016;65:361-367.
Apollonio DE, Glantz SA. Minimum ages of legal access for tobacco in the United States from 1863 to 2015. Am J Public Health. 2016;106(7):1200.
Institute of Medicine. In: Bonnie RJ, Stratton K, Kwan LY, editors. Public Health Implications of Raising the Minimum Age of Legal Access to Tobacco Products. Washington: National Academies Press (US); 2015.
Ling PM, Glantz SA. Why and how the tobacco industry sells cigarettes to young adults: evidence from industry documents. Am J Public Health. 2002;92(6):908.
Tobacco Twenty-One [https://tobacco21.org/]. Accessed 5 Oct 2016.
Kessel Schneider S, Buka SL, Dash K, Winickoff JP, O'Donnell L. Community reductions in youth smoking after raising the minimum tobacco sales age to 21. Tob Control. 2016;25(3):355.
King BA, Jama AO, Marynak KL, Promoff GR. Attitudes toward raising the minimum age of Sale for tobacco among U.S. adults. Am J Prev Med. 2015;49(4):583–8.
Lavis JN, Posada FB, Haines A, Osei E. Use of research to inform public policymaking. Lancet. 2004;364(9445):1615.
Lavis JN, Oxman AD, Lewin S, Fretheim A. SUPPORT Tools for evidence-informed health Policymaking (STP). Health Res Policy Syst. 2009;7 Suppl 1:I1.
Cartwright N, Hardie J. Evidence-based policy: a practical guide to doing it better. Oxford: Oxford University Press; 2012.
Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. Jama. 2003;290(12):1624–32.
Hunink MGM, Weinstein MC, Wittenberg E, Drummond MF, Pliskin JS, Wong JB, Glasziou PP. Decision making in health and medicine: integrating evidence and values. Cambridge: Cambridge University Press; 2014.
Mays N, Pope C, Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy. 2005;10(Suppl 1):6–20.
Jacobs JA, Jones E, Gabella BA, Spring B, Brownson RC. Tools for implementing an evidence-based approach in public health practice. Prev Chronic Dis. 2012;9:E116.
Brownson RC, Fielding JE, Maylahn CM. Evidence-based decision making to improve public health practice. Front Public Health Serv Syst Res. 2013;2(2):2.
Lavis JN, Ross SE, Hurley JE. Examining the role of health services research in public policymaking. Milbank Q. 2002;80(1):125.
Lavis JN, Robertson D, Woodside JM, McLeod CB, Abelson J. How can research organizations more effectively transfer research knowledge to decision makers? Milbank Q. 2003;81(2):221.
Elliott H, Popay J. How are policy makers using evidence? Models of research utilisation and local NHS policy making. J Epidemiol Community Health. 2000;54(6):461–8.
Innvær S, Vist G, Trommald M, Oxman A. Health policy-makers' perceptions of their use of evidence: a systematic review. J Health Serv Res Policy. 2002;7(4):239.
Lima JC, Siegel M. The tobacco settlement: an analysis of newspaper coverage of a national policy debate, 1997-98. Tob Control. 1999;8(3):247–53.
Happer C, Philo G. The role of the media in the construction of public belief and social change. J Soc Pol Psychol. 2013;1(1):321.
Jamieson AM. The messenger as policy maker: thinking about the press and policy networks in the Washington community. Democratization. 1996;3(1):114.
Schwitzer G. How the media left the evidence out in the cold. Bmj. 2003;326(7403):1403.
Lee ST, Basnyat I. From press release to news: mapping the framing of the 2009 H1N1 a influenza pandemic. Health Commun. 2013;28(2):119–32.
Hooker C, King C, Leask J. Journalists' views about reporting avian influenza and a potential pandemic: a qualitative study. Influenza Other Respir Viruses. 2012;6(3):224–9.
Selvaraj S, Borkar DS, Prasad V. Media coverage of medical journals: do the best articles make the news? PLoS One. 2014;9(1):e85355.
Grilli R, Ramsay C, Minozzi S. Mass media interventions: effects on health services utilization. Cochrane Database Syst Rev. 2001;4:CD000389.
Winsten JA. Science and the media: the boundaries of truth. Health Aff (Project Hope). 1985;4(1):5.
Long M, Slater MD, Lysengen L. US news media coverage of tobacco control issues. Tob Control. 2006;15(5):367–72.
Nelson DE, Pederson LL, Mowery P, Bailey S, Sevilimedu V, London J, Babb S, Pechacek T. Trends in US newspaper and television coverage of tobacco. Tob Control. 2015;24(1):94–9.
Krauth D, Apollonio D. Accuracy of popular media reporting on tobacco cessation therapy in substance abuse and mental health populations. BMJ Open. 2015;5(3):e007169.
Eckler P, Rodgers S, Everett K. Characteristics of community newspaper coverage of tobacco control and its relationship to the passage of tobacco ordinances. J Community Health. 2016;41(5):953–61.
Oxman AD, Guyatt GH, Cook DJ, Jaeschke R, Heddle N, Keller J. An index of scientific quality for health reports in the lay press. J Clin Epidemiol. 1993;46(9):987.
Altman D. Practical statistics for medical research. New York: Chapman & Hall/CRC Press; 1990.
Apollonio DE, Bero LA. Challenges to generating evidence-informed policy and the role of systematic reviews and (perceived) conflicts of interest. J Commun Healthcare. 2016;9(2):135.
The authors acknowledge Evans Whitaker for assistance with the literature search, and Nancy Hessol for reading the manuscript and suggesting revisions.
This work was supported by NIH CA140236, California TRDRP 26IR-0014, and the UCSF Research Allocation Program. The funders played no role in the conduct of the research or preparation of the manuscript.
Availability of data and materials
The data supporting the conclusions of this article are publicly available and cited in the references section.
Ethics approval and consent to participate
Not applicable to this research design.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Huey, J., Apollonio, D.E. A content analysis of popular media reporting regarding increases in minimum ages of legal access for tobacco. BMC Public Health 18, 1129 (2018). https://doi.org/10.1186/s12889-018-6020-6
- Popular Media Reports
- Tobacco Issues
- Translation Research
- Tobacco Industry Marketing
- Adult Smoking Rates