- Research article
- Open Access
Severity and susceptibility: measuring the perceived effectiveness and believability of tobacco health warnings
BMC Public Health volume 18, Article number: 468 (2018)
Pictorial tobacco health warning labels (HWLs) have been shown to be more effective than text-only HWLs in changing smoking attitudes and intentions. However, there is contradictory evidence regarding how the severity of the content of HWLs influences responses to them.
We examined the perceived believability and effectiveness of HWLs in an online study using a convenience sample of non-smokers (N = 437) and smokers (N = 436). HWLs were in one of three presentation formats: (text-only, a moderately severe image or highly severe image) and focussed on three disease outcomes (lung cancer, blindness or tooth and gum disease). Participants rated the effectiveness and believability of each HWL and also rated their perceived susceptibility to each disease.
A 2 (smoking status) × 3 (presentation format) × 3 (disease outcome) ANOVA was run for both believability and effectiveness ratings. The most severe pictorial HWLs received the highest believability and effectiveness ratings and as expected, the text-only HWLs received the lowest. Lung cancer HWLs were rated most believable and effective, with the blindness HWLs receiving the lowest scores. A 2 (smoking status) × 3 (disease outcome) ANOVA was conducted on the ratings of perceived susceptibility to the three diseases. Smokers considered themselves to be more susceptible to all three diseases, and among smokers, perceived susceptibility to the diseases was positively correlated with effectiveness and believability ratings of the HWLs.
Our findings support previous evidence that pictorial HWLs are rated as more effective and believable than text-only warnings, and provide some support for the use of severe or ‘grotesque’ HWLs on tobacco products. Our data also suggest that HWLs should aim to increase perceived susceptibility to disease, as this was positively related to perceived message effectiveness and believability.
Understanding how the features of tobacco health warning labels (HWLs) influence overall message believability and effectiveness is politically important. Indeed, the tobacco industry claimed that proposed pictorial HWLs in the USA were not simply providing ‘factual and uncontroversial information’ and instead were ‘shocking and repelling’, ultimately barring their introduction . Pictorial tobacco HWLs have consistently been shown to be more effective than text-only HWLs in changing smoking attitudes and intentions [2, 3]. While graphic or ‘gruesome’ pictorial HWLs are generally found to be effective , there is contradictory evidence regarding how the severity of the content of HWLs and the diseases presented influences their effectiveness and believability. Previous research suggests that highly severe HWLs increase smokers’ intention to quit smoking as compared with less severe HWLs  and are more believable than symbolic warnings . Furthermore, high emotion HWLs have been shown to elicit greater emotional reaction than lower-emotion HWLs, with emotional reaction to HWLs related to higher risk perceptions of smoking . In contrast, other research on a range of different health outcomes indicates that highly severe warnings may result in defensive reactions [8,9,10]. This finding is supported by the Extended Parallel Process Model of fear-appeals  which suggests that fear appeals such as HWLs which only increase threat, without offering solutions for how to deal with this threat (i.e. efficacy messages) lead to maladaptive ‘fear control’ responses.
The degree to which a HWL is perceived as believable and effective is also likely to be influenced by the diseases presented [4, 12, 13] and one’s perceived susceptibility to developing these diseases [11, 14]. Fotuhi and colleagues found that compared to successful and failed quitters, continuing smokers had higher levels of risk-minimising beliefs , while Park and colleagues observed that smokers gave more uncertain answers regarding their risk of disease as compared with former smokers . HWLs which include messages regarding susceptibility to smoking-related disease have also been shown to increase message effectiveness .
Here we aimed to understand, using a series of unfamiliar tobacco HWLs, the influence of message severity, disease outcome and perceived susceptibility to disease, on the perceived believability and effectiveness of these HWLs among smokers and non-smokers.
Study design and participants
In this online survey, adult smokers and non-smokers were shown nine HWLs representing three disease outcomes in three different presentation formats. Participants rated HWL effectiveness and believability and their susceptibility to the three diseases. Given the large sample size required (see the Statistical Analysis section below), the survey platform Qualtrics was used to run the survey and we recruited participants opportunistically using the online crowdsourcing platform Prolific Academic. Participants took approximately 10 min to complete the survey and were reimbursed £1 for their participation (an amount commensurate with Prolific Academic reimbursement guidelines). We pre-registered the study protocol on the Open Science Framework (https://osf.io/wuz4a/).
We developed nine HWLs, using three disease outcomes, and three presentation formats (stimuli are available from the Bristol Research Data Repository (http://data.bris.ac.uk/data/; https://doi.org/10.5523/bris.381hbwlv26t9w2ipkng7f941e3). We conducted two pilot studies to determine which disease outcomes should be used (see the study protocol for more information: https://osf.io/wuz4a/). The three disease outcomes selected were: ‘Smoking causes 9 out of 10 lung cancers’, ‘Smoking increases the risk of blindness’ and ‘Smoking damages your teeth and gums’. The three presentation formats were: text-only, a ‘moderately severe’ image (i.e. an image showing moderate or average physical effects of smoking on either internal or external body parts) and a ‘highly severe’ image (i.e. an image showing a severe or worst case physical effect of smoking on either internal or external body parts). The two pilot studies were also used to determine that the images used were indeed ‘moderately’ or ‘highly’ severe, where participants were asked ‘how graphic (i.e. showing gruesome and vivid physical effects of the smoking related disease) is this picture’ and responded on a 1–10 Likert scale. Images with a mean score between 7 and 10 comprised the ‘highly severe’ images, and those with a score from 4 to below 7 comprised the moderately severe category.
Believability and effectiveness ratings
Perceived HWL believability was assessed using the single question ‘How believable is this health warning?’ Perceived HWL effectiveness was assessed using the single question ‘How effective is this health warning?’ To assist participants in answering this question, the following text was also provided ‘e.g., in encouraging smokers to quit, increasing concerns about smoking, and discouraging youth from starting to smoke’. These three effectiveness examples have been found to be highly correlated with questions regarding ‘overall effectiveness’ (Cronbach’s α = 0.97) [4, 6]. Believability and effectiveness questions were answered on a visual 1–10 Likert scale, with 1 labelled ‘not at all’ and 10 labelled ‘extremely’ [4, 12, 18, 19].
After viewing the HWLs, participants rated their perceived susceptibility to develop each of the three disease outcomes on a one to seven scale by answering the question ‘How likely do you think you are to suffer from each of the following diseases or ailments in the future?’ . Participants reported whether they currently have this disease or have done in the past, with the following options: ‘I do not/ have not had this disease’, ‘I have had this disease in the past’ or ‘I currently have this disease’.
Participants provided informed consent, demographic information, smoking status and if a smoker, reported for how long they had been a smoker . The nine HWLs were presented randomly and individually on screen and participants were given as long as required to view the HWL and answer the believability and effectiveness questions (the order of which was randomised). Participants then completed the perceived susceptibility questions and were provided with debriefing information and reimbursement instructions.
A sample size calculation indicated that 872 participants would be required and details of this can be found in the study protocol. A 2 (smoking status) × 3 (presentation format) × 3 (disease outcome) ANOVA was run for both believability and effectiveness ratings. A 2 (smoking status) × 3 (disease outcome) ANOVA was conducted on the ratings of perceived susceptibility to the three diseases. For both analyses, ANOVA allowed us to examine both the main effects of our independent variables (i.e. the effects of our independent variables averaging across the levels of the other independent variables) and the interactions between them. This allows us to examine, for example, whether there are general differences in effectiveness scores between the presentation formats (i.e., the main effect of presentation format) and whether there are differences in effectiveness scores for the different presentation formats between smokers and non-smokers (i.e., the presentation format × smoking status interaction). We specified our planned use of ANOVA in our study protocol, which was published online prior to starting data collection. The data that form the basis of the results are available from the Bristol Research Data Repository, https://doi.org/10.5523/bris.381hbwlv26t9w2ipkng7f941e3.
Characteristics of participants
Participants were required to be aged 18 or older, live in the UK (to ensure similar exposure to UK tobacco HWLs) and be either a non-smoker (n = 437) or a smoker (n = 436). Detailed participant characteristics are shown in Additional file 1: Table S1. Smokers and non-smokers differed on education, income and prevalence of tooth and gum disease, but not lung cancer or blindness.
Believability and effectiveness ratings
Results are presented in Fig. 1. Mean believability and effectiveness scores were moderately highly correlated (r = 0.60, p < 0.001). Greenhouse-Geisser corrected statistics are reported for the main effects of outcome and presentation on our dependant variables effectiveness and believability.
Smokers gave lower believability (F(1,871) = 46.74, p < 0.001, ηp2 = 0.05) and effectiveness ratings (F(1,871) = 38.51, p < 0.001, ηp2 = 0.04) than non-smokers. A main effect of presentation format was observed for both believability (F(2,1014) = 263.12, p < 0.001, ηp2 = 2.32) and effectiveness ratings (F(2,1348) = 1017, p < 0.001, ηp2 = 0.54). Bonferroni corrected post-hoc t-tests indicated that highly severe pictorial HWLs were more believable (t (870) = 6.37, p < 0.001) and effective (t (870) = 18.98, p < 0.001) than moderately severe pictorial HWLs, which in turn were more believable (t (870) = 17.08, p < 0.001) and effective (t (870) = 30.02, p < 0.001) than text-only HWLs.
A main effect of disease outcome was observed for both believability (F(2,1607) = 3804.0, p < 0.001, ηp2 = 0.42) and effectiveness ratings (F(2,1684) = 229.26, p < 0.001, ηp2 = 0.21). Bonferroni corrected post-hoc t-tests indicated that lung cancer HWLs were more believable (t (870) = 4.10, p < 0.001) and effective (t (870) = 11.93, p < 0.001) than tooth and gum disease HWLs, which in turn were more believable (t (870) = 138.44, p < 0.001) and effective (t (870) = 10.30, p < 0.001) than blindness HWLs. Given the differences between smokers and non-smokers for income and education, we conducted an ANCOVA with income and education as covariates. This analysis allowed us to statistically control for any potential effects of these covariates on our dependent variables. This analysis did not meaningfully change the results described above. Data that comprise these analyses are available from the Bristol Research Data Repository.
Greenhouse-Geisser corrected statistics for the ANOVA are reported (see Fig. 2). Data on perceived susceptibility to disease were not available for one non-smoker and five smokers. A main effect of outcome (F(2,1709) = 390.70, p < 0.001, ηp2 = 0.31) was observed, with participants rating their susceptibility to tooth and gum disease to be higher than lung cancer (t (864) = 10.04, p < 0.001), which in turn was perceived to be higher than blindness (t (864) = 18.30, p < 0.001). A main effect of smoking status (F(1,865) = 284.09, p < 0.001, ηp2 = 0.25) was also observed, with smokers rating their susceptibility to the diseases as higher than non-smokers.
In a series of exploratory analyses using Pearson’s correlation, we examined the relationship between perceived susceptibility to a disease and the believability and effectiveness ratings of the HWLs presenting that disease. As shown in Additional file 2: Table S2, among smokers, perceived susceptibility to each disease was positively correlated with both the believability and effectiveness scores given for the HWLs presenting that disease. However, among non-smokers, there was only evidence of a weak positive relationship between perceived susceptibility to blindness and the believability of the blindness HWLs. There was little evidence for other correlations among non-smokers.
We find that highly severe pictorial HWLs are perceived as being both more believable and effective than moderately severe pictorial HWLs and text-only HWLs among both smokers and non-smokers. This provides some support for the use of highly severe or ‘grotesque’ HWLs on tobacco products, as proposed by the FDA.
We find that the lung cancer HWLs were perceived as most effective and believable, while the blindness HWLs were rated least effective and believable by smokers and non-smokers. Perceived susceptibility to these diseases is likely to have influenced these ratings. Indeed, both the Extended Parallel Process Model  and the Risk Perception Attitude (RPA) Framework  indicate that perceived susceptibility to a disease is important in evoking a fear-response to a HWL. Among smokers, but not non-smokers, Pearson’s correlations indicated that perceived susceptibility to all three diseases was positively correlated with effectiveness and believability scores for the related HWL, indicating that perceived susceptibility to a disease is important in determining believability. This suggests that HWLs should not only present the most debilitating disease outcomes, but also those which smokers are most susceptible to (or those they perceive themselves to be susceptible to). HWLs could also present messages which explicitly increase individual’s perceived susceptibility to disease. Interestingly, participants rated their susceptibility to blindness as lower than both lung cancer and tooth and gum disease, reflecting their actual susceptibility [22,23,24,25,26,27] and among non-smokers, while the lung cancer and teeth and gum disease HWLs were rated as relatively highly believable and effective, the blindness HWL was only rated as believable among those who felt more susceptible to blindness. Future research should investigate which diseases smokers feel they are most susceptible to and develop HWLs which target these diseases.
Our study has a number of important strengths, in particular the large sample size including non-smokers and smokers. However, collecting data from a convenience sample such as this also represents a problem, as there is evidence that our sample had a higher level of education and income than the general population . Although our ANCOVA analyses controlling for these variables did not meaningfully change our results, future research should specifically target individuals with lower income and levels of education. Our study is also limited by its reliance on self-report data and further experimental work should determine whether these self-report responses reflect participants’ actual behavioural responses to these HWLs. In addition, given that the believability and effectiveness ratings of health warnings were correlated, it is possible that our single item question for each was measuring a single underlying factor. Finally, the analysis of the relationship between susceptibility ratings and believability and effectiveness ratings was exploratory and this finding should be treated with caution until replicated, particularly as it is possible that viewing the HWLs prior to estimating perceived susceptibility to disease may have influenced these ratings. Future research should experimentally manipulate perceived susceptibility to disease to fully understand this relationship.
We find that pictorial HWLs are rated as more effective and believable than text-only warnings and that highly severe pictorial warnings are rated as more effective and believable than moderately severe pictorial warnings. This supports the use of severe pictorial warnings on tobacco products, although we also note that perceived susceptibility to smoking related diseases is an important determinant in message effectiveness and believability. HWLs should therefore focus on diseases for which smokers are highly susceptible, or include messages that explicitly increase perceived susceptibility for that disease.
Analysis of covariance
Analysis of variance
Food and drug administration
Health warning label
Probability value (p-value)
Pearson correlation coefficient (Pearson’s r)
- ηp 2 :
Partial eta squared
United States District Court for the District of Columbia, R.J.Reynolds Tobacco Co, Lorillard Tobacco Co, Commonweath Brands Inc, Liggett Group LLC, Santa Fe Natural Toabcco Co Inc vs US FDA, Margaret Hamburg, Kathleen Sebelius. 2011. https://www.tobaccocontrollaws.org/litigation/decisions/us-20111107-r.j.-reynolds-tobacco-co.-v.-u.
Hammond D. Health warning messages on tobacco products: a review. Tob Control. 2011;20:327–37.
Noar SM, et al. Pictorial cigarette pack warnings: a meta-analysis of experimental studies. Tobacco Control. 2015;25(4):341–54.
Hammond D, et al. Perceived effectiveness of pictorial health warnings among Mexican youth and adults: a population-level intervention with potential to reduce tobacco-related inequities. Cancer Causes Control. 2012;23:57–67.
Kees J, et al. Understanding how graphic pictorial warnings work on cigarette packaging. J Public Policy Mark. 2010;29(2):265–76.
Mutti S, et al. Perceived effectiveness of text and pictorial health warnings for smokeless tobacco packages in Navi Mumbai, India, and Dhaka, Bangladesh: findings from an experimental study. Tobacco Control. 2015;25(4):437–43.
Evans AT, et al. Cigarette graphic warning labels are not created equal: They can increase or decrease smokers’ quit intentions relative to text-only warnings. Nicotine Tob Res. 2016;19(10):1155–62.
Peters G-JY, Ruiter RAC, Kok G. Threatening communication: a critical re-analysis and a revised meta-analytic test of fear appeal theory. Health Psychol Rev. 2013;7(sup1):S8–S31.
Harris PR, et al. Self-affirmation reduces smokers' defensiveness to graphic on-pack cigarette warning labels. Health Psychol. 2007;26(4):437–46.
Brown SL, Smith EZ. The inhibitory effect of a distressing anti-smoking message on risk perceptions in smokers. Psychol Health. 2007;22(3):255–68.
Witte K. Putting the fear back into fear appeals: the extended parallel process model. Commun Monogr. 1992;59(4):329–49.
Thrasher JF, et al. Can pictorial warning labels on cigarette packages address smoking-related health disparities? Field experiments in Mexico to assess pictorial warning label content. Cancer Causes Control. 2012;23(1):69–80.
Cameron LD, Williams B. Which images and features in graphic cigarette warnings predict their perceived effectiveness? Findings from an online survey of residents in the UK. Ann Behav Med. 2015;49(5):639–49.
Rimal RN, Real K. Perceived risk and efficacy beliefs as motivators of change. Hum Commun Res. 2003;29(3):370–99.
Fotuhi O, et al. Patterns of cognitive dissonance-reducing beliefs among smokers: a longitudinal analysis from the international tobacco control (ITC) four country survey. Tob Control. 2013;22(1):52–8.
Park ER, et al. A qualitative study of lung cancer risk perceptions and smoking beliefs among national lung screening trial participants. Nicotine Tob Res. 2014;16(2):166–73.
Nagelhout GE, et al. Pictorial cigarette warning labels: effects of severity and likelihood of risk messages. Nicotine Tob Res. 2016;18(5):1315–23.
Huang L-L, et al. Predictive and external validity of a pre-market study to determine the most effective pictorial health warning label content for cigarette packages. Nicotine Tob Res. 2016;18(5):1376–81.
Thrasher JF, et al. Cigarette warning label policy alternatives and smoking-related health disparities. Am J Prev Med. 2012;43(6):590–600.
Schuldt JP, Pearson AR. Nutrient-centrism and perceived risk of chronic disease. J Health Psychol. 2015;20(6):899–906.
Slovic, P., Cigarette smokers: rational actors or rational fools? 2001.
Peto R, et al. Smoking, smoking cessation, and lung cancer in the UK since 1950: combination of national statistics with two case-control studies. BMJ. 2000;321(7257):323–9.
Bergström J. Tobacco smoking and chronic destructive periodontal disease. Odontology. 2004;92(1):1–8.
Haffajee AD, Socransky SS. Relationship of cigarette smoking to attachment level profiles. J Clin Periodontol. 2001;28(4):283–95.
Hyman JJ, Reid BC. Epidemiologic risk factors for periodontal attachment loss among adults in the United States. J Clin Periodontol. 2003;30(3):230–7.
Kelly SP, et al. Smoking and cataract: review of causal association. J Cataract Refract Surg. 2005;31(12):2395–404.
Thornton J, et al. Smoking and age-related macular degeneration: a review of association. Eye. 2005;19(9):935–44.
Office for National Statistics. Census Analysis, Local Area Analysis of Qualifications Across England and Wales Release; 2011. p. 2011.
This work was supported by the Medical Research Council Integrative Epidemiology Unit at the University of Bristol, which is supported by the Medical Research Council and the University of Bristol (MC_UU_12013/6 and MC_UU_12013/7) and by an ESRC New Investigator Grant awarded to OMM (ES/R003424/1). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. OMM, AS and MRM are members of the United Kingdom Centre for Tobacco and Alcohol Studies, a UKCRC Public Health Research: Centre of Excellence which receives funding from the British Heart Foundation, Cancer Research UK, Economic and Social Research Council, Medical Research Council, and the National Institute for Health Research, under the auspices of the UK Clinical Research Collaboration.
Availability of data and materials
Health warning stimuli and the data that form the basis of the results are available from the Bristol Research Data Repository (http://data.bris.ac.uk/data/; https://doi.org/10.5523/bris.381hbwlv26t9w2ipkng7f941e3).
Ethics approval and consent to participate
We obtained ethics approval from the University of Bath Department of Psychology Research Ethics Committee (ethics approval code:16–059). Participants provided consent online after reading the study information sheet online. The study was conducted according to the revised Declaration of Helsinki (2013) and the 1996 ICH Guidelines for Good Clinical Practice E6(R1).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Maynard, O.M., Gove, H., Skinner, A.L. et al. Severity and susceptibility: measuring the perceived effectiveness and believability of tobacco health warnings. BMC Public Health 18, 468 (2018). https://doi.org/10.1186/s12889-018-5385-x
- Health warnings
- Public policy
- Packaging and labelling