Development and validation of a short and easy-to-use instrument for measuring health literacy: the Health Literacy Instrument for Adults (HELIA)
BMC Public Health volume 20, Article number: 656 (2020)
Health literacy is the ability to access to, understand, evaluate and use of essential health information to make basic health decisions. This study aimed to develop and psychometrically evaluate an instrument for measuring health literacy among adults (the Health Literacy Instrument for Adults - HELIA).
In addition to a literature review, a panel of specialists from different disciplines was formed to generate an item pool. Then, a framework was defined to develop the initial questionnaire based on a definition of health literacy and the most important global public health issues. The initial questionnaire contained 66 items. Next, 15 experts in public health were approached to assess content validity. Consequently, 19 items were removed and a provisional version of the questionnaire with 47 items was provided. Finally, a random sample of adults completed the questionnaire and psychometric properties of the instrument were assessed.
Overall, 323 adults aged 18 to 65 years old completed the questionnaire. When the exploratory factor analysis was performed, 33 items were loaded, which indicated a 5-factor solution for the questionnaire that jointly explained 52.9% of the variance observed. The factors were as follows: access to information (6 items), reading (4 items), understanding (7 items), appraisal (4 items), and decision making/behavioral intention (12 items). Confirmatory factory analysis also indicated a good fit to the data for the five-latent structure (χ2/df = 1.60, SRMR = 0.049; RMSEA = 0.043; CFI = 0.98; NFI = 0.95; NNFI = 0.98 and GFI = 0.87). Additional analysis for internal consistency showed satisfactory results with Cronbach’s alpha coefficients ranging from 0.72 to 0.89. Intraclass correlation coefficient (test-retest analysis) also showed acceptable stability for the questionnaire (ICC = 0.84). The mean score for health literacy as measured by the HELIA was 76.3 (SD = 15.1) out of 100 for the study sample.
The findings suggest that the Health Literacy Instrument for Adults (HELIA) is a valid and reliable instrument for measuring health literacy. It is a short and easy-to-use instrument that could be applied in different settings.
There is general agreement that health literacy goes beyond the ability to read, write and understand the meanings of words and numbers in health care settings . Health literacy comprises various competencies and depends on individual and community factors. These factors include different issues ranging from cultural issues to health care, public health and other relevant systems and settings in which people obtain and use health information .
Health literacy is a complicated concept  that is a global issue  and its existence is a way to achieve good health. One common definition of health literacy is offered by the World Health Organization (WHO) and reads as follows: ‘the cognitive and social skills which determine the motivation and ability of individuals to gain access to, understand and use information in ways which promote and maintain good health’ . It is believed that people with inadequate health literacy might suffer from poor health, have little information about disease prevention, participate less in clinical preventive services against chronic illnesses, and have trouble comprehending health instructions or interpreting them correctly [6, 7]. Furthermore, limited health literacy is associated with poor adherence to medical treatment and inappropriate communication with health professionals, more hospitalization, increased medical and health expenditure, higher mortality and morbidity and poorer self-care . Therefore, the measurement of health literacy is an essential component of any effort to prevent consequences of limited health literacy and health care discrepancies .
The number of instruments developed to address and evaluate health literacy is growing rapidly. The most widely and frequently used instruments are the Rapid Estimate of Adult Literacy in Medicine (REALM) , the Test of Functional Health Literacy in Adults (TOFHLA) , and the Newest Vital Sign (NVS) . These instruments have been criticized for several reasons, including for assessing only a few domains of health literacy, not being suitable for use in intervention studies or not having been developed with a health promotion perspective . Furthermore, most of these scales were developed and used in clinical settings . Haun et al. in a comprehensive review of the literature from 1999 through 2013 identified 51 instruments and reported that most instruments represent a narrow set of conceptual dimensions with limited modes of administration and missing information on key psychometric properties. They recommended that as researchers develop new measures, a full range of conceptual dimensions of health literacy and better validation studies should be included to establish sound evidence for measuring health literacy . As such, instruments that were developed recently are improved greatly [16, 17]. More recently The WHO Regional Office for Europe provided a comprehensive review of older and the recent instruments (31 instruments) and concluded that at the policy level, frameworks and indicators that cover various domains are needed to enable consistent and comparable population monitoring and evaluations .
However, it should be noted that there are three types of health literacy instruments: general health literacy instruments, condition (disease or content) specific measures, and instruments that are developed for specific populations . The WHO Regional Office for Europe also followed a very similar categorization for health literacy instruments . Of these, most investigators usually use the general measures that are applicable to different conditions and populations. Thus, the focus of this study is on developing a general instrument for measuring health literacy. In doing so we provided a short list of existing measures that includes the advantages and disadvantages of existing well-known general health literacy instruments (Table 1; for a more comprehensive list see [15, 18]).
The aim of this study was to develop a rigorous and valid instrument for measuring health literacy for adults, yet easy to use, and multidimensional. Although the current study is not unique, perhaps could contribute to the existing knowledge on the topic as the instrument reported here was developed in a non-western country.
The conceptual framework
The core conceptual model presented by Ratzan and Parker inspired the current study’s conceptual framework of health literacy. The concept comprised the ability to obtain health information (access); understand health information (understanding); ability to assess and evaluate the health information (appraisal); and use the information to make a decision (apply or use health-related information) .
We used a similar methodology that previously was described in details elsewhere . In brief, first a review of existing health literacy measures was conducted. The review was performed in 2012 and has since been updated. Consequently a panel of specialists in public health, health education/health promotion, health care management, mental health, oral health, maternal and child health, and community medicine was held. The selection of experts was based on their research interest, previous works on health literacy and managerial responsibility in the health care system. Panelists’ characteristics are presented in Table 2. Following ten sessions, of at least 3 h, an item pool of 400 statements was generated. Items were generated using brainstorming in each session and a member of the panel was responsible for listing the items. The panelists were limited to suggesting items related to the potential subscales (reading, access, understanding, appraisal, and behavioral intention). Then items were checked for duplicates and excluded when they were inconsistent with the intended subscales. Accordingly, the panel reduced the number of items to 66. Upon initial agreement on selected items a 5-point Likert scale (never =1, rarely =2, sometimes = 3, usually = 4, always = 5) was used to indicate the lowest to highest level of the respondents’ abilities. In the next step, content validity and face validity of the questionnaire was evaluated.
A group of 15 experts in public health was invited to assess the questionnaire. First, we asked the experts to check the items for relevance, clarity and simplicity on a 4-point Likert scale that comprised the options very relevant, relevant, relatively relevant, and not relevant. The experts were then asked to indicate necessity of each item by rating a 3-point Likert scale of essential, useful but not essential, and not essential. Next, experts were asked to comment on wording, and grammar. As a result, 19 items were removed and a provisional version of the questionnaire with 47 items was provided.
To determine the face validity of the questionnaire, 10 individuals aged 18–65 years were selected using a purposive sampling to verify the clarity, relevance and difficulty of each item. None of items were removed or changed at this stage and the Health Literacy Instrument for Adults (HELIA) was prepared for psychometric analysis.
Design and data collection: Psychometric properties of the HELIA were examined by conducting a cross sectional study. In this regard, a random sample of adults aged 18 to 65 and living in Tehran was recruited. The sample size was calculated as the number of items in the questionnaire multiplied by 5 . The participants completed the HELIA at their homes under trained interviewers’ supervision. Demographic data including the participants’ age, gender, education, job and source of health information, were also recorded.
Statistical analysis: The structural validity and reliability of the HELIA were examined using Exploratory Factor Analysis (EFA) with varimax rotation and internal consistency respectively. The Kaiser-Meyer-Olkin (KMO) and Bartlett’s Test of Sphericity were used to determine the appropriateness of the sample for factor analysis [30, 31]. Eigenvalues above 1 and factor loadings greater than or equal to 0.50 were considered appropriate to verify the number of possible underlying factors. Furthermore, confirmatory factor analysis was performed while a five-factor model (access, reading, understanding, appraisal and behavioral intention) was specified. Several goodness-of-fit indicators including: chi-square ratio (χ2/df), goodness of fit index (GFI), the root mean square error of approximation (RMSEA), standardized root mean square residual (SRMR), normed fit index (NFI) and comparative fit index (CFI) were selected for reporting the analysis outcomes. The following thresholds were considered to verify the model’s goodness of fit: χ2/df < 2.0, CFI, NFI, NNFI, and GFI ≥ 0.90–0.95, SRMR ≤0.05–0.08, and RMSEA ≤0.05–0.06 [32,33,34,35]. Finally, the Cronbach’s alpha coefficient (acceptable level of 0.7) for each dimension and the whole scale was calculated to examine internal consistency . Additionally the Intraclass Correlation Coefficient (ICC) was calculated by performing test-retest analysis to establish instrument stability. For this purpose, a convenience sample of 30 individuals aged 18 to 65 (mean age 34.9 ± 10.1, 18 female and 12 male) attending to a health center in Tehran was recruited. They have completed the questionnaire twice with a 1-week interval. The correlations of 0.75 or higher were considered satisfactory . All statistical analyses were performed using SPSS version 17.0. The confirmatory factor analysis was performed using LISREL 8.8 software.
In all 336 adults aged 18 to 65 took part in the study and completed the questionnaire. Of these, 13 individuals were excluded due to incomplete response to the questionnaire. Thus, the data obtained from 323 participants were analyzed. The mean age of participants was 37.89 ± 13.31 years, 54% were females, and 39% were housewives. The characteristics of participants are shown in Table 3.
1. Exploratory factor analysis: The adequacy of sample size was confirmed by KMO and Bartlett’s Test of Sphericity (KMO = 0.919 and χ2 = 4101.78, p < 0.0001). The initial analysis indicated a 9-factor solution with eigenvalues greater than 1 that jointly accounted for 58.9% of the variance observed. After careful assessment, four factors were excluded for the following reasons:
a. There was a factor with two items more relevant to behavioral intension and thus the factor was excluded and the items conjugated to factor 1 (behavioral intension subscale).
b. Item loading on three factors did not satisfy the expected threshold. Examples of some low loading items read as follows: I can find health information about physical activity such as walking and exercise; I can understand health information on diet and obesity; I can fill-in medical forms when needed.
Thus after deletion of the low loading items (with one exception), eventually 33 items were loaded on 5 factors: access to information (4 items), reading (6 items), understanding (7 items), appraisal (4 items) and behavioral intention (12 items), that jointly accounted for 52.9% of the variance observed. The detailed results are shown in Table 4 [Additional file 1].
2. Confirmatory factor analysis: The result obtained from the confirmatory factor analysis is depicted in Fig. 1. The results provided a good fit to the data. The fit indexes were as follows: χ2 = 778.33; χ2/df = 1.60; SRMR = 0.049; RMSEA (90% CI) = 0.043 (0.038–0.049); CFI = 0.98; NFI = 0.95; and NNFI = 0.98; GFI = 0.87 (Table 5). The correlations between latent factors are also presented in Table 6.
Reliability was assessed by estimating the Cronbach’s alpha coefficient. The results showed that all factors had acceptable internal consistency. The Cronbach’s alpha coefficient for each subscale and the questionnaire as a whole ranged from 0.72 to 0.89 (Table 7). Further analysis indicated that the alpha coefficient could not be improved if any further items deleted. The stability of the HELIA and its sub-scales as measured by the Intraclass Correlation Coefficient (ICC) was also found to be satisfactory. All ICCs were above acceptable threshold (Table 7).
The mean health literacy score for the study sample was 76.3 (SD = 15.1). Overall 78.6% of the respondents showed adequate health literacy while the remaing 21.4% had limited health literacy. There was no significant difference in health literacy among male and female respondents (P = 0.33), although women scored higher compared to male respondents (77.5 vs. 74.9 respectively). However, health literacy score was sginficatly different among people who differed in age, education, and employment status as expected. These are presented in Table 8. The scoring manual for the HELIA is supplemented (Additional file 2).
The findings showed that the HELIA is a valid instrument for measuring health literacy among adult populations and could be considered as a useful measure along with other recently developed instruments [16, 17, 38, 39]. However, it is important to note that although the methods we used were scientific, they were traditional and not as strong as the methods of two recently developed measures [16, 17]. For instance, for development of the Health Literacy Questionnaire (HLQ) Osborne et al. followed a validity-driven method that involved systematic grounded approaches in which existing theories were not considered until later in the development process of the questionnaire . In fact, they first focused on individuals’ and professionals’ lived experiences and then used the definition of health literacy proposed by the World Health Organization.
The HELIA has a multidimensional structure that can be easily used for public health purposes. Although not identical to the HELIA, the European Health Literacy Survey is also a relatively comprehensive instrument. It has two sections, a core health literacy section and a section on determinants and outcomes associated with health literacy. Indeed in the European Health Literacy Survey model ‘health literacy refers to an evolving set of competencies that do not remain static over time and can be regarded as a means to an end rather than a fixed state, to which a person should aspire’ . However, the very well known and popular instruments cover only a few dimensions of health literacy and none assess the broad range of abilities such as access to information, reading, understanding, appraisal, and decision making (behavioral intention). For example, the Test of Functional Health Literacy in Adults (TOFHLA) assesses only reading, comprehension and numeracy skills and it seems that completion of this test would be difficult for those who are not well educated to complete. The Rapid Estimate of Adult Literacy in Medicine (REALM), another well-known scale for measuring health literacy, is also examine only reading and recognition of medical words. Although it is a brief and easy to use instrument, it comprises just one dimension.
The HELIA contains five subscales (dimensions), which we believe is an important feature of this instrument covering the basic concept and constructs that build the meaning of health literacy. Additionally the items that included in the HELIA are relevant to public health in general and to healthy life styles in particular. In fact, underlying concepts included in the instrument cover the three most important global public health topics, which are issues related to cardiovascular diseases (nutrition items), cancers and accidents. These topics were arranged in a way that people with both limited literacy and a high level of education could easily respond to items. We did not want to test people’s knowledge but rather were interested in examining skills relevant to health literacy. Indeed, we believe a range of people with education ranging from primary to higher could relate the items to themselves and provide honest responses to the questionnaire. However, one should notice that the current version of the HELIA has some limitations. For instance, because numeracy skill is an important issue in a health care context, it is necessary to add a few more items to the questionnaire.
Nevertheless all these efforts should be greeted because relying on one measure might not properly help policy and practice. It is argued that since social, environmental and cultural factors influence health literacy in different populations, the need for integration of definitions and models of health literacy are essential . In this respect, it seems that items addressing numerical literacy and media literacy might also be necessary in new versions of existing health literacy instruments including the HELIA. Furthermore, to measure how valid the invented instrument is, it is necessary to compare the results of measurements with other recognized instruments and to show that at least some scales show comparable results (concurrent or criterion validity). The current study did not include such analysis and future studies should therefore employ a previously validated instrument and report on concurrent or criterion validity. One more limitation is the fact that the instrument was tested in one location using a cross-sectional approach and stability (test-retest analysis) was examined in a separate sample. Finally, it is important to remember that no external assessment was applied for the HELIA to objectively assess different skills. For instance when the respondent says that she or he is always understands the content, all the answers the respondents give are self-estimated skills and not objective ones. Thus, some measures should be integrated into the questionnaire to assess actual skills.
The results showed that the Health Literacy Instrument for Adults (HELIA) is a valid and reliable measure for assessing health literacy among adults.
Availability of data and materials
The data is available from the corresponding author upon reasonable request. The Persian version of the HELIA also will be available from the corresponding author.
Intraclass correlation coefficient
Health Literacy Instrument for Adults
Rapid Estimate of Adult Literacy in Medicine
Test of Functional Health Literacy in Adults
Newest Vital Sign
Health Literacy Questionnaire
47-item European Health Literacy Survey Questionnaire
16-item European Health Literacy Survey Questionnaire
All Aspects of Health Literacy Scale
Schnitzer AE, Rosenzweig M, Harris B. Health literacy: a survey of the issues and solutions. J Consum Health Internet. 2011;15:164–79.
Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med. 2011;155:97–107.
Tavousi M, Ebadi M, Shakerinejad G, Hashemi A, Fattahi E, Montazeri A. Definitions of health literacy: a systematic review of the literature. Payesh. 2013;13:119–24 [Persian].
Nutbeam D, Kickbusch I. Advancing health literacy: a global challenge for the 21st century. Health Promot Int. 2000;15:183–4.
WHO. Health Promotion Glossary. Geneva: World Health Organization; 1998.
Palumbo R. Discussing the effects of poor health literacy on patients facing HIV: a narrative literature review. Int J Health Policy Manag. 2015;4:417–30.
Robert J. Improving health literacy to reduce health inequalities. London: Public Health England, 2015. [Available at: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/460709/4a_Health_Literacy-Full.pdf 2019].
Parker RM, Williams MV, Weiss BD, Baker DW, Davis TC, Doak CC, Somers SA. Health literacy: report of the council on scientific affairs. Ad hoc committee on health literacy for the council on scientific affairs. JAMA. 1999;281:552–7.
Kickbusch I, Pelikan JM, Apfel F, Tsouros AD. Health literacy: the solid facts. Copenhagen: World Health Organization Regional Office for Europe, 2013. [Available at: http://www.euro.who.int/__data/assets/pdf_file/0008/190655/e96854.pdf 2018].
Davis TC, Crouch MA, Long SW, Jackson RH, Bates P, George RB, Bairnsfather LE. Rapid assessment of literacy levels of adult primary care patients. Fam Med. 1991;23:433–5.
Parker RM, Baker DW, Williams MV, Nurss JR. The test of functional health literacy in adults: a new instrument for measuring patients' literacy skills. J Gen Intern Med. 1995;10:537–41.
Weiss BD, Mays MZ, Martz W, Castro KM, DeWalt DA, Pignone MP, Mockbee J, Hale FA. Quick assessment of literacy in primary care: the newest vital sign. Ann Fam Med. 2005;3:514–22.
Kanj M, Mitic W. Health literacy and health promotion: definitions, concepts and examples in the East Mediterranean region. 7th global conference on health promotion working paper. Nairobi: World Health Organization; 2009. Available from: https://www.dors.it/documentazione/testo/201409/02_2009_OMS%20Nairobi_Health%20Literacy.pdf.
Nutbeam D. The evolving concept of health literacy. Soc Sci Med. 2008;67:2072–8.
Haun JN, Valerio MA, McCormack LA, Sørensen K, Paasche-Orlow MK. Health literacy measurement: an inventory and descriptive summary of 51 instruments. J Health Commun. 2014;19(Suppl. 2):302–33.
Osborne RH, Batterham RW, Elsworth GR, Hawkins M, Buchbinder R. The grounded psychometric development and initial validation of the health literacy questionnaire (HLQ). BMC Public Health. 2013;13:658.
Sørensen K, Van den Broucke S, Pelikan JM, Fullam J, Doyle G, Slonska Z, Kondilis B, Stoffels V, Osborne RH, Brand H, HLS-EU Consortium. Measuring health literacy in populations: illuminating the design and development process of the European health literacy survey questionnaire (HLS-EU-Q). BMC Public Health. 2013;13:948.
Rowlands G, Trezona A, Russell S, Lopatina M, Pelikan J, Paasche-Orlow M, Drapkina O, Kontsevaya A, Sørensen K. What is the evidence on the methods, frameworks and indicators used to evaluate health literacy policies, programmes and interventions at the regional, national and organizational levels? Copenhagen: WHO Regional Office for Europe; 2019. Available from http://www.ncbi.nlm.nih.gov/books/NBK549289/.
Morris N, MacLean CD, Chew LD, Littenberg B. The single item literacy screener: evaluation of a brief instrument to identify limited reading ability. BMC Fam Pract. 2006;7:21.
Chew LD, Griffin JM, Partin MR, Noorbaloochi S, Grill JP, Snyder A, Bradley KA, Nugent SM, Baines AD, Vanryn M. Validation of screening questions for limited health literacy in a large VA outpatient population. J Gen Intern Med. 2008;23:561–6.
Rawson KA, Gunstad J, Hughes J, Spitznage MB, Potter V, Waechter D, Rosneck J. The METER: a brief, self-administered measure of health literacy. J Gen Intern Med. 2010;25:67–71.
McCormack L, Bann C, Squiers L, Berkman ND, Squire C, Schillinger D, Ohene-Frempong J, Hibbard J. Measuring health literacy: a pilot study of a new skills-based instrument. J Health Commun. 2010;15:51–71.
Hahn EA, Choi SW, Griffith JW, Yost KJ, Baker DW. Health literacy assessment using talking touch screen technology (health LiTT): a new item response theory-based measure of health literacy. J Health Commun. 2011;16:150–62.
Begoray DL, Kwan B. A Canadian exploratory study to define a measure of health literacy. Health Promot Int. 2012;27:23–32.
Sorensen K, Pelikan JM, Rothlin F, Ganahl K, Slonska Z, Doyle G, et al. Health literacy in Europe: comparative results of the European health literacy survey (HLS-EU). Eur J Pub Health. 2015;25:1053–8.
Chinn D, McCarthy C. All aspects of health literacy scale (AAHLS): developing a tool to measure functional, communicative and critical health literacy in primary healthcare settings. Patient Educ Couns. 2013;90:247–53.
Ratzan SC, Parker RM. Introduction. In: Selden CR, Zorn M, Ratzan SC, Parker RM, editors. In national library of medicine current bibliographies in medicine: health literacy. Bethesda: National Institutes of Health; 2000.
Montazeri A, Tavousi M, Rakhshani F, Azin SA, Jahangiri K, Ebadi M, Naderimagham S, Solimanian A, Sarbandi F, Motamedi A, Mahdi Naghibi Sistani MM. Health Literacy for Iranian Adults (HELIA): development and psychometric properties. Payesh. 2014;13:589–99 [Persian].
Floyd FJ, Widaman KF. Factor analysis in the development and refinement of clinical assessment instruments. Psychol Assess. 1995;7:286–99.
Nunnally JC, Bernstein IH. Psychometric Theory. 4th ed. New York: McGraw-Hill; 1994.
Harrington D. Confirmatory factor analysis. New York: Oxford University Press; 2009.
Marsh HW, Hau K, Wen Z. In search of golden rules: comment on hypothesis testing approaches to setting cut-off values for fit indexes and dangers in over generalizing Hu and Bentler’s findings. Struct Equ Model. 2004;11:320–41.
Byrne BM. Structural equation Modelling. Mahwah: Lawrence Erlbaum Associates Publishers; 1998.
Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model. 1999;6:1–55.
Hatcher L. A step-by-step approach to using the SAS system for factor analysis and structural equation modeling. Cary: SAS Institute Inc; 1994.
Litwin MS. How to measure survey reliability and validity. London: Sage Publications; 1995.
Shoukri MM. Measures of Interobserver agreement and reliability. 2nd ed. Boca Raton: CRC Press, Taylor & Francis Group; 2011.
Ghanbari S, Ramezankhani A, Montazeri A. Mehrabi Y health literacy measure for adolescents (HELMA): development and psychometric properties. PLoS One. 2016;11:e0149202.
Naghibi Sistani MM, Montazeri A, Yazdani R, Murtomaa H. New oral health literacy instrument for public health: development and pilot testing. J Investig Clin Dent. 2014;5(4):313–21.
Sorensen K, Van den Broucke S, Fullam J, Doyle G, Pelikan J, Slonska Z, Brand H. Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health. 2012;12:80.
We are grateful to all participants who made this study possible.
The author(s) received no specific funding for this project.
Ethics approval and consent to participate
The ethics committee of Iranian Academic Center for Education, Culture and Research (ACECR) approved the study. All participants signed the written consent form.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Tavousi, M., Haeri-Mehrizi, A., Rakhshani, F. et al. Development and validation of a short and easy-to-use instrument for measuring health literacy: the Health Literacy Instrument for Adults (HELIA). BMC Public Health 20, 656 (2020). https://doi.org/10.1186/s12889-020-08787-2