Skip to main content
Download PDF

Validity and reliability of the swedish versions of the HLS-EU-Q16 and HLS-EU-Q6 questionnaires

BMC Public Health volume 23, Article number: 724 (2023) Cite this article

Abstract

Background

Health Literacy is a crucial factor for health. In Europe, many people have limited health literacy (i.e. difficulties with accessing, understanding, appraising and using health information). This study aimed to evaluate the psychometrics of the Swedish versions of the HLS-EU-Q16 and HLS-EU-Q6, instruments that aims to assess health literacy.

Methods

In this prospective psychometric study convenience sampling was used, which gave a study population of 347 Swedish-speaking adults. The psychometric evaluation included item distributional statistics, construct validity testing, and principal component analysis to assess structural validity. Internal consistency and test-retest reliability was also investigated.

Results

For the Swedish version of HLS-EU-Q16, no floor effects were detected but a ceiling effect was noted among 28% of the respondents. Construct validity was supported as four out of five expected correlations was confirmed (educational level, self-perceived health, electronic health literacy and HLS-EU-Q6). In terms of structural validity, the principal component analysis yielded a four-factor structure with most items loading significantly only to one factor. The Swedish version of HLS-EU-Q16 had acceptable internal consistency (Cronbach’s α = 0.89, split-half reliability = 0.93) and test-retest reliability showed stability over time (Cohen’s κ = 0.822). For the Swedish version of HLS-EU-Q6, neither floor nor ceiling effects were observed. Construct validity was supported as HLS-EU-Q6 correlated as our a priori stated hypothesis. The principal component analysis did not support the unidimensionality of the scale as a two-factor structure was identified. The Swedish version of HLS-EU-Q6 had acceptable internal consistency (Cronbach’s α = 0.77, split-half reliability = 0.80) and test-retest reliability showed stability over time (Cohen’s κ = 0.812). According to the Swedish version of the HLS-EU-Q16, 71% of the participants were classified as having sufficient comprehensive health knowledge (CHL), while only 33% were classified as having this when the HLS-EU-Q6 was used.

Conclusions

The Swedish versions of the HLS-EU-Q16 and HLS-EU-Q6 have acceptable psychometric properties, and based on the results we recommend its use to measure CHL. However, we are hesitant to use Sw-HLS-EU-Q6 in estimating different CHL levels and further studies need to be conducted to establish validity and accuracy of the thresholds of HLS-EU-Q6.

Peer Review reports

Background

Health literacy (HL) is a social determinant of health [1] and one of the three cornerstones of health promotion [2]. In this study we refer to HL as something that “is linked to literacy and entails people’s knowledge, motivation and competences to access, understand, appraise and apply health information in order to make judgements and take decisions in everyday life concerning health care, disease prevention and health promotion to maintain or improve quality of life during the life course”(page 3) according to Sorensen et al.’s definition of comprehensive HL (CHL) [3].

In Europe a third to nearly half of the population has low CHL, meaning that at least one in every three people has difficulties with accessing, understanding, appraising, and using health information [4]. The main socio-demographic and socioeconomic factors associated with CHL are age [5], level of education, income, perceived social status [5, 6], and ethnicity [6]. Furthermore, it is evident that there is a social gradient in CHL [1] and that limited CHL can contribute to unequal health [6]. Examples of negative health outcomes that have been associated with limited CHL are difficulties in understanding health information [7], worse preventive behaviours and higher vaccine hesitancy regarding Covid-19 [8], poorer self-perceived general health [5], more visits to general practitioners [5, 9], and longer stays in hospitals [9]. Other examples of associations are less disease knowledge [7, 10], poor self-management behaviours [10], and poor medical adherence [7].

Tailored interventions to improve HL can result in increased empowerment and improved decision-making skills, thus helping people take a more active part in their treatment and care [11, 12]. However, in order to measure if the intervention has a positive effect on HL, reliable and valid HL questionnaires are needed [13]. The Health Literacy Survey European Questionnaire, HLS-EU-Q, is a self-reporting instrument consisting of 47 items that was developed by Sørensen et al. in 2012. It is based on a systematic literature review that obtained an all-inclusive conceptual model along with Sorensen’s definition of CHL [3] as used in this study. The original English version of the HLS-EU-Q47 has been translated into more than 20 languages [14]. Two shorter and less time-consuming versions, HLS-EU-Q16 (3 min) and HLS-EU-Q6 (1 min), have been constructed from the HLS-EU-Q47. The HLS-EU-Q16 consists of 16 of the 47 items in the HLS-EU-Q47 that were selected using a one-parametric dichotomous Rasch model. The HLS-EU-Q6 consists of 6 items from the HLS-EU-Q16 that were selected based on results from a confirmatory factor analysis modelling and on higher item difficulty [15].

Several language versions of the HLS-EU-Q16 have been evaluated for its psychometric properties, showing that it is psychometrically sound [16,17,18,19,20,21], although the Japanese version of HLS-EU-Q16 had weak validity [20]. Psychometric evaluations of the short-short form, the HLS-EU-Q6, are few and with diverse results [18, 21]. The Italian version has been found to be reliable and valid [18], but the validity of the French version could not be established [21]. The Swedish version of the HLS-EU-Q16 (Sw-HLS-EU-Q16) was adapted and translated in line with guidelines for the translation of questionnaires [22] and so far used in one study including a Swedish speaking population [23]. However, it has not yet been psychometrically evaluated. The Swedish HLS-EU-Q6 (Sw-HLS-EU-Q6) has not been used in any study nor has it been validated. It is therefore important to examine the validity and reliability of the of Sw-HLS-EU-Q16 and Sw-HLS-EU-Q6.

Methods

Aim

The aim of this study was to psychometrically evaluate the Swedish versions of the HLS-EU-Q16 and HLS-EU-Q6 as well as to investigate the response patterns.

Study design

This prospective psychometric evaluation study is a part of a research project aiming to measure electronic health literacy and CHL in a population of Swedish and Arabic-speaking persons in Sweden [24,25,26,27,28].

Setting, sample, and data collection

This study aimed for a target sample of 300 participants as informed by the recommended sample size for psychometric validation studies [29]. The inclusion criteria were ≥ 18 years of age, having Swedish as a native language, and being available on the day of the data collection. The participants were recruited by convenience sampling from university courses, craft training, larger workplaces with academic and non-academic staff, non-governmental organisations serving elderly people, athletic clubs, and two choirs. The arenas included a diversity of groups of different ages, sexes, and levels of education. The different arenas selected for recruitment were visited at one or more time points during the data collection period. Potential participants were informed verbally and in writing about the study design by either the last author (JW) or by key stakeholders (i.e., organisation managers or others) selected by the researchers.

In the test-retest analysis 35 participants were invited to answer the questionnaire twice within one week. A sample size of at least 25 participants in the retest was deemed applicable [30]. To compare answers from the test and retest on an individual level, the participants marked their questionnaires with a code comprising the first three letters of their mother’s name and the year she was born. The data collection was carried out from February to May 2019.

Questionnaires

The Sw-HLS-EU-Q16 is a self-reported tool with Likert-type responses (“very easy”, “easy”, “difficult”, “very difficult”) and an associated sum score that measures individuals’ ability to access, understand, appraise, and apply health information. An overall Sw-HLS-EU-Q16 index score was calculated in three steps according to the developer [15, 31]. First, the response categories for the 16 items were dichotomized into easy (very easy and easy) with a value of 1 and difficult (difficult and very difficult) with a value of 0. Second, an overall index score was calculated by adding all the values obtained. Third, the index score was divided into three categories: inadequate (0–8 score points), problematic (9–12 score points), and sufficient (13–16 score points) CHL. To calculate the HLS index score, the respondent needs to have answered at least 14 of the 16 items.

To produce the score of the Sw-HLS-EU-Q6, the response categories were not dichotomised as for the Sw-HLS-EU-Q16 according Pelikan et al. 2014 [15]. Instead, the value for the response category was used: “very easy” = 4; “easy” = 3; “difficult” = 2; “very difficult” = 1; “don’t know/refusal” = missing [15]. First, the index score was calculated by adding all the values obtained and dividing by the number of items (requires response on at least 5 of the 6 items). Second, the index score was divided into three categories: inadequate CHL (≤ 2 score points), problematic CHL (> 2 and ≤ 3 score points), and sufficient CHL (score > 3) CHL. The cut-off values used for determining the three CHL levels for the Sw-HLS-EU-Q16 and Sw-HLS-EU-Q6 were the same as those recommended by Pelikan et al. [15] and that were used in Rouquette et al.’s study [21]. In addition, socio-demographic data (age, biological sex, and education) were collected.

Psychometric testing and data analysis

The inclusion criterion was a valid index score for the HLS-EU-Q16. Data are presented as mean, SD, number, percentage (%), or range. All percentages are rounded up to the nearest integer. The psychometric testing was guided by the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) and its recommendations for analysis when using classical test theory [32].

Floor and ceiling effects were examined on item and CHL score level by calculating the percentage of lowest or highest possible score received by the respondents and were considered acceptable if < 15% scored at the floor or ceiling [33]. Frequency of missing data was evaluated toward the criteria of < 5% [34].

Construct validity considered how the instruments correlated with other instruments or variables [35].Spearman’s rank coefficient was used to analyse the correlation between Sw-HLS-EU-Q16 index score, Sw-HLS-EU-Q6 index score, age, level of education, self-perceived health, and Swedish Electronic Health Literacy Scale (Sw-eHEALS) [24] sum score. Sw-eHEALS measures electronic HL, i.e. HL skills in relation to online information and applications [36, 37]. General self-perceived health was assessed with the question How do you assess your overall health status, and the response categories were very poor, poor, fair, good, or very good [27]. Based on our previous research [24, 25], we hypothesised that the Sw-HLS-EU-Q16 and Sw-HLS-EU-Q6 should be negatively correlated with high age and positively correlated with high level of education, high self-perceived health, and high score on Sw-eHEALS as well as positively correlated with each other (i.e. higher HL score on the Sw-HLS-EU-Q16 would imply higher HL score on the Sw-HLS-EU-Q6). A correlation coefficient magnitude between 0 and 0.1 was negligible, between 0.1 and 0.39 was weak, between 0.4 and 0.69 was moderate, between 0.7 and 0.89 was strong, and between 0.9 and 1.0 was very strong [38]. Structural validity refers to the extent to which the structure of the instrument adequately reflects the hypothesised dimensionality of the construct being measured [34]. In line with previous research [16, 39] evaluating the validity of HLS-EU-Q16, principal component analysis (PCA) was performed. The Kaiser-Meyer-Olkin index (criteria > 0.8) and Bartlett’s test of sphericity (p < 0.05) was used to assess appropriateness of the data set. Factor extraction was based on eigenvalues > 1 and visual examination of the scree-plot. The factor solution was rotated using oblique (oblimin) rotation [40].

Test-retest reliability, analyses the instruments consistency over time [33] and was analysed with weighted quadratic Cohen’s κ coefficient in order to measure the agreement between the two time points. A value of ≥ 0.70 was considered acceptable [41].

Internal consistency reliability is “the interrelatedness among the items” (page 2. Mokkink et al., 2010) [35]. It was assessed using Cronbach’s α and split-half reliability was calculated using Spearman–Brown’s coefficient with a reliability coefficient of 0.70–0.95 considered acceptable. To investigate response patterns between participants with valid Sw-HLS-EU-Q16 HL scores and those without, differences in sex were analysed with Chi-square test and differences in age were analysed with Student’s t-test. Differences in age, educational levels, general self-perceived health, Sw-HLS-EU-Q16, and Sw-HLS-EU-Q6 levels between participants were analysed with the Wilcoxon signed-rank test. The Mann–Whitney U-test was used to analyse differences in educational level. Two-tailed P-values under 0.05 were considered to be statistically significant.

Results

Demographics of the sample

A total of 347 respondents participated. Of these, 12 (3.5%) were excluded due to no valid CHL score for the Sw-HLS-EU-Q16 (i.e. >2 missing items). There was a significant difference in age between included and excluded participants who did not have a valid Sw-HLS-EU-Q16 index score (mean age 48.7 vs. 69.4, p = 0.001), but there were no significant differences in regard to gender (p = 0.274) or educational level (p = 0.895). No structural patterns in terms of difficulty in responding to certain items were observed.

For the total included sample, the mean age was 48.7 years (range 19–98 years) and around half were females (n = 168; 51%). The majority had at least 10 years of education (n = 303; 91%) and perceived their own general health to be good or very good (n = 290; 87%). Accordingly, for the Sw-HLS-EU-Q16 a total of 239 (71%) participants had sufficient CHL levels, and for the Sw-HLS-EU-Q6 a total of 112 (33%) participants were classified as having sufficient CHL (Table 1).

Table 1 Demographics of the test group (n = 335) and of the test-retest group (n = 38)
Full size table

The retest group consisted of 38 respondents. The mean age was 46 years (range 26–89 years), 23 (60%) were females, the majority had graduated from university (n = 29; 76%), and 35 (92%) perceived their general health as good or very good. The respondents’ mean index score for Sw-HLS-EU-Q16 was 13.9 (SD 2.8), and 31 (82%) were classified as having sufficient CHL. For the Sw-HLS-EU-Q6, the respondents had a mean index score of 3 (SD 0.5), and 17 (45%) were classified as having sufficient CHL (Table 1).

Item distribution statistics and floor and ceiling effects

Table 2 shows the distribution statistics for each item on the Sw-HLS-EU-Q16 and Sw-HLS-EU-Q6. No floor effects were observed on the item level, and the proportion of respondents scoring the lowest possible score was < 15%. All items except “judge when you may need to get a second opinion from another doctor” and “decide how you can protect yourself from illness based on information in the media” showed ceiling effects, and the proportion of respondents scoring the highest possible score was > 15%. Most of the items (13/16) had full variance (i.e. at least one respondent for each scoring option), and missing values were < 1% (n = 3) (Table 2).

Table 2 Distributional statistics for individual Sw-HLS-EU-Q16/Q6 items
Full size table

Sw-HLS-EU-Q16: A ceiling effect was noted with a total of 94 (28%) respondents scoring the maximum CHL index score of 16. No floor effects were detected, and no respondents received the minimum CHL index score of 0. Sw-HLS-EU-Q6: No floor or ceiling effects were noted, and a total of 11 (3%) respondents scored the maximum CHL index score of 4 and no respondents received the minimum CHL index score of 1.

Construct and structural validity

Sw-HLS-EU-Q16 was statistically weakly positively correlated with higher education level and higher self-perceived health, moderately positively correlated with higher Sw-eHEALS sum score, and strongly positively correlated with higher Sw-HLS-EU-Q6 index score. The Sw-HLS-EU-Q6 was also weakly positively correlated with higher education level and higher self-perceived health, moderately positively correlated with higher Sw-eHEALS sum score, and strongly positively correlated with higher Sw-HLS-EU-Q16 index score. No correlations were found between age and the Sw-HLS-EU-Q16 or Sw-HLS-EU-Q6. The Spearman correlation between the Sw-HLS-EU-Q16 and Sw-HLS-EU-Q6 scores was 0.84 (Table 3).

Table 3 Spearman rho correlations between Sw-HLS-EU-Q16, Sw-HLS-EU-Q6, age, education, self-perceived health and Sw-eHEALS
Full size table

For Sw-HLS-EU-Q16, the PCA yielded a four-factor model based on eigenvalues > 1 (KMO index value 0.890; Barlett’s test of sphericity < 0.001; variance explained 60.6%). However, in interpreting the scree-plot, a one factor solution could also be considered as a final structure. For the four-factor model, the items that clustered together suggest that factor 1 (item o, l, n, k, m and p) represents “Access, understand and process information in relation to health”; factor 2 (item d, g, c and j) represents “Understand information and follow instructions from healthcare professionals”; factor 3 (item e and f) represents “Process and apply information from healthcare”; and factor 4 (item a, b, I and h) represents “Access and understand information in relation to illness). All items loaded significantly (> 0.4) to only one factor except item J (Understand why you need health screenings) that had no significant loading and item K (Judge if the information on health risks in the media is reliable) that cross loaded on factor 1 and 3 (Table 4).

Table 4 Factor loadings of Swe-HLS-EU-Q16 (PCA, Oblimin rotation, n = 335)
Full size table

For Sw-HLS-EU-Q6, the PCA yielded a two-factor model based on eigenvalues > 1 (KMO index value 0.781; Barlett’s test of sphericity < 0.001; variance explained 64.5%). The items that clustered together suggest that factor 1 (item m, o and h) represents “Access and understand information in relation to health and illness” and factor 2 (item e, f and k) represent “Process and apply information from healthcare professionals and media”. All items loaded significantly only to one factor (Table 5).

Table 5 Factor loadings of Swe-HLS-EU-Q6 (PCA, Oblimin rotation, n = 335)
Full size table

Internal consistency

Cronbach’s α for the Sw-HLS-EU-Q16 and Sw-HLS-EU-Q6 was 0.89 and 0.77, respectively, and the split-half reliability computed by Spearman–Brown’s coefficient for the Sw-HLS-EU-Q16 and Sw-HLS-EU-Q6 was 0.93 and 0.80, respectively (Table 6).

Table 6 Test-retest reliability statistics for the Sw-HLS-EU-Q16 and Sw-HLS-EU-Q6, (n = 38)
Full size table

Test-retest reliability

Test-retest reliability of the Sw-HLS-EU-Q16 index score, Sw-HLS-EU-Q6 index score, and Sw-HLS-EU-Q16 levels was acceptable with Cohens κ > 0.7. For Sw-HLS-EU-Q6 levels, Cohen’s κ was 0.63 (Table 6). On the item level, the test-retest reliability was acceptable for 2 of 16 items (Cohen’s κ = 0.21–0.74).

Discussion

Our results indicate that the Swedish version of the HLS-EU-Q16 is a valid instrument, but the validity of the Swedish version of HLS-EU-Q6 is questionable. A higher proportion of participants were classified as having sufficient CHL when the HLS-EU-Q16 was used than when the HLS-EU-Q6 was used, namely 71% vs. 33%, respectively, in our Swedish version. This has also been described in the French version where 58% (HLS-EU-Q16) vs. 26% (HLS-EU-Q6) had sufficient CHL [21]. The discrepancy between the two versions in an Italian validation study was smaller at 33.0% vs. 24.6% [18], respectively. However, the proportion of participants with sufficient CHL assessed with the HLS-EU-Q6 was still much smaller compared with both our Swedish study and the French validation study [21]. The low proportion of participants with sufficient CHL when using the HLS-EU-Q16 was noticed by the authors themselves who suggested that the differences could be due to aspects such as cultural norms, the role of the family, and the usability of media sources in the specific area in the country [18]. Furthermore, a low proportion of participants with sufficient CHL was identified in the validation of the Brazilian Portuguese version of HLS-EU-Q6, in which only 2% of the participants were classified as having sufficient levels of CHL [42]. By examining how the participants in our study responded at the item level on the Sw-HLS-EU-Q16, we found that five of the items (items E, F, H, K, O) that had been most frequently responded to as being difficult or very difficult are included in Sw-HLS-EU-Q6. This is probably the main reason for why a higher proportion of participants were classified as having lower CHL when using the Sw-HLS-EU-Q6 than when using the Sw-HLS-EU-Q16. However, there was a strong correlation between the Sw-HLS-EU-Q6 and Sw-HLS-EU-Q16 (r = 0.84) even though the majority of the six items included in the Sw-HLS-EU-Q6 were scored lower than other items in the Sw-HLS-EU-Q16. The reason for this strong correlation can be explained by the fact that a correlation analysis does not take into account the interval between the variables [33]. [33] The constructer of the HLS-EU-Q6 also found strong correlations with the HLS-EU-Q47 (r = 0.90) and HLS-EU-Q16 (r = 0.82) [43]. The constructors (Jurgen Pelikan, personal communication 22/11/2021) further stated that reduction from 16 to 6 items led to significant loss of information and loss of representativeness of the theoretical scope [15] and thus it was not recommended to construct levels for the HLS-EU-Q6. The authors of the French version came to the same conclusion [21].

The discrepancy in CHL levels between the two HLS-EU versions is problematic, especially when comparing and implementing results and when drawing conclusions from the specific population. Moreover, the results from the analyses regarding internal consistency, validity, and test-retest reliability of the Sw-HLS-EU-Q6 were not conclusive. On one hand, no ceiling effects were noted, and construct validity as well as internal consistency were acceptable. However, our results showed that the Sw-HLS-EU-Q6 was not unidimensional as the PCA yielded a two-factor model. Thus, in this stage we, in line with the authors of the French validation study [21], do not recommend using the HLS-EU-Q6. In order to be able get a short version of the HLS-EU-Q47 such as HLS-EU-Q6, further work is needed in selecting appropriate items and revision of the thresholds described by Pelikan et al. [15]. Furthermore, comparisons of the distribution of CHL levels between the Sw-HLS-EU-Q16 and Sw-HLS-EU-Q6 with a Swedish version of the HLS-EU-Q47 would be valuable. However, there is yet no Swedish version of the latter.

Our results indicate that the psychometric properties of the Sw-HLS-EU-Q16 are acceptable in terms of internal consistency, test-retest reliability and construct validity. Our results are in line with previous studies, for example, the Sw-HLS-EU-Q16 had a Cronbach’s α = 0.89, which is in line with results from the French [21, 44], Icelandic [16], and Indian [19] versions of the HLS-EU-Q16. On the other hand, a ceiling effect of 28% was observed. This ceiling effect was also observed in the two French validation studies of the HLS-EU-Q16, with values of 21% [43] and 25% [21]. In our study all items except “judge when you may need to get a second opinion from another doctor” and “decide how you can protect yourself from illness based on information in the media” showed ceiling effects. This skewed distribution may indicate a country-specific limitation, which may affect the accuracy of the questionnaire. In terms of structural validity, the factor structure identified in the present study is in line with the Islandic [16] and Romanian versions [39], but factor loading pattern of the Sw-HLS-EU-Q16 differed from the original model as well as with the two previous mentioned studies. The results from the present study and previous research indicate that neither the domains nor the competencies of CHL that underlie the questionnaire manifest the same way across cultures [5]. We therefore only recommend calculating a sum score when using the Sw-HLS-EU-Q16.

Interestingly, 90% of the participants in our study thought it was easy/very easy.

“to find information on treatments of illnesses”. It is known that patients or relatives perform searches on the internet for information before and after clinical consultation in order to obtain more information and to make their own decision on the suggested diagnosis and treatment [45]. However, whether the person can assess if the information obtained is correct and reliable is debatable. On the other hand, the findings from our study indicate that the participants seem to have trust in and understand the instructions from health care, and a high proportion of the participants thought it was easy/very easy to: “understand your doctor’s or pharmacist’s instruction on how to take a prescribed medicine” (98%), “follow instructions from your doctor or pharmacist” (98%), “understand why you need health screenings” (94%), and “understand what your doctor says to you” (93%). These results are in line with the results from the validation of the Italian version of the HLS-EU-Q16 [18].

This study has some limitations to note. We aimed for a study population that included groups of different ages, genders and educational levels. However, it ended up that most of the participants had at least 10 years of education (many of whom also had some form of university education). This makes it difficult to assess whether the results of the study also apply to low-educated people in general. However, the overall results from the cognitive interviews conducted in a previous study to test the content validity of the instrument indicate that the instrument could be understood by less educated people even if it was not completely easy (unpublished results). A study from the Netherlands that shows that the instrument can be valid even among low-educated people show similar results. This as it also highlights that it would be good if some wordings were simplified, and extra contextual information were added [46]. Furthermore, we used a sample of participants from the capital of Sweden and a nearby city, areas that may not be representative of the majority of Swedish speakers. Our results must therefore be replicated on samples of subjects from other parts of Sweden before anything can be said about their robustness, especially in rural Sweden. Moreover, the use of self-administered questionnaires is restricted to people who can read and understand Swedish, and the HLS-EU-Q16 measures self-reported CHL, which is not the same as measuring the person’s knowledge of health.

Furthermore, answering self-reported HL questionnaires has been criticized for the cognitive burden imposed on the respondents [16, 47]. It would also be of value to validate the instruments in a study where data were collected using face-to-face interviews because this is how the instrument originally was used. This would include people with limited functional HL who might have difficulties with filling in forms [48]. Still, there were very few missing answers, and only 3.5% of the questionnaires were excluded due to no valid CHL score for the Sw-HLS-EU-16. Based on these limitations, we suggest that other methods should be combined with these subjective methods, such as interviews and observations, in order to increase the authenticity and objectivity of the data.

Conclusion

The Swedish version of the HLS-EU-Q6 and HLS-EU-Q16 have acceptable psychometric properties, and based on these results they can be used to measure CHL. However, we are hesitant to use Sw-HLS-EU-Q6 in estimating different CHL levels, as the agreement with the Sw-HLS-EU-Q16 when dividing the CHL index into inadequate, problematic and sufficient was poor. Further studies need to be conducted to establish validity and accuracy of the thresholds of HLS-EU-Q6. In addition, we recommend further studies comparing the Swedish short versions with the HLS-EU-Q47, i.e., the original version of the instrument and other instruments measuring CHL.

Data availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

CHL:

Comprehensive health literacy

COSMIN:

The COnsensus-based Standards for the selection of health Measurement Instruments

HLS-EU-Q:

European Health Literacy Survey Questionnaire

KMO:

Kaiser-Meyer-Olkin

PCA:

Principal Component analysis

Sw-HLS-EU-Q:

Swedish European Health Literacy Survey Questionnaire

Sw-eHEALS:

Swedish Electronic Health Literacy Scale

References

  1. Nutbeam D, Lloyd JE. Understanding and responding to health literacy as a Social Determinant of Health. Annu Rev Public Health. 2021;42:159–73.

    Article  PubMed  Google Scholar 

  2. World Health Organization (WHO). Shanghai declaration on promoting health in the 2030 agenda for Sustainable Development. Health Promot Int. 2017;32(1):7–8.

    Article  Google Scholar 

  3. Sørensen K, Van den Broucke S, Fullam J, Doyle G, Pelikan J, Slonska Z, et al. Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health. 2012;12(1):80.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Baccolini V, Rosso A, Di Paolo C, Isonne C, Salerno C, Migliara G, et al. What is the prevalence of low health literacy in European Union Member States? A systematic review and Meta-analysis. J Gen Intern Med. 2021;36(3):753–61.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Sørensen K, Pelikan JM, Röthlin F, Ganahl K, Slonska Z, Doyle G, et al. Health literacy in Europe: comparative results of the european health literacy survey (HLS-EU). Eur J Public Health. 2015;25(6):1053–8.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Stormacq C, Van den Broucke S, Wosinski J. Does health literacy mediate the relationship between socioeconomic status and health disparities? Integrative review. Health Promot Int. 2019;34(5):e1–e17.

    Article  PubMed  Google Scholar 

  7. Peltzer S, Hellstern M, Genske A, Jünger S, Woopen C, Albus C. Health literacy in persons at risk of and patients with coronary heart disease: a systematic review. Soc Sci Med. 2020;245:112711.

    Article  PubMed  Google Scholar 

  8. Turhan Z, Dilcen HY, Dolu İ. The mediating role of health literacy on the relationship between health care system distrust and vaccine hesitancy during COVID-19 pandemic. Curr Psychol. 2022;41(11):8147–56.

    Article  PubMed  Google Scholar 

  9. Friis K, Pedersen MH, Aaby A, Lasgaard M, Maindal HT. Impact of low health literacy on healthcare utilization in individuals with cardiovascular disease, chronic obstructive pulmonary disease, diabetes and mental disorders. A danish population-based 4-year follow-up study. Eur J Public Health. 2020;30(5):866–72.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Shah JM, Ramsbotham J, Seib C, Muir R, Bonner A. A scoping review of the role of health literacy in chronic kidney disease self-management. J Ren Care. 2021;47(4):221–33.

    Article  PubMed  Google Scholar 

  11. Visscher BB, Steunenberg B, Heijmans M, Hofstede JM, Devillé W, van der Heide I, et al. Evidence on the effectiveness of health literacy interventions in the EU: a systematic review. BMC Public Health. 2018;18(1):1–12.

    Article  Google Scholar 

  12. Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Viera A, Crotty K et al. Health literacy interventions and outcomes: an updated systematic review.Evid Rep Technol Assess (Full Rep). 2011;(199):1–941.

  13. Liu H, Zeng H, Shen Y, Zhang F, Sharma M, Lai W, et al. Assessment tools for health literacy among the general population: a systematic review. Int J Environ Res Public Health. 2018;15(8):1711.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Pelikan JM, Ganahl K, Van den Broucke S, Sørensen K. Measuring health literacy in Europe: introducing the european health literacy Survey Questionnaire (HLS-EU-Q). Bristol: Policy Press; 2019.

    Google Scholar 

  15. Pelikan J, Röthlin F, Ganahl K. Introduction to HL measurement procedures of the HLS-EU study. In: 2nd European HL Conference. Aarhus; 2014.

  16. Gustafsdottir SS, Sigurdardottir AK, Arnadottir SA, Heimisson GT, Mårtensson L. Translation and cross-cultural adaptation of the European Health literacy Survey Questionnaire, HLS-EU-Q16: the icelandic version. BMC Public Health. 2020;20(1):1–11.

    Article  Google Scholar 

  17. Bas-Sarmiento P, Poza-Méndez M, Fernández-Gutiérrez M, González-Caballero JL, Falcón Romero M. Psychometric Assessment of the European Health literacy Survey Questionnaire (HLS-EU-Q16) for Arabic/French-Speaking Migrants in Southern Europe. Int J Environ Res Public Health. 2020;17(21):8181.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Lorini C, Lastrucci V, Mantwill S, Vettori V, Bonaccorsi G. Measuring health literacy in Italy: a validation study of the HLS-EU-Q16 and of the HLS-EU-Q6 in italian language, conducted in Florence and its surroundings. Ann Ist Super Sanita. 2019;55(1):10–8.

    PubMed  Google Scholar 

  19. Dsouza JP, Van den Broucke S, Pattanshetty S. Validity and reliability of the Indian Version of the HLS-EU-Q16 Questionnaire. Int J Environ Res Public Health. 2021;18(2):495.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Maie A, Kanekuni S, Yonekura Y, Nakayama K, Sakai R. Evaluating short versions of the European Health literacy Survey Questionnaire (HLS-EU-Q47) for health checkups. Health Evaluation and Promotion. 2021;48(4):351–8.

    Article  Google Scholar 

  21. Rouquette A, Nadot T, Labitrie P, Van den Broucke S, Mancini J, Rigal L, et al. Validity and measurement invariance across sex, age, and education level of the french short versions of the European Health literacy Survey Questionnaire. PLoS ONE. 2018;13(12):e0208091.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46(12):1417–32.

    Article  CAS  PubMed  Google Scholar 

  23. Dahlman D, Ekefäll M, Garpenhag L. Health literacy among swedish patients in opioid substitution treatment: a mixed-methods study. Drug Alcohol Depend. 2020;214:108186.

    Article  PubMed  Google Scholar 

  24. Wångdahl J, Jaensson M, Dahlberg K, Nilsson U. The Swedish Version of the Electronic Health literacy scale: prospective psychometric evaluation study including thresholds levels. JMIR mHealth and uHealth. 2020;8(2):e16316.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Wångdahl J, Dahlberg K, Jaensson M, Nilsson U. Arabic Version of the Electronic Health literacy scale in arabic-speaking individuals in Sweden: prospective psychometric evaluation study. J Med Internet Res. 2021;23(3):e24466.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Bergman L, Nilsson U, Dahlberg K, Jaensson M, Wångdahl J. Health literacy and e-health literacy among arabic-speaking migrants in Sweden: a cross-sectional study. BMC Public Health. 2021;21(1):2165.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Wangdahl JM, Dahlberg K, Jaensson M, Nilsson U. Psychometric validation of swedish and arabic versions of two health literacy questionnaires, eHEALS and HLS-EU-Q16, for use in a swedish context: a study protocol. BMJ Open. 2019;9(9):e029668.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Bergman L, Nilsson U, Dahlberg K, Jaensson M, Wångdahl J. Validity and reliability of the arabic version of the HLS-EU-Q16 and HLS-EU-Q6 questionnaires. BMC Public Health. 2023;23(1). https://doi.org/10.1186/s12889-023-15226-5.

  29. VanVoorhis CW, Morgan BL. Understanding power and rules of thumb for determining sample sizes. Tutorials in Quantitative Methods for Psychology. 2007;3(2):43–50.

    Article  Google Scholar 

  30. Bujang MA, Baharum N. A simplified guide to determination of sample size requirements for estimating the value of intraclass correlation coefficient: A review.Archives of Orofacial Science. 2017;12(1).

  31. Pelikan JM, Ganahl K. Die europäische Gesundheitskompetenz-Studie: Konzept, Instrument und ausgewählte Ergebnisse. Health Literacy Forschungsstand und Perspektiven Bern: Hogrefe. 2017:93–126.

  32. Mokkink LB, Prinsen CA, Patrick DL, Alonso J, Bouter LM, de Vet HC et al. COSMIN Study Design checklist for Patient-reported outcome measurement instruments. Amsterdam, The Netherlands; 2019.

  33. De Vet HC, Terwee CB, Mokkink LB, Knol DL. Measurement in medicine: a practical guide. Cambridge university press; 2014.

  34. Polit DF, Yang F. Measurement and the measurement of change: a primer for the health professions. Philadelphia: Wolters Kluwer Health; 2016.

    Google Scholar 

  35. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010;63(7):737–45.

    Article  PubMed  Google Scholar 

  36. Norman CD. Skinner. eHEALS: the eHealth literacy scale. J Med Internet Res. 2006;8(4):e27.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Karnoe A, Kayser L. How is eHealth literacy measured and what do the measurements tell us? A systematic review. Knowl Manage E-Learning: Int J. 2015;7(4):576–600.

    Google Scholar 

  38. Schober P, Boer C, Schwarte LA. Correlation coefficients: appropriate use and interpretation. Anesth Analg. 2018;126(5):1763–68.

    Article  PubMed  Google Scholar 

  39. Coman MA, Forray AI, Van den Broucke S, Chereches RM. Measuring health literacy in Romania: validation of the HLS-EU-Q16 Survey Questionnaire. Int J Public Health. 2022;67:1604272.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Hair JF. Multivariate data analysis: a global perspective. 7th ed. Upper Saddle River, N.J.: Pearson Education; 2010.

    Google Scholar 

  41. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34–42.

    Article  PubMed  Google Scholar 

  42. Mialhe FL, Moraes KL, Bado FMR, Brasil VV, Sampaio HADC, Rebustini F. Psychometric properties of the adapted instrument european health literacy Survey Questionnaire short-short form. Rev Lat Am Enfermagem. 2021;29:e3436.

    Article  PubMed  PubMed Central  Google Scholar 

  43. The HLS19 Consortium of the WHO Action Network M-POHL. International Report on the methodology, results, and recommendations of the European Health literacy Population Survey 2019–2021 (HLS19) of M-POHL. Austrian National Public Health Institute, Vienna; 2021.

  44. Rouquette A, Rigal L, Mancini J, Guillemin F, Van den Broucke S, Allaire C, Azogui-Levy S, Ringa V, Hassler C. Health literacy throughout adolescence: invariance and validity study of three measurement scales in the general population. Patient Educ Couns. 2021;105(4):996–1003.

    Article  PubMed  Google Scholar 

  45. Cacciamani GE, Dell’Oglio P, Cocci A, Russo GI, Abreu ADC, Gill IS, et al. Asking “Dr. Google” for a second opinion: the Devil is in the details. Eur Urol Focus. 2021;7(2):479–81.

    Article  PubMed  Google Scholar 

  46. Storms H, Claes N, Aertgeerts B, Van den Broucke S. Measuring health literacy among low literate people: an exploratory feasibility study with the HLS-EU questionnaire. BMC Public Health. 2017;17(1):475.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health (Oxf). 2005;27(3):281–91.

    Article  PubMed  Google Scholar 

  48. Cornett S. Assessing and addressing health literacy.Online J Issues Nurs2009;14(3).

Download references

Acknowledgements

We would like to thank our respondents for participating in the study.

Funding

Open access funding provided by Karolinska Institute. This study is supported by Vetenskapsrådet (the Swedish Research Council), grant number 2015–02273.

Author information

Authors and Affiliations

  1. Department of Neurobiology, Care Sciences and Society, Karolinska Institute, Solna, Sweden

    Lina Bergman, Ulrica Nilsson & Josefin Wångdahl

  2. Department of Perioperative Medicine and Intensive Care, Karolinska University Hospital, Stockholm, Sweden

    Ulrica Nilsson

  3. Faculty of Medicine and Health, School of Health Sciences, Örebro University, Örebro, Sweden

    Karuna Dahlberg & Maria Jaensson

  4. Department of Public Health and Caring Sciences, Uppsala University, Uppsala, Sweden

    Josefin Wångdahl

  5. Aging Research Center, Karolinska Institutet & Stockholm University, Tomtebodavägen 18a, Solna, 171 77, Sweden

    Josefin Wångdahl

Authors
  1. Lina Bergman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ulrica Nilsson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Karuna Dahlberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maria Jaensson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Josefin Wångdahl
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

J.W. conducted the data collection. L.B and J.W. performed the statistical analysis and interpretation of data, wrote and critically revised the draft. U.N., M.J., K.D. and J.W. contributed to design, interpretation of data and critically revised the draft. All authors approved the final manuscript.

Corresponding author

Correspondence to Josefin Wångdahl.

Ethics declarations

Ethics approval and consent to participate

The study has been approved by the Regional Ethical Review Board in Stockholm, Sweden no. 2019/5:1 (since January 2019 renamed to the Swedish Ethical Review Authority) and follows the principles outlined in the 1964 Helsinki Declaration and its subsequent amendments. Informed consent was obtained from all the study participants.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bergman, L., Nilsson, U., Dahlberg, K. et al. Validity and reliability of the swedish versions of the HLS-EU-Q16 and HLS-EU-Q6 questionnaires. BMC Public Health 23, 724 (2023). https://doi.org/10.1186/s12889-023-15519-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12889-023-15519-9

Keywords

BMC Public Health

ISSN: 1471-2458

Contact us