A national health literacy scale was developed in China in 2012, though no studies have validated it. In this investigation, we assessed the reliability, construct validity, and measurement invariance of that scale.
Methods
A population-based sample of 3731 participants in Hunan Province was used to validate the Chinese Resident Health Literacy Scale based on item response theory and classical test theory (including split-half coefficient, Cronbach’s alpha, and confirmatory factor analysis). Measurement invariance was examined by differential item functioning.
Results
The overall Cronbach’s alpha of the scale was 0.95 and Spearman-Brown coefficient 0.94. Confirmatory factor analysis showed that the test measured a unidimensional construct with three highly correlated factors. Highest discrimination was found among participants with limited to moderate health literacy. In all, 64 items were selected from the original scale based on factor loading, Pearson’s correlation coefficient, and discrimination and difficulty parameters in item response theory. Measurement invariance was significant but slight. According to the two-level linear model, health literacy was associated with education level, occupation, and income.
Conclusions
The 2012 national health literacy scale was validated, and 64 items were selected based on classical test theory and item response theory. The revised version of the scale has strong psychometric properties with minor measurement invariance.
Keywords
Health literacyItem response theoryConfirmatory factor analysisMeasurement invariance
Background
The concept of health literacy was introduced in China in 2005 by the Chinese government through a manual entitled “Basic Knowledge and Skills of People’s Health Literacy” [1, 2]. That manual used the definition of health literacy of the World Health Organization: the cognitive and social skills which determine the motivation and ability of individuals to gain access to, understand and use information in ways which promote and maintain good health [3]. Under that definition, health literacy goes beyond the narrow concept of health education and individual behavior-oriented communication: it addresses the environmental, political, and social factors that determine health. The US Institute of Medicine defines health literacy in a similar way: health literacy is a set of skills that enable people to participate more fully in society instead of simply functional capabilities [4]. The ability to read and write is the foundation for health literacy, upon which a range of complementary skills can be built [5].
Based on the existing situation in China, health literacy was defined by its government as a set of capabilities in three domains in 2008: conceptual knowledge and attitudes; behavior and lifestyle; and health-related skills. The first nationwide survey on health literacy in China was conducted in 2008, and it focused on health knowledge [6]. The second national survey was conducted in 2012, with an emphasis on basic reading ability, arithmetic, and understanding medical information [7].
Internationally, the most commonly used measure of health literacy is the Rapid Estimate of Adult Literacy in Medicine and its shortened version; these assess an adult patient’s ability to read common medical terms and lay expressions for body parts and illnesses [8–10]. The Test of Functional Health Literacy in Adults and its shortened version are timed tests of reading comprehension of medical information [11, 12]. Other measures of health literacy in clinical settings include the following: the Medical Achievement Reading Test; the Newest Vital Sign; the Set of Brief Screening Questions; Functional, Communicative and Critical Health Literacy; the eHealth Literacy Scale; the Cancer Health Literacy Test; and the Diabetes Numeracy Test [13–20]. These measures focus on a single dimension of health literacy, rather than identifying its multidimensional nature [21].
By contrast, some measures have expanded the scope of medical care-related literacy; they include the following: the Health Activity Literacy Scale; the Demographic Assessment of Health Literacy; the 2003 National Assessment of Adult Literacy; the Adult Literacy and Life Skills Survey; and the Health Literacy Assessment Using Talking Touchscreen Technology [22–27]. These scales and questionnaires are more comprehensive because they involve different health-related competencies. However, they are considered proxy measures owing to the lack of an explicit definition of the concept of health literacy [28].
In China, health literacy in clinical settings has been measured using translated versions of scales used overseas as well as original Chinese scales among certain populations [29–38]. For example, health literacy among older adults has been measured using the Chinese version of the Rapid Estimate of Adult Literacy in Medicine [33]; the translated version of the Test of Functional Health Literacy in Adults was employed to measure health literacy among adolescents aged 12–16 years in Nanning; the eHealth Literacy Scale was translated and used on a sample of senior high school students [34]; the Chinese version of the Diabetes Numeracy Test was used in a cluster-randomized trial in patients with diabetes [35]; and a three-question measure of health literacy derived from a systematic review was applied among cataract patients [36, 39]. However, all these studies investigated cross-sectional health literacy without evaluating the instruments employed [40–42]. In the present study, we assessed the reliability and construct validity of the Chinese Resident Health Literacy Scale based on item response theory (IRT) and classical test theory using a population-based sample from Hunan Province in 2012. We also examined the association between health literacy and sociodemographic factors.
Methods
Participants
The participants were residents aged 15 to 69 years who had lived in the sampled regions for more than 6 of the previous 12 months. Such individuals as patients, students, military personnel, and prisoners resident in hospitals, school dormitories, nursing homes, military bases, and prisons were excluded from the survey.
We used a population-based stratified sampling frame, as shown in Fig. 1. The sampling strata included 13 cities or counties in Hunan Province, three streets or towns in each city or county, and two communities or villages (where the number of households exceeded 750) in each street or town. If there were fewer than 750 households within a community or village, the neighboring units were combined until that total was met. In each household, information regarding all family and non-relative members (e.g., hired nannies) aged 15–69 years who had been living there for more than 6 of the previous 12 months was recorded including gender (male or female) and age (elder to younger). One member in each house was selected for the survey by means of a Kish grid [43]. Unselected members were not allowed to complete the survey as a substitution.
Fig. 1
Study sampling process
The research protocol was reviewed and approved by the Medical Ethic Committee of the National Health and Family Planning Commission of China. All participants who agreed to participate in the study signed an informed consent form at the beginning of the survey.
Study design
The Chinese Resident Health Literacy Scale was developed based on a manual published by the Chinese Ministry of Health in 2008—“Basic Knowledge and Skills of People’s Health Literacy” (trial edition) [1]. The scale was designed by experts in public health, health education and promotion, and clinical medicine using the Delphi method. Details of the development procedure have been described in a previous paper [44]. The scale contains 80 items and three dimensions: (1) knowledge and attitudes; (2) behavior and lifestyle; and (3) health-related skills. The questions cover six aspects: scientific views of health; infectious diseases; chronic diseases; safety and first aid; medical care; and health information. As indicated in Table 1, there are four types of questions in the scale: true-or-false; single-answer (only one correct answer in multiple-choice questions); multiple-answer (more than one correct answer in multiple-choice questions); and situation questions. With multiple-answer questions, a correct response had to contain all the correct answers and no wrong ones. Situation questions were given following a paragraph of instruction or medical information.
Table 1
Examples of items
Type of items
Examples
Dimension
Scope
True-or-false
• A01 - Antibiotic is effective in preventing influenza.
Knowledge and attitude
Infectious disease
• A07 - Nutrients in vegetables and fruits are similar; so vegetables can be replaced by fruits.
Behavior and lifestyle
Chronic disease
Single-answer
• B01 - The integrated conception of health is: (1) Complete physical well-being without disease. (2) Physical and mental well-being. (3) A state of complete physical, mental, and social well-being and not merely the absence of disease or infirmity. (4) I do not know.
Knowledge and attitude
Scientific views of health
• B36 - When the fire emergency occurs, the correct way to escape is: (1) Encase your head with your arms or cloths and rush out the fire. (2) Wet your cloths and head, or cover yourself with a wet towel and rush out the fire. (3) Flap the fire with you cloths and escape simultaneously. (4) I do not know.
Health-related skills
Safety and first aid
Multiple-answer
• C06 - Which of the following strategies can prevent chronic disease: (1) Quit smoking and limit the intake of alcohol. (2) Balance the nutrition. (3) Exercise moderately. (4) Be in good mood. (5) I do not know.
Knowledge and attitude
Chronic disease
• C15 - Which descriptions about health management service for patients with type 2 diabetes are correct: (1) Only patients above 60 years old can receive the service. (2) All diagnosed patients in a community can receive the service. (3) Patients can receive four times of FBG testing for free. (4) Free FBG testing are unlimited, depending on the severity of disease. (5) I do not know.
Behavior and lifestyle
Medical care
Situation questions
• D03 - (A paragraph of instruction book for amoxicillin is given before the question) The drug may cause which of the following adverse reactions: (1) Nausea. (2) Depression. (3) Insomnia. (4) I do not know.
Health-related skills
Health information
• D04 - (A paragraph of introduction to body mass index is given before the question) Mr. Li is 45 years old and 27.7 in BMI (kg/m2). Which of the following categories does he belong to, according to the Chinese adult BMI reference: (1) Obese. (2) Normal. (3) Overweight. (4) I do not know.
Health-related skills
Health information
Before the field study, a survey team was established in each of the 13 cities or counties; the team comprised a principal, a coordinator, four to six investigators, a quality controller, and a data manager. All these team members received training for the sampling method, research tools, and quality control. A simulated survey was conducted during the training, and the investigators’ eligibility was assessed before performing the field survey.
Written informed consent was obtained from all participants before the survey. The scale was self-administered. However, if a participant was unable to complete the scale owing to impaired vision or other such reasons, an interview was used as an alternative. In that situation, the investigators would complete the questions in a neutral fashion on behalf of the participants.
Statistical analyses
Because repeated measures were not used, test-retest reliability was not determined. The split-half coefficient and Cronbach’s alpha were estimated before and after the item-selection procedure.
IRT was used to evaluate the precision of the measurements. IRT is a family of associated mathematical models that relate latent traits (ability) to the probability of responses to items in an assessment, and it has been widely used in psychometrics and health assessment [45]. It specifies a nonlinear relationship between binary, ordinal, or categorical responses and the latent trait (health literacy in this case). Compared with classical test theory approaches, the advantages of IRT include the following: near-equal interval measurement; representation of respondents and items on the same scale; and independence of person estimates from the particular set of items used for estimation [46].
We applied a two-parameter logistic IRT model for dichotomous responses. The two-parameter logistic model includes a difficulty parameter and discrimination parameter for each item. The difficulty parameter is the point on the ability scale that corresponds to a probability of a correct response of 0.5; the discrimination parameter estimates how well an item can differentiate among respondents with different levels of ability. Because the “I don’t know” choice was included for all questions, guessing parameters were not considered. Items with a discrimination parameter of 0.5 to 2.0 and a difficulty parameter corresponding to a certain region of the ability scale (−3.0 to 3.0) provide the most information [45, 47]. Parameters were estimated using a marginal maximum-likelihood method. The IRT model was recalibrated after the item-selection procedure. Measurement invariance of the scale among the different subgroups (by gender and race) was estimated using differential item functioning in the IRT model.
Pearson’s correlation coefficient was determined. An eligible items had to be significantly and at least moderately (0.4 to 0.7) correlated to the total score of its domain; hence, the correlation coefficient between them had to be above 0.4 [48]. The construct validity was assessed by confirmatory factor analysis (CFA). An assumed structure of the scale (three dimensions) was tested using a structural equation model. Since the items were binary measures, the unweighted least-squares method was employed for parameter estimation in the structural equation model. The chi-square value, goodness-of-fit index, root of the mean square residual, and parsimony goodness-of-fit index were used to assess the model fit. Several studies have recommended that the factor loading should be above 0.4 [49–51].
Items that met two or more of the following criteria were removed: (1) discrimination parameter <0.5 or >2.0; (2) difficulty parameter < −3.0 or >3.0; (3) factor loading <0.4; and (4) Pearson’s correlation coefficient <0.4. In addition, items with strong discrimination (≥1.0) were selected to form a short version of the scale.
The demographic variables were described, and raw scores among the different subgroups were compared using analysis of variance. After item selection, the association between health literacy scores and demographic variables was tested by means of a multilevel linear model.
The IRT calibrations were conducted using PARSCALE 4.1 (Scientific Software International Inc., Lincolnwood, USA). CFA was performed in AMOS 17.0 (Arbuckle JL and SPSS Inc., Chicago, USA). Multilevel model estimation was carried out with MLwiN 2.1 (Rasbash J, Charlton C, Browne WJ, Healy M, and Cameron B, Centre for Multilevel Modelling, University of Bristol, UK). Other analyses were conducted using SAS 9.2 (SAS Institute Inc., Cary, USA). The significance level was 0.05 for all statistical tests.
Results
In all, 3900 participants were sampled, and 3731 (95.7 %) completed the survey without apparent logical errors or missing items. As indicated in Table 2, there were significant differences in the health literacy scores among the subgroups of age, education level, occupation, annual per capita income, and residence (P <0.05), but not among the subgroups of gender and race. The proportion of correct responses to the 80 items varied from 10.8 to 96.7 % (Table 2).
Table 2
Sociodemographic characteristics of the participants and association with health literacy scores
N (%)
Health literacy scores
P-value
Gender
Male
1890 (50.7)
50.9 ± 15.7
0.896
Female
1841 (49.3)
51.0 ± 16.0
Age (year)
15–24
345 (9.3)
54.7 ± 15.8
<0.001
25–34
592 (15.9)
54.3 ± 13.9
35–44
901 (24.1)
52.0 ± 15.4
45–54
846 (22.7)
49.9 ± 15.6
55–64
781 (20.9)
47.5 ± 16.9
65–69
266 (7.1)
48.7 ± 16.2
Race
Han
3445 (92.3)
50.9 ± 15.7
0.898
Minorities
276 (7.4)
50.8 ± 17.7
Missing
10 (0.3)
Education
No formal education
225 (6.0)
40.5 ± 18.8
< 0.001
Primary school
1031 (27.6)
47.6 ± 16.8
Junior school
1441 (38.6)
50.5 ± 15.1
High school
691 (18.5)
55.4 ± 13.3
College
319 (8.5)
60.8 ± 9.3
Graduate school
11 (0.3)
61.0 ± 12.0
Missing
13 (0.3)
Occupation
Civil servant
34 (0.9)
59.0 ± 12.0
< 0.001
Teacher
53 (1.4)
57.4 ± 10.2
Medical staff
74 (2.0)
63.4 ± 10.8
Student
68 (1.8)
55.1 ± 16.2
Farmer
2610 (70.0)
48.7 ± 16.4
Worker
188 (5.0)
53.1 ± 12.1
Other
690 (18.5)
56.0 ± 12.9
Missing
14 (0.4)
Annual income per capita (CNY)
≤ 10,000
1498 (40.2)
47.5 ± 15.7
< 0.001
10,001–20,000
883 (23.7)
50.7 ± 15.0
20,001–30,000
73 (1.9)
57.9 ± 14.5
> 30,000
857 (23.0)
55.5 ± 15.9
Reject to answer
420 (11.2)
Type of residence
Urban
1458 (39.1)
52.0 ± 14.2
0.001
Rural
2273 (60.9)
50.3 ± 16.8
Survey method
Self-report
2870 (76.9)
51.1 ± 14.2
0.223
Interview
861 (23.1)
50.4 ± 20.4
The Spearman–Brown split-half coefficient was 0.94. The overall Cronbach’s alpha was 0.95; Cronbach’s alpha of the three dimensions was as follows: 0.90 (knowledge and attitude, 38 items); 0.83 (behavior and lifestyle, 22 items); and 0.85 (skills, 20 items). The two-parameter logistic model fitted the data well (P >0.05). The difficulty and discrimination parameters from the IRT model appear in Table 3. Most items exhibited good discriminative power and moderate difficulty. As shown in Fig. 2, the test information reached a peak when the participants’ ability was between −1 and 0, which indicates that the measurement was most discriminative among participants with limited to medium-level abilities in health literacy.
Table 3
Evaluation of items based on item response theory and confirmatory factor analysis
Correct (%)
Item correlation to dimension score
Factor loading
IRT parameters
Discrimination
Difficulty
Dimension 1: Knowledge and attitudes
A01
Prevention of influenza
66.2
0.44
0.37
0.60
−0.82
A02
Hypertension
85.3
0.40
0.35
0.75
−1.78
A03
Health food
88.3
0.28
0.24
0.51
−2.67a
A04
Transfusion
64.4
0.51
0.47
0.80
−0.63
A05
AIDS detection and consultation
92.1
0.32
0.28
0.74
−2.43a
A06
Pregnancy and hazardous job
91.2
0.31
0.25
0.65
−2.54a
A08
Range of body temperature
76.5
0.27
0.20
0.33
−2.25a
A11
Food label
96.7
0.23
0.19
0.74
−3.21a
A15
Health examination
72.8
0.44
0.39
0.68
−1.11
B01
Definition of health
71.5
0.59
0.55
1.18
−0.79
B02
Improve resident’s health
78.4
0.56
0.50
1.19
−1.06
B03
Blood donation
80.4
0.50
0.46
0.99
−1.24
B04
Transmission of hepatitis B
70.7
0.51
0.48
0.83
−0.89
B05
Self-measured blood pressure
51.7
0.50
0.48
0.74
−0.11
B07
Early signals of cancer
60.4
0.59
0.57
1.06
−0.41
B08
Chronic diseases
56.7
0.50
0.49
0.76
−0.32
B09
Management of gas poisoning
80.4
0.49
0.43
0.93
−1.29
B10
Treatment of tuberculosis
80.9
0.40
0.36
0.66
−1.62
B11
Folic acid supplementation
56.5
0.53
0.53
0.85
−0.30
B12
Occupational protection
69.6
0.53
0.48
0.87
−0.81
B13
Iodine deficiency
71.9
0.53
0.51
0.95
−0.87
B14
Pesticide residues
41.0
0.31
0.31
0.40
0.56a
B18
Children immunization
85.4
0.47
0.42
0.99
−1.53
B23
Health knowledge
71.2
0.44
0.41
0.66
−1.04
B26
Warning icons
71.0
0.57
0.55
1.08
−0.80
B31
Adult pulse
72.7
0.49
0.46
0.81
−0.99
B34
Dog bite
90.4
0.29
0.21
0.57
−2.72a
B38
Expired food
87.0
0.46
0.42
1.01
−1.62
C02
Liver function
27.9
0.48
0.52
0.88
0.85
C03
Transmission of tuberculosis
30.4
0.43
0.47
0.72
0.83
C04
Child fever and rash
51.9
0.58
0.59
1.01
−0.12
C05
Osteoporosis
10.8
0.42
0.48
1.41
1.53
C06
Prevention of chronic diseases
43.4
0.55
0.55
0.91
0.20
C07
Health food selection
54.5
0.60
0.62
1.09
−0.21
C08
Dead livestock
70.5
0.52
0.50
0.86
−0.87
C17
Pesticide custody
61.7
0.34
0.29
0.32
−0.90a
D06
Weight control
70.9
0.50
0.47
0.73
−0.38
D07
Obesity-related disease
75.2
0.53
0.53
0.90
−0.86
Dimension 2: Behavior and lifestyle
A07
Fruit and vegetable
75.2
0.40
0.36
0.53
−1.44
A09
Internet addiction
90.4
0.20
0.16
0.29
−4.74a
A10
Child and adolescent depression
87.6
0.37
0.32
0.68
−2.07a
A12
Lack of sleep
92.4
0.30
0.25
0.68
−2.60a
A13
Obtain health knowledge
93.3
0.30
0.26
0.72
−2.61a
A14
Attitude toward chronic diseases
57.0
0.38
0.32
0.41
−0.45a
B06
Smoking
54.9
0.55
0.63
0.83
−0.21
B15
Sport and electrolytes
70.0
0.48
0.44
0.71
−0.90
B16
Mental health
86.0
0.51
0.50
1.34
−1.34
B17
Public health service
46.3
0.56
0.52
0.88
0.12
B19
Fever symptom
79.3
0.42
0.33
0.64
−1.51
B20
Adverse drug reaction
79.0
0.39
0.31
0.57
−1.62a
B21
Health care card for pregnant women
52.6
0.45
0.40
0.57
−0.14
B27
Medical care procedure
71.8
0.57
0.54
1.14
−0.78
B29
House ventilation
65.7
0.50
0.45
0.79
−0.66
B39
Drug addiction
61.1
0.52
0.52
0.76
−0.48
C01
Mental health promotion
50.8
0.58
0.55
0.99
−0.05
C10
Bean products
23.4
0.52
0.49
1.03
1.00
C11
Sport and health
41.5
0.60
0.58
1.10
0.26
C13
Cough and sneeze
32.4
0.51
0.45
0.81
0.70
C14
Length of hospital stay
59.8
0.56
0.54
0.90
−0.39
C15
Management of type 2 diabetes
26.8
0.55
0.50
1.03
0.84
Dimension 3: Skills
B22
Medical institutions
77.4
0.55
0.52
1.12
−0.88
B24
Infectious diseases
78.2
0.58
0.55
1.35
−0.82
B25
Food poisoning
78.6
0.57
0.54
1.36
−0.85
B28
OTC drugs
37.3
0.54
0.51
0.59
1.05
B30
Reading a thermometer
51.2
0.59
0.56
0.80
0.18
B32
Prenatal examination
41.2
0.44
0.39
0.38
1.05
B33
Condoms and STDs
84.2
0.52
0.51
1.19
−1.13
B35
Slight burns and scalds
70.9
0.49
0.44
0.69
−0.80
B36
Fire escape
77.8
0.46
0.41
0.66
−1.24
B37
Bleeding wound management
22.7
−0.04
−0.16
0.03
0.00a
B40
Electrical injury
70.6
0.43
0.40
0.42
−1.04
C09
Cardiopulmonary resuscitation
53.7
0.65
0.65
1.06
0.09
C12
Diabetes
43.8
0.51
0.46
0.53
0.60
C16
Brest feeding
42.3
0.51
0.50
0.56
0.70
C18
Lightning storms
76.9
0.44
0.40
0.58
−1.23
D01
Indications of amoxicillin
25.0
0.55
0.54
0.86
1.42
D02
Use of amoxicillin
64.4
0.50
0.43
0.55
−0.38
D03
Adverse reaction of amoxicillin
54.2
0.65
0.62
1.07
0.05
D04
Calculation of BMI
41.8
0.62
0.58
0.94
0.60
D05
Classification of BMI
45.7
0.59
0.53
0.77
0.46
aItems that met two or more criteria for deletion were removed
Fig. 2
Test information and participant ability. Ability signifies health literacy estimated using the maximum-likelihood method. Ability in the item response theory (IRT) model practically (though not exclusively) ranged from −3 to +3. The test information reached a peak when the ability was between −1 and 0; this indicates that the measurement exhibited highest discriminative power among participants with limited and under-average ability with respect to health literacy
With the CFA results, the three-factor model showed slightly better fit than the one-factor model. Correlations among the three factors (knowledge and attitudes; behavior and lifestyle; skills) were 0.96–0.98, which indicates good evidence for unidimensionality, i.e., the dominant dimension of health literacy. Factor loading and the correlation coefficient between items and dimensional scores are presented in Table 3.
In all, 16 items were removed from the scale according to the criteria of item selection; 10 of them were true-or-false questions, which showed poor discriminative power and small factor loading. Sixty-four items were selected according to classical and modern test theory standards. The Spearman-Brown coefficient was 0.94. The overall Cronbach’s alpha was 0.95; Cronbach’s alpha of the three dimensions was as follows: 0.90 (knowledge and attitude, 30 items); 0.83 (behavior and lifestyle, 16 items); and 0.86 (skills, 18 items). Goodness-of-fit of the CFA and the IRT models improved slightly compared with the original scale. Factor loading, difficulty parameters, and discrimination parameters of all the items met the criteria.
A shorter version of the scale, comprising 19 items with discrimination parameters ≥1.0, was also created. The shorter version consisted of eight items in the knowledge and attitude dimension, five items in the behavior and lifestyle dimension, and six items in the health-related skill dimension. The overall Cronbach’s alpha was 0.88; Cronbach’s alpha of the three dimensions was 0.76, 0.64, and 0.77 respectively. The split-half coefficient was 0.87. The correlation coefficients and factor loadings of all the items were above 0.4 (mostly >0.5), and the discrimination parameters of all the items were 0.5–2.0 (mostly 1.0–2.0).
Differential item functioning in the IRT model was used to examine measurement invariance. The chi-square tests showed significant measurement invariance in both gender and race (P <0.05); however, the slope and threshold parameters were very close between male and female as well as between urban and rural groups.
The association between health literacy (revised scale) and demographic variables was explored using a two-level model because intracluster correlation was identified at the level of cities. As indicated in Table 4, education level, occupation, and income were associated with health literacy. Participants with higher socioeconomic status (higher education level and greater income) were more likely to have adequate health literacy. The intracluster correlation coefficient at the city level was 34.5 %.
Table 4
Association between health literacy and sociodemographic variables based on a two-level linear model
Coefficient
95 % CI
Education
No formal education
Reference
Primary school
6.46
(4.74, 8.17)
Junior school
9.78
(8.06, 11.48)
High school
13.15
(11.22, 15.07)
College
18.34
(15.89, 20.78)
Graduate school
18.87
(11.50, 26.24)
Occupation
Farmer
Reference
Civil servant
4.41
(0.20, 8.61)
Teacher
2.02
(−1.50, 5.53)
Medical staff
7.76
(4.56, 10.95)
Student
−0.48
(−3.57, 2.61)
Worker
3.50
(1.64, 5.36)
Other
2.94
(1.59, 4.28)
Annual income per capita (CNY)
< 10,000
Reference
10,001–20,000
0.50
(−0.49, 1.48)
20,001–30,000
5.99
(3.28, 8.70)
> 30,000
0.70
(−0.41, 1.80)
Discussion
To validate the scale used in the 2012 National Health Literacy Survey, we performed this study using a population-based sample in Hunan Province. Classical test theory (Cronbach’s alpha, split-half coefficient, and factor analysis) and modern test theory (IRT) were used in validating the scale. We found that the 2012 scale of health literacy meets psychometric standards. The overall Cronbach’s alpha was 0.95. The assumption that the scale measures a unidimensional construct was supported by the three-factor model fit being approximately that of the one-factor model fit and the three factors (knowledge and attitudes; behavior and lifestyle; skills) being highly correlated. Among the 80 items tested, 16 performed poorly and were removed. The remaining 64 items yielded a reliable estimate of health literacy, especially among participants with moderate and limited health literacy. The short version of the scale, which comprises 19 items with discrimination parameters ≥1.0, did not meet the standards for individual measurement (reliability ≥0.9). Nevertheless, the short version may still be effective for group comparisons [52].
In IRT, an item is useful only when it has good discrimination and its difficulty corresponds to a certain range in the ability scale: questions that are too hard or too easy provide little information [53]. However, if the discrimination is too high (i.e., greater than 2.5, as seen in clinical and psychological studies), the measured construct is often conceptually narrow. We limited the discrimination parameters to 0.5–2.0 because health literacy is a relatively broad concept. In this study, we identified items with inappropriate discrimination and difficulty. Most of them also had low factor loadings and correlation to the dimension score. However, the test used in the present study is time consuming. It usually took 30 min for an adult to complete the test; it took even longer for participants with limited literacy. Thus, in the future, it will be necessary to develop computerized adaptive testing and provide participants with short, tailored tests that have scores comparable to those of fixed-length tests.
Differential item functioning showed significant measurement invariance within both gender and race; however, the slope and threshold parameters were extremely close between the male and female as well as between the urban and rural groups. We observed no large differences between gender and race groups. The sample size in our study was sufficiently large to detect such slight differences. Thus, our results suggest that the Chinese Resident Health Literacy Scale may be efficiently applied for Chinese subjects of different genders and races for comparable scores.
The demographic factors associated with health literacy included education level, occupation, and annual income. Participants with higher education and better economic status were more likely to have adequate health literacy. Gender, age, race, and type of residence were found to be insignificant in the regression. The multilevel model identified an obvious intracluster correlation at the city level (primary unit in the sampling frame), with an intracluster correlation coefficient of 34.5 %. Health literacy is the outcome of health promotion, and both health literacy and socioeconomic factors are determinants of health. However, the potential of health education as a tool for promoting the social determinants of health has been neglected [54]. Health education should not focus only on changing personal lifestyles and improving compliance with disease management, but also on raising awareness of the social determinants of health [5].
Some limitations of this study deserve mention. First, we did not assess the content validity since the scale was initially developed by an expert panel from the Ministry of Health. Second, we did not perform repeated measures during the field study. Thus, the test-retest reliability was not determined. Third, as noted above, the test is time consuming: it usually took 30 min for an adult to complete and even longer for participants with limited literacy or other conditions.
Despite these limitations, this study has a number of implications. First, the original scale was found to be appropriate in terms of reliability and validity. We removed 16 items according to factor analysis and IRT, and the scores of the 64-item scale correlated highly with the scores of the original scale. Accordingly, the main conclusions of the 2012 National Health Literacy Survey were unaffected by validation of the scale it employed. Second, a shortened 19-item version was created because applying the original scale was very time consuming. The 19-item version was found to be slightly inferior to the original scale in terms of reliability (Cronbach’s alpha decreased from 0.95 to 0.88); however, it would still be effective for group comparisons and population studies. Third, the instruments used in the National Health Literacy Survey in 2008 and 2012 were different. Therefore, a direct comparison based on raw scores would be inappropriate. In the present study, IRT provided an opportunity for longitudinal comparison.
Conclusions
We evaluated and revised the Chinese Resident Health Literacy Scale based on IRT and classical test theory using a population-based sample in Hunan, China. The revised 64-item scale was found to have strong psychometric properties and be free of obvious measurement invariance within the race and gender groups employed in this study. This is the first investigation to evaluate and revise the instrument used in the 2012 National Health Literacy Survey in China. The findings of this study support use of the new instrument in research into health literacy in public health settings, and this investigation offers useful implications for future studies.
Abbreviations
IRT:
Item response theory
CFA:
Confirmatory factor analysis
Declarations
Acknowledgement
This work was supported by the Natural Science Foundation of China (81402770).
Authors’ Affiliations
(1)
Department of Epidemiology and Health Statistics, School of Public Health, Central South University
(2)
Clinical Epidemiology Program, Ottawa Hospital Research Institute
References
Chinese Ministry of Health. 66 tips of health: Chinese resident health literacy manual. Beijing: People’s Medical Publishing House; 2008.Google Scholar
Li XH. Brief introduction on identification and dissemination of the Basic Knowledge and Skill of People’s Health Literacy by Chinese government. Chinese J Health Educ. 2008;24(5):385–8.Google Scholar
World Health Organization. Health promotion glossary. Geneva: World Health Organization; 1998.Google Scholar
Institute of Medicine. Health literacy: a prescription to end confusion. Washington DC: National Academies Press; 2004.Google Scholar
Wang P, Mao Q, Tao M, Tian X, Li Y, Qian L, et al. Survey on the status of health literacy of Chinese residents in 2008. Chin J Health Educ. 2010;26(4):243–6.Google Scholar
Li Y. Introduction of 2012 Chinese residents health literacy monitoring program. Chin J Health Educ. 2014;30(6):563–5.Google Scholar
Davis TC, Crouch MA, Long SW, Jackson RH, Bates P, George RB, et al. Rapid assessment of literacy levels of adult primary care patients. Fam Med. 1991;23(6):433–5.PubMedGoogle Scholar
Arozullah AM, Yarnold PR, Bennett CL, Soltysik RC, Wolf MS, Ferreira RM, et al. Development and validation of a short-form, rapid estimate of adult literacy in medicine. Med Care. 2007;45(11):1026–33.View ArticlePubMedGoogle Scholar
Parker RM, Baker DW, Williams MV, Nurss JR. The test of functional health literacy in adults: a new instrument for measuring patients’ literacy skills. J Gen Intern Med. 1995;10(10):537–41.View ArticlePubMedGoogle Scholar
Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J. Development of a brief test to measure functional health literacy. Patient Educ Couns. 1999;38(1):33–42.View ArticlePubMedGoogle Scholar
Weiss BD, Mays MZ, Martz W, Castro KM, DeWalt DA, Pignone MP, et al. Quick assessment of literacy in primary care: the newest vital sign. Ann Fam Med. 2005;3(6):514–22.View ArticlePubMedPubMed CentralGoogle Scholar
Chew LD, Bradley KA, Boyko EJ. Brief questions to identify patients with inadequate health literacy. Fam Med. 2004;36(8):588–94.PubMedGoogle Scholar
Ishikawa H, Takeuchi T, Yano E. Measuring functional, communicative, and critical health literacy among diabetic patients. Diabetes Care. 2008;31(5):874–9.View ArticlePubMedGoogle Scholar
Dumenci L, Matsuyama R, Riddle DL, Cartwright LA, Perera RA, Chung H, et al. Measurement of cancer health literacy and identification of patients with limited cancer health literacy. J Health Commun. 2014;19 Suppl 2:205–24.View ArticlePubMedPubMed CentralGoogle Scholar
Ishikawa H, Nomura K, Sato M, Yano E. Developing a measure of communicative and critical health literacy: a pilot study of Japanese office workers. Health Promot Int. 2008;23(3):269–74.View ArticlePubMedGoogle Scholar
Huizinga MM, Elasy TA, Wallston KA, Cavanaugh K, Davis D, Gregory RP, et al. Development and validation of the Diabetes Numeracy Test (DNT). BMC Health Serv Res. 2008;8:96.View ArticlePubMedPubMed CentralGoogle Scholar
Hanson-Divers EC. Developing a medical achievement reading test to evaluate patient literacy skills: a preliminary study. J Health Care Poor Underserved. 1997;8(1):56–69.View ArticlePubMedGoogle Scholar
Malloy-Weir LJ, Charles C, Gafni A, Entwistle VA. Empirical relationships between health literacy and treatment decision making: a scoping review of the literature. Patient Educ Couns. 2015;98(3):296–309.View ArticlePubMedGoogle Scholar
Hanchate AD, Ash AS, Gazmararian JA, Wolf MS, Paasche-Orlow MK. The Demographic Assessment for Health Literacy (DAHL): a new tool for estimating associations between health literacy and outcomes in national surveys. J Gen Intern Med. 2008;23(10):1561–6.View ArticlePubMedPubMed CentralGoogle Scholar
Kutner M, Greenberg E, Jin Y, Paulsen C. The health literacy of America’s adults: results from the 2003 National Assessment of Adult Literacy. Washington, DC: National Center for Education Statistics; 2006.Google Scholar
Canadian Council on Learning. Health literacy in Canada: initial results from the International Adult Literacy and Skills Survey 2007. Ottawa: Canadian Council on Learning; 2007.Google Scholar
Australian Bureau of Statistics. Health literacy, Australia 2006. Canberra: Australian Bureau of Statistics; 2008.Google Scholar
Hahn EA, Choi SW, Griffith JW, Yost KJ, Baker DW. Health literacy assessment using talking touchscreen technology (Health LiTT): a new item response theory-based measure of health literacy. J Health Commun. 2011;16 Suppl 3:150–62.View ArticlePubMedPubMed CentralGoogle Scholar
Jordan JE, Osborne RH, Buchbinder R. Critical appraisal of health literacy indices revealed variable underlying constructs, narrow content and psychometric weaknesses. J Clin Epidemiol. 2011;64(4):366–79.View ArticlePubMedGoogle Scholar
Leung AY, Cheung MK, Chi I. Supplementing vitamin D through sunlight: associating health literacy with sunlight exposure behavior. Arch Gerontol Geriatr. 2015;60(1):134–41.View ArticlePubMedGoogle Scholar
Leung AY, Lou VW, Cheung MK, Chan SS, Chi I. Development and validation of Chinese Health Literacy Scale for Diabetes. J Clin Nurs. 2013;22(15–16):2090–9.View ArticlePubMedGoogle Scholar
Wang J, He Y, Jiang Q, Cai J, Wang W, Zeng Q, et al. Mental health literacy among residents in Shanghai. Shanghai Arch Psychiatry. 2013;25(4):224–35.PubMedPubMed CentralGoogle Scholar
Lam LT, Yang L. Is low health literacy associated with overweight and obesity in adolescents: an epidemiology study in a 12-16 years old population, Nanning, China, 2012. Arch Public Health. 2014;72(1):11.View ArticlePubMedPubMed CentralGoogle Scholar
Simon MA, Li Y, Dong X. Levels of health literacy in a community-dwelling population of Chinese older adults. J Gerontol A Biol Sci Med Sci. 2014;69 Suppl 2:S54–60.View ArticlePubMedPubMed CentralGoogle Scholar
Guo SJ, Yu XM, Sun YY, Nie D, Li XM, Wang L. Adaptation and evaluation of Chinese version of eHEALS and its usage among senior high school students. Chin J Health Educ. 2013;29(2):106–8.Google Scholar
Xu WH, Rothman RL, Li R, Chen Y, Xia Q, Fang H, et al. Improved self-management skills in Chinese diabetes patients through a comprehensive health literacy strategy: study protocol of a cluster randomized controlled trial. Trials. 2014;15:498.View ArticlePubMedPubMed CentralGoogle Scholar
Lin X, Wang M, Zuo Y, Li M, Lin X, Zhu S, et al. Health literacy, computer skills and quality of patient-physician communication in Chinese patients with cataract. PLoS One. 2014;9(9), e107615.View ArticlePubMedPubMed CentralGoogle Scholar
Sun X, Yang S, Fisher EB, Shi Y, Wang Y, Zeng Q, et al. Relationships of health literacy, health behavior, and health status regarding infectious respiratory diseases: application of a skill-based measure. J Health Commun. 2014;19 Suppl 2:173–89.View ArticlePubMedGoogle Scholar
Sun X, Shi Y, Zeng Q, Wang Y, Du W, Wei N, et al. Determinants of health literacy and health behavior regarding infectious respiratory diseases: a pathway model. BMC Public Health. 2013;13:261.View ArticlePubMedPubMed CentralGoogle Scholar
Powers BJ, Trinh JV, Bosworth HB. Can this patient read and understand written health information. JAMA. 2010;304(1):76–84.View ArticlePubMedGoogle Scholar
Ye XH, Yang Y, Gao YH, Chen SD, Xu Y. Status and determinants of health literacy among adolescents in Guangdong, China. Asian Pac J Cancer Prev. 2014;15(20):8735–40.View ArticlePubMedGoogle Scholar
Wang C, Li H, Li L, Xu D, Kane RL, Meng Q. Health literacy and ethnic disparities in health-related quality of life among rural women: results from a Chinese poor minority area. Health Qual Life Outcomes. 2013;11:153.View ArticlePubMedPubMed CentralGoogle Scholar
Wang X, Guo H, Wang L, Li X, Huang M, Liu Z, et al. Investigation of Residents’ Health Literacy Status and Its Risk Factors in Jiangsu Province of China. Asia Pac J Public Health. 2015;27(2):NP2764–2772.View ArticlePubMedGoogle Scholar
Kish L. A procedure for objective respondent selection within the household. J Am Statist Assoc. 1949;44(247):380–7.View ArticleGoogle Scholar
Xiao L, Cheng YL, Ma Y, Chen GY, Hu JF, Li YH, et al. A study on applying Delphi method for screening evaluation indexes of health literacy of China adults. Chin J Health Educ. 2008;24(2):81–4.Google Scholar
Reise SP, Waller NG. Item response theory and clinical measurement. Annu Rev Clin Psychol. 2009;5:27–48.View ArticlePubMedGoogle Scholar
Hambleton RK, Swaminathan H. Item response theory: principles and applications. Boston, Massachusetts: Kluwer-Nijhoff; 1985.View ArticleGoogle Scholar
Baker FB. The basics of item response theory. 2nd ed. The United States of America: ERIC Clearinghosue on Assessment and Evaluation; 2001.Google Scholar
Richard T. Interpretation of the correlation coefficient: a basic review. J Diagn Med Sonog. 1990;6(1):35–9.View ArticleGoogle Scholar
Gerbing DW, Anderson JC. An updated paradigm for scale development incorporating unidemensionality and its assessment. J Marketing Res. 1988;25(2):186–92.View ArticleGoogle Scholar
Gorsuch RL. Exploratory factor analysis: its role in item analysis. J Pers Assess. 1997;68(3):532–60.View ArticlePubMedGoogle Scholar
Velicer WF, Fava JL. Effects of variable and subject sampling on factor pattern recovery. Psychol Methods. 1998;3(2):231–51.View ArticleGoogle Scholar
Thomas ML. The value of item response theory in clinical assessment: a review. Assessment. 2011;18(3):291–307.View ArticlePubMedGoogle Scholar
Nutbeam D. Health literacy as a public health goal: a challenge for contemporary health education and communication strategies into the 21st century. Health Promot Int. 2000;15(3):259–67.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
