- Research article
- Open Access
The development and evaluation of a sub-health self-rating scale for university students in China
BMC Public Health volume 19, Article number: 330 (2019)
Sub-health status is defined as declines in vitality, physiological function and capacity for adaptation, but without the presence of clinical or sub-clinical disease. We have developed and evaluated a comprehensive questionnaire, the Sub-Health Self-Rating Scale (SSS), to assess sub-health status in university students.
The items for the draft questionnaire were discussed in focus groups. The WHOQOL-BREF was selected as the validity reference. From a professional perspective and large sample evaluation, the scale ultimately consisted of 58 items. The reliability and validity of the SSS was examined in undergraduate students and 1000 questionnaires were randomly selected from the samples for expert evaluation.
Cronbach’s α of the total scale was 0.942. The dimensions of physiological, psychological and social had high reliability: 0.915, 0.856 and 0.850, respectively. Based on scree plot and related theories, there were 10 factors to be extracted. The correlation coefficient between the total scale and sub-scale was high. The dimensions of physiological, psychological and social had high correlations with the total scale: 0.929, 0.803 and 0.774, respectively. The sub-health cut-off point of the total scale was 72; for the physiological field, it was 72; for the psychological field, it was 60; and the social field, it was 56. The fit between the expert evaluation method and the scale method was 0.758. The lower the score, the worse the health condition.
We established and evaluated a valid instrument (SSS) that encompasses physiological, psychological and social factors to investigate sub-health status. It is short and easy to complete, and therefore suitable for use with undergraduate students.
The World Health Organization (WHO) defined health in its broader sense in 1946 as “a state of complete physical, mental, and social well-being and not merely the absence of disease or infirmity”  . Although this definition lacks operational value, it remains the most enduring of all definitions. With greater understanding of health, the definition has deepened to take account of sub-health status (SHS), which is an intermediate state between disease and health, as proposed by Wang .
In China, as a result of rapid economic development, many people face pressures from work and home lives, perhaps working overtime or through the night. As a result, the body clock is affected, the immune system declines [3, 4], and some appear with SHS  . TCM clinical guidelines for sub-health released by the China Association of Chinese Medicine pointed out that the SHS is one that shows declines of vitality, physiological function and capacity for adaptation, but it is not defined as clinical or sub-clinical disease . Over the years, the concept of sub-health has been widely accepted in many other countries such as Japan , Canada and Australia [8, 9]. For example, a Japanese scholar has conducted a physical check-up of sub-health and investigated the diagnostic criteria for sub-health for Japanese people, and established a comprehensive sub-health examination system, the Complete Sub-Health Check-up, which includes ten major examination items .
According to our survey in Guangdong, a sub-health state affected 65.1% of the total survey population . Although the prevalence rate of sub-health is high, there has been a lack of objective clinical diagnostics for sub-health. In addition, its clinical manifestation is complex, subjective and difficult to quantify. It is necessary to develop an objective, simple, self -testing and convenient scale to confirm the status of sub-health.
During such an important period of their lives, undergraduate students repeatedly face changes and challenges. During this developmental stage, a number of students appear with physical, psychological and social problems. Our previous investigations revealed that a number of undergraduates displayed sub-health state [11,12,13]. It is highly appropriate for undergraduate students to use a self-assessment scale to evaluate their health status.
There are some sub-health state questionnaires already established and evaluated in China, such as SHSQ-25, MSQA and SHMS V1.0 [14,15,16], but these are not suitable for use with undergraduate students. SHSQ-25 cannot completely measure the physiological, psychological and social aspects and MSQA is aimed at adolescents. As a result, it was essential to develop a reliable and valid instrument to assess SHS. Therefore, we developed and evaluated a comprehensive questionnaire, the Sub-Health Self-Rating Scale (SSS), to assess SHS for undergraduates in China (Additional file 1). It is short and easy to complete, and therefore suitable to use in studies of undergraduate students.
The items for the draft questionnaire were identified according to the WHO’s definition of health and Chinese social culture. In addition, we conducted a literature review and solicited expert opinion (comprising two health managers, four physicians, a psychologist and two clinical epidemiologists). The final questionnaire included 83 items.
The second stage was to evaluate the scale. A total of 6000 questionnaires were handed out in a medical university; 5792 questionnaires were handed back (96.53% response rate). Stratified sampling is used to sample at each grade. There is no difference in the age and gender of the sample. The 193 students who did not complete the questionnaires were excluded. Ultimately, 5599 questionnaires were analyzed. Cronbach’s α of the draft questionnaire was 0.949. Cronbach’s α values of the physiological, psychological and social subs-cale were 0.906, 0.881 and 0.865, respectively. Construct validity used exploratory factor analysis (Varimax). According to the value of eigenvalues above 1, we extracted 16 factors (include positive feeling factor, passive feeling factor, pain factor, menstruation factor, sleep factor, capability& self-respect factor, fatigue factor, social relationship factor, urine factor, eyes factor, constipation factor, diarrhea factor, anxious factor, irritable factor, forced factor, hair factor). The cumulative proportion was 52.29%. The Kaiser-Meyer-Olkin (KMO) statistic was 0.960. The criterion validity showed that the scale was highly related to the WHOQOL-BREF (r > 0.60), which was statistically significant (P < 0.001). The correlation coefficients between WHOQOL-BREF and the scale were 0.812. There was a notable difference in mean scores between the health and sub-health state in the dimensions (t > 35.00, P < 0.000). A total of 63 items were eventually extracted from the draft SSS.
The third stage was to determine the final items. After the draft questionnaire was evaluated, focus groups were again used to discuss whether the items were relevant, clear, unambiguous and written in a language that would be understandable to potential respondents. According to the results of factor analysis, we increased 1 factor (skin factor), put together 2 factors (constipation factor, diarrhea factor) and removed 6 factors (menstruation factor, eyes factor, anxious factor, irritable factor, forced factor, hair factor). From a professional perspective, we deleted nine items, maintained 54 items, and added four items for skin factors. The scale ultimately consisted of 58 items, which had satisfactory sensibility, representation and internal consistency.
Determining the validity of the questionnaire
Seven thousands questionnaires were handed out during the 9 month study period; 6232 completed responses were received (89.03% response rate) and 6205 questionnaires were analyzed.
The SSS was filled in within 20 to 30 min of the student being in the classroom. The students volunteered for our study. Students under 18 years old provided verbal consent from next of kin, carers or guardians. Verbal consent was deemed appropriate because the students volunteered for the study. If they did not want to take part in the questionnaire survey, they could refuse. Secondly, our purpose was to study the health status of the undergraduate students rather than to intervene. All student data were kept in strictest confidence. Although some of the students were under 18, they were mature and understood the aim of the study. The study was approved by the Ethics Committee of Nanfang Hospital in Guangzhou, China  LunShenZi (No. 035). The ethics committee also approved the consent procedure.
There are socio-demographic indicators in the scale, including age, gender, grade, birthplace, marriage status and medical history.
The SSS consisted of 58 items, which were divided into three symptom dimensions (physiological, psychological, social) and ten factors (Table 1). The ten factors were labeled as follows – F1: positive feeling factor, F2:passive feeling factor, F3:pain factor, F4: digestive factor, F5: sleep factor, F6: capability& self-respect factor, F7: fatigue factor, F8: social relationship factor, F9: skin factor, F10: urine factor. Each item had five answer categories corresponding to the degree of each symptom (never, occasionally, sometimes, constantly and always). In the data analysis, never was assigned to 5, occasionally to 4, sometimes to 3, constantly to 2 and always to 1. Then, we added up the scores of all symptoms. The scores of 16 of the symptoms had to be inversely transformed before adding up.
A total of 1000 questionnaires were randomly selected from the valid 6205 cases for expert evaluation to determine the fit between scale and expert evaluation.
Computational method for the total and sub-scale
The raw score is the sum of the item score in the total scale or the sub-scale. The scores of 16 of the symptoms are inversely transformed before adding up. The conversion formula is as follows:
The linear T score formula is as follows: T = 50 + 10 (X + −X)/S, where X is the raw score, −X is the total mean score, S is the population standard deviation.
All the data were analyzed using SPSS version 13.0. The reliability verification applied a generalizability coefficient, and the validity verification included factor analysis to verify the validity, criteria validity and distinguish validity. Five methods based on the dispersion, correlation coefficient, factor analysis, Student’s t-test and Cronbach’s α were employed to analyze the items. The scale was conducted feasibility analysis. Exploratory and confirmatory factor analysis of 1-order factor and 2-order factor were used to evaluate the infra-structure of the scale. The reliability analysis included Cronbach’s α, split-half reliability and the mean inter-item correlation. The validity analysis included content and construct validity.
Characteristics of participants
A total of 6205 university students from grades 1 to 5 were recruited to participate in the survey. The exclusion criteria for participants included having organic disease diagnosed in a clinical laboratory examination. The 176 students, whose medical reports were abnormal, were excluded. Ultimately, 6029 students (2606 male and 3423 female), aged 15 to 29 years (mean age 20.88 years, SD = 1.5) were analyzed. The acceptance rate and finish rate were 89.03 and 99.67%, respectively.
Internal consistency analysis shows that Cronbach’s α of the total scale was 0.942. The dimensions of physiological, psychological and social had a high reliability: 0.915, 0.856 and 0.850, respectively. Cronbach’s α of each factor was between 0.7 and 0.88. The split-half reliability of the total scale was 0.938. The dimensions of physiological, psychological and social were 0.933, 0.890 and 0.881, respectively. The split-half reliability of each factor was between 0.73 and 0.90.
The KMO measure of sampling adequacy was 0.947 and the Bartlett test of sphericity was statistically significant (χ2 = 7778.7; P = 0.000). The method of construct validity was exploratory factor analysis (Varimax). Confirmatory factor analysis (CFA) showed a reasonable fit of the data in the factor structure: χ2 = 222,264.171, RMSEA = 0.0536, GFI = 0.867, AGFI = 0.853, CFI = 0.965. There were 12 factors for which the eigenvalue was above 1. The cumulative proportion was 60.53%. Based on scree plot and related theories, 10 factors were extracted. The cumulative proportion was 56.63%. The correlation coefficient between the total scale and sub-scale was high. The dimensions of physiological, psychological and social were 0.929, 0.803, and 0.774, respectively. The correlation coefficient between each factor and its own domain was 0.52–0.89 (Table 2). The correlation coefficient between each item and factor was 0.51–0.88 (Table 3).
Establishing the cut-off point for sub-health among college students
Our previous studies show that the total prevalence of sub-health was between 57.2 and 65.1% [10,11,12,13, 17,18,19,20,21,22]; the prevalence of physiological sub-health was between 42.5 and 55.1%; the prevalence of psychological sub-health was between 32.5 and 35.8%; the prevalence of social sub-health was between 32.5 and 40.7%. The percentage method was used to determine the cut-off point for sub-health in SSS: the reference line of the total scale is 60%, was 50% for the physiological field, was 35% for the psychological field, and was 35% for the social field. The raw score, transformed score and T score of the total scale were 224, 72, and 52, respectively. The raw score, transformed score and T score of the physiological sub-scale were 128, 72 and 50; of the psychological sub-scale, they were 48, 60 and 46; of the social sub-scale, they were 26, 56 and 45, respectively. The lower the score, the worse the health condition (Table 4).
Fit between the expert evaluation method and scale method
There are five expert evaluated 1000 questionnaires which were randomly selected from the valid 6205 cases. The expert evaluation of sub-health status was performed according to the clinical guidelines for Sub-health published by the China Association of Chinese Medicine . The 5 expert evaluated 200 scales respectively. If the expert cannot sure, the scale will be discussed together. If they are still uncertain, they will interview the student for confirmation. The expert evaluation of 1000 questionnaires showed that the total prevalence of sub-health was 64.49%; physiological sub-health was 47.78%; psychological sub-health was 37.15%; and social sub-health was 36.84%. The fit between expert evaluation method and the scale method was 0.758. The fit in the physiological, psychological and social sub-scale was 0.815, 0.797 and 0.787, respectively.
The ROC curve to detect the cut-off point of the scale
We had presented the ROC curve of the 1000 questionnaires to detect the cut-off point of the scale (Figs. 1, 2, 3, 4). The area under the curve in the physiological, psychological, social sub-scale and total scale was 0.987, 0.985, 0.990, 0.958, respectively (Table 5). The SSS was significantly positive to evaluate the sub-health state (P = 0.000). Table 6 shows the Youden’s index in each dimension. The maximum value of Youden’s index in the physiological, psychological, social sub-scale and total scale was 0.90, 0.94, 0.94, 0.81 (Table 6). The optimum cut-off point in the physiological, psychological, social sub-scale and total scale was 126.5, 48.5, 25.5 and 221.5. Ultimately, according to the standard above, 2150 were healthy and 3879 were ‘sub-healthy’ in 6029 students.
Sub-health status is considered to be an intermediate status between disease and health. In the traditional Chinese medicine guidelines released by the China Association of Chinese Medicine (CACM), it is characterised by a decline in vitality, in physiological function and in the capacity for adaptation. The prevalence rate of Sub-health status was high (between 57.2 and 65.1% [10,11,12,13, 17,18,19,20,21,22]). Although the prevalence of sub-health is high, there has been a lack of objective clinical diagnostics for it. A number of Sub-health status questionnaires have been established and evaluated in China, such as Suboptimal Health Status Questionnaire (SHSQ)-25, Multidimensional Sub-health Questionnaire of Adolescents (MSQA) and Sub-Health Measurement Scale V1.0 (SHMS V1.0) [14,15,16]; however, (SHSQ)-25 is targeted at physiological and psychological Sub-health state and MSQA is aimed at adolescents. SHMS V1.0, on the other hand, is also suitable for adolescents. These are not suitable for use with undergraduate students.
The aim of the survey was to develop a valid, simple questionnaire for measuring SHS. Since 2005, our group has been dedicated to developing a sub-health scale. The item pool of the draft questionnaire was discussed in focus groups. WHOQOL-BREF was selected as the validity reference. From a professional perspective and large sample evaluation, the scale ultimately consisted of 58 items. The reliability and validity of the SSS was examined in relation to undergraduate students, and the divided converted score of the total scale was 72.
Firstly, the draft came from a literature review, clinical experience and practice, consultations with experts and closed questionnaires and health-related quality of life scales. The experts in different fields discussed, modified and deleted items, thus ensuring the scale and content of the project more accurately reflected the actual situation of sub-health. The draft scale was revised and modified repeatedly, leading to the final 83 items of the SSS.
Secondly, the draft scale was evaluated in a large sample. Item analysis indicated that the items in the draft SSS had satisfactory sensibility, representation and internal consistency. The index of reliability included Cronbach’s α and split-half reliability, which achieved the psychometric demand. The structure validity achieved excellent levels of content, construct and criterion-related validity. The draft SSS accurately reflected the characteristic of sub-health. As a result, it can be used to evaluate sub-health among undergraduate students. However, the cumulative proportion was not high enough, and the models of SSS need to be revised and reduced. Five methods (dispersion, correlation coefficient, factor analysis, t-test and Cronbach’s α) were employed to analyze the 83 items, 63 of which were selected. From a professional perspective, we deleted nine items, which were not representative or have universal application. We added four items for skin factor. The scale ultimately consisted of 58 items.
Thirdly, a large sample survey in China showed that the SSS was highly reliable and valid. In addition, the proportions of acceptance rate (89.03%) and finish rate (99.67%) were high, indicating that participants responded carefully to the questions. The overall Cronbach’s α was good, 0.942. When internal consistency is analyzed in the sub-scales, Cronbach’s α of the three sub-scales was relatively high (0.850–0.915). The homogeneity reliability test showed that reliability coefficients were > 0.70 and total scale reliability coefficients were > 0.90. The scale had high homogeneity. Each factor was highly related to its own domain, and had a low correlation with other domains. The correlation coefficient between the sub-scale and the total scale was also high. The multidimensional structure of SSS was further checked by CFA, and a good fit of the data was observed. The total scale and sub-scales were highly related to the WHOQOL-BREF. The questionnaire was also able to discriminate between healthy and ‘sub-healthy’ undergraduates.
Fourthly, the cut-off point for sub-health in SSS was established by a percentage method. At present, the methods of building the cutoff point of scale, are mainly including dispersion, percentile, index, and related methods. Because each method has different characteristics, generally according to the distribution of the sample and the nature of the scale, different methods are used. The scale is related to the physical and psychological indicators of the human body. The samples of these indicators are normally distributed, also non-normally distributed. The percentile method is suitable for both normally distributed samples and non-normally distributed samples [24, 25]. Therefore, we selected the percentile method to build the cutoff point of scales.
At last, the fit between the expert evaluation method and the scale method was up to 0.70, which is appropriate and valid. The ROC curve of the 1000 selected questionnaires was presented to detect the cut-off point of the scale. Youden’s index is often used in conjunction with ROC analysis. The index is defined for all points of an ROC curve, and the maximum value of the index may be used as a criterion for selecting the optimum cut-off point when a diagnostic test gives a numeric rather than a dichotomous result [26,27,28]. The maximum value of Youden’s index in psychological and social sub-scale is equal to our percentage method result. Our percentage method result in physiological sub-scale and total scale is close to the maximum value of Youden’s index. The SSS scale can appropriate evaluate the sub-health state of students.
First of all, this scale is only a self-assessment scale, suitable for the student population. Only a few schools have been investigated. The next step is to expand the sample size to investigate more schools and improve the scale. Second, the current There is no objective measurement standard for sub-health. It is difficult to accurately measure the sub-health status only through the self-rating scale. Some people, who is in psychological and social mental illnesses, may be mistaken for sub-health. As a result, we must use a variety of methods and tools to measure sub-health status. For example, combined with current medical history, past history, physical examination results, plus sub-health self-rating scale, anxiety self-rating scale, depression self-rating scale, WHO neurosis screening table, etc. for comprehensive evaluation, and finally obtain more accurate results.
We established and evaluated a valid instrument, SSS, which encompasses the domains of physiological, psychological and social health, to investigate SHS. The questionnaire is short and easy to complete, and is therefore suitable for use with undergraduate students.
Multidimensional Sub-health Questionnaire of Adolescents
- SHMS V1.0:
Sub-Health Measurement Scale V1.0
Suboptimal Health Status Questionnaire-25
Sub-Health Self-Rating Scale
Traditional Chinese Medicine
the WHO Quality of Life Brief Scale
Grad FP. The preamble of the constitution of the World Health Organization. Bull World Health Organ. 2002;80(12):981–4.
Wang YX. Sub-health--New concept of health in the 21st century. Jiangxi: Jiangxi Science and Technology Press; 2002.
Gonzalez BD, et al. Cancer survivors in the workplace: sleep disturbance mediates the impact of cancer on healthcare expenditures and work absenteeism. Support Care Cancer. 2018;26(12):4049–55.
Kwekkeboom KL, et al. The role of inflammation in the pain, fatigue, and sleep disturbance symptom cluster in advanced Cancer. J Pain Symptom Manag. 2018;55(5):1286–95.
Liang YZ, et al. Relationship between stress-related psychosocial work factors and suboptimal health among Chinese medical staff: a cross-sectional study. BMJ Open. 2018;8(3):e018485.
Medicine CAOC. The TCM clinical guidelines of sub-health. Beijing: Chinese Press of Traditional Chinese Medicine; 2006.
Ke B, Liang Y. Anti-aging and complete sub-health checkup. Clin Funct Nutriology. 2011;3(3):137–40.
Dunstan RH, et al. Development of a complex amino acid supplement, fatigue Reviva, for oral ingestion: initial evaluations of product concept and impact on symptoms of sub-health in a group of males. Nutr J. 2013;12:115.
Davy CP, Patrickson M. Implementation of evidence-based healthcare in Papua New Guinea. Int J Evid Based Healthc. 2012;10(4):361–8.
Sun XM, et al. A epidemiological study of sub-health in Guangdong. Shandong Med J. 2008;48(4):59–60.
Bi J, et al. Association of lifestyle factors and suboptimal health status: a cross-sectional study of Chinese students. BMJ Open. 2014;4(6):e005156.
Chen JY, et al. Associations between health-promoting lifestyle and suboptimal health status in Guangdong: a cross sectional study. Nan Fang Yi Ke Da Xue Xue Bao. 2016;36(4):538–43.
Chen J, et al. The role of healthy lifestyle in the implementation of regressing suboptimal health status among college students in China: a nested case-control study. Int J Environ Res Public Health. 2017;14(3):240.
Yan YX, et al. Development and evaluation of a questionnaire for measuring suboptimal health status in urban Chinese. J Epidemiol. 2009;19(6):333–41.
Cao H, et al. Problematic internet use in Chinese adolescents and its relation to psychosomatic symptoms and life satisfaction. BMC Public Health. 2011;11:802.
Xue J, et al. Assessment of the reliability and validity of the sub-health measurement scale Version1.0. Nan Fang Yi Ke Da Xue Xue Bao. 2011;31(1):33–8.
Li J, et al. Analysis of the clinical manifestations of sub-healthy status in a hospital. Guangdong Med J. 2007;28(5):800–2.
Huo YH, et al. Analysis of clinical manifestations of subheath status in a medical college of Guangdong Province. Nan Fang Yi Ke Da Xue Xue Bao. 2007;27(4):448–9.
Chen JY, et al. Effect of health-promoting lifestyle on outcomes of suboptimal health status. Nan Fang Yi Ke Da Xue Xue Bao. 2016;37(2):184–91.
Cheng J, et al. Self-rated health status and subjective health complaints associated with health-promoting lifestyles among urban Chinese women: a cross-sectional study. PLoS One. 2015;10(2):e0117940.
Wang T, et al. Effects of TCMC on transformation of good health status to suboptimal health status: a nested case-control study. Evid Based Complement Alternat Med. 2015;2015:259727.
Wu S, et al. Work-recreation balance, health-promoting lifestyles and suboptimal health status in southern China: a cross-sectional study. Int J Environ Res Public Health. 2016;13(3):339.
Caoc M. The TCM clinical guidelines of suboptimal health status. Beijing: China Press TCM; 2006.
Liang Z. Analysis on the effect of assessment Normal of National Citizens’ status. J Shenyang Sport Univ. 2007;26(2):16–8.
Jiang C, et al. Study of National Physical Fitness Evaluation Standard. Sport Sci. 2004;24(3):33–6.
Schisterman EF, et al. Optimal cut-point and its corresponding Youden index to discriminate individuals using pooled blood samples. Epidemiology. 2005;16(1):73–81.
Powers and D.M. W. Evaluation: from precision, recall and F-score to ROC, Informedness, Markedness & Correlation. J Mach Learn Technol. 2011;2(1):37–63.
Thompson T, et al. The Weiss functional impairment rating scale-parent form for assessing ADHD: evaluating diagnostic accuracy and determining optimal thresholds using ROC analysis. Qual Life Res. 2017;26(7):1879–85.
We thank our study participants and administrators of selected units for their contribution to the data collection, case ascertainment.
This work was supported by NSFC-Guangdong joint fund (No. U1132001), National Science Foundation of China (No. 81703891), Natural Science Foundation of Guangdong Province, China (No.2016A030310311) and TCM State Administration Foundation of Guangdong Province, China (No.20161014) in the design, collection, analysis and writing of manuscript.
Availability of data and materials
Data are available upon reasonable request from the corresponding author.
Ethics approval and consent to participate
The study was approved by the ethics committee of Nanfang Hospital in Guangzhou, China. It complied with the principles outlined in the Helsinki Declaration. All procedures were performed in accordance with the ethical standards. Written consent was obtained from each participant. Students under 18 years old provided verbal consent from next of kin, carers or guardians. The Ethics Committee thought students under 18 were mature enough to understand the aim of the study. Verbal consent was deemed appropriate because the students volunteered for the study. If they did not want to take part in the questionnaire survey, they could refuse.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Bi, Jl., Chen, J., Sun, Xm. et al. The development and evaluation of a sub-health self-rating scale for university students in China. BMC Public Health 19, 330 (2019). https://doi.org/10.1186/s12889-019-6650-3
- University students