Can global or national disability weights represent provincial level?
BMC Public Health volume 23, Article number: 461 (2023)
A universal set of disability weights(DWs) is mainly based on the survey of North America, Australia and Europe, whereas the participants in Asia was limited. The debate hasn’t yet settled whether a universal DW is desirable or useful.The focus of the debate is its representativenes-s.After all, the DWs come from people's subjective evaluation of pain, and they may vary according to cultural background.The differences of the DWs could have implications for the magnitude or ranking of disease burdens.The DWs of Anhui Province has not been completely presented.This paper aims to obtain the DWs suitable for the general population of Anhui Province of China, and attempts to explore the differences between different DWs by comparing the DWs with the similar-cultural background and the DWs with cross-cultural background.
A web-based survey was conducted to estimate the DWs for 206 health states of Anhui province in 2020. Paired comparison (PC) data were analyzed and anchored by probit regression and fitting loess model. We compared the DWs in Anhui with other provinces in China and those in Global burden of disease (GBD) and Japan.
Compared with Anhui province, the proportion of health states which showed 2 times or more differences ranged from 1.94% (Henan) to 11.17% (Sichuan) in China and domestic provinces. It was 19.88% in Japan and 21.51% in GBD 2013 respectively. In Asian countries or regions, most of the health states with top 15 DWs belonged to the category of mental, behavioral, and substance use disorders. But in GBD, most were infectious diseases and cancer. The differences of DWs in neighboring provinces were smaller than other geographically distant provinces or countries.
PC responses were largely consistent across very distinct settings,but the exceptions do need to be faced squarely.The differences of DWs among similar-cultural regions were smaller than cross-cultural regions. There is an urgent need for relevant gold standards.
In terms of resource allocation, impact monitoring of medical reform, effect evaluation of intervention measures and determination of health priorities, the burden of disease is an important reference [1,2,3,4]. Disability-adjusted life years (DALYs)is a generally recognized measure of disease burden, which comprehensively considers the impact of death and disability on health [1, 5, 6]. The DWs are an important parameter for calculating DALYs, and are used to quantify health levels associated with non fatal outcomes. The DWs range from 0(equivalent to full health) to 1(equivalent to death). A universal set of global DW was estimated in GBD study. As an important parameter, the debate hasn’t yet settled whether a universal set of DWs is desirable or useful [5,6,7,8,9,10], although many improvements have been made since it was first proposed. The global DWs are mainly based on the survey of North America, Australia and Europe [2, 3, 5, 7, 11, 12]. However, the DWs come from people's subjective evaluation of pain. People's perception of pain will be different in different countries or regions due to differences in cultural background [10,11,12]. The differences of the DWs could have substantial implications for the magnitude or ranking of disease burdens . Korea [13,14,15], Japan  and China  have successively published their DWs. China is a populous country with a vast territory, and there are similar-cultural backgrounds among domestic provinces, but there are still some differences. Anhui’s data hasn’t been fully presented in Chinese study, this paper aims to obtain the DWs suitable for the general population of Anhui Province of China, and attempts to explore the differences between different DWs by comparing the DWs with the similar-cultural background and the DWs with cross-cultural background.
Study design and participants
A web-based survey of permanent residents aged 18–69 in Anhui province in 2020 was conducted through a self-administered questionnaire. The exclusion criteria for the participants were as follows: residents living in functional areas, such as work sheds, military, student dormitories, nursing homes; pregnant women; patients with mental disorders; uncooperative participants(??).
The questionnaire included a fixed personal demographics information and a PC which include 18 questions. Each question was composed of two options of different health states randomly selected from 206 health states, participants chose the healthier through their subjective feelings. Questions 3, 10 and 16 were used for quality control, and their contents overlapped with other questions.
Description of health states and sample size
Supplementary Table S1 showed lay descriptions for 206 health states of the present study which is mainly based on GBD study. Each pair was required to be drawn no less than 10 times to ensure the stability of parameters, the minimum sample size was 28,150 (15 valid health states pairs per questionnaire, 206 × 205 ÷ 15 × 10 = 28,150).
Survey procedure and quality control
Participants filled in the questionnaire through the national or provincial two-dimensional code of electronic questionnaire identified by mobile phone, and each we-chat (a popular instant messenger app in China) account could only fill in the questionnaire once. The questionnaires with inconsistent answers for the quality control questions or too short response time were eliminated at the end of the process. The project areas will carry out more targeted promotion and control deviation according to the weekly feedback of the project team throughout the survey.
PC data were used to draw heat map and evaluate the data quality. Probit regression was constructed through PC data to obtain the coefficients that needs to be anchored in the interval of 0–1. Taking the coefficients as the independent variable, and the DWs of corresponding 172 health states in GBD 2013 transformed by logit as the dependent variable, loess model was used for anchoring the DWs. Using the random concept of Monte Carlo, 100 random probit regression coefficients were generated from the mean value and standard error of regression coefficients in each health state, so as to determine the 95% uncertainty interval (UI) of DW in each health state. Pearson correlation analysis was used to examine the correlation between the DWs of Anhui Province and that of other studies, P-values below 0.01 were deemed statistically significant. All of the analyses were performed with R (version 4.1.2).
Analysis of DWs in anhui province
39,446 valid questionnaires were collected in Anhui province, which including 6258 questionnaires generated through the national two-dimensional code. Supplementary Table S2 showed the basic characteristics of participants and the comparison with the provincial population characteristics. Female or the participants aged 30–49 accounted for about 60%. Less than 10% were people with low education level. Except for the educational level, there was no significant difference between the sample population and the provincial population in terms of gender, age, ethnic and regional distribution(p-values > 0.01).The smooth color transition in the heat map showed that the data had small measurement error and good internal consistency (Fig. 1).
The DWs in Anhui province ranged from 0.003 (UI 0.002–0.008) for mild impairment of distance vision and mild anemia to 0.668 (UI 0.512–0.784) for severe heroin and other opioid dependence. DWs in different health states by descending order didn’t violate common sense, the only exception was the DW of cancer was lower than that of mild anemia. For the same states with different severity, only in 5 states such as tuberculosis with or without HIV infection, terminal phase with or without medication (for cancers and end-stage kidney or liver disease), varying degrees of hearing loss or distance vision and neck pain, the orders of DW didn’t increase with severity (Supplementary Table S3).
This study involved 206 health states of 11 kinds of diseases, which is completely consistent with Chinese study  except for the serial number of health states, of which 172 were consistent with the GBD 2013 and 161 with Japanese study  (Supplementary Table S4). A more in-depth comparison will be made for the following aspects:
Comparison with other provinces in China
Supplementary Figure SF1 showed the DWs of Anhui province were highly correlated with the results of 8 provinces with provincial representativeness sample in China (r ≥ 0.985, p < 0.001). Compared with them, none of Anhui's DW was 2 times or more. In Table 1, the numbers of health states which showed 2 times or more differences compared with Anhui were 4 (1.94%) in neighboring Henan, 7 (3.40%) in Fujian and Liaoning, 10 (4.85%) in neighboring Jiangsu, 14 (6.80%) in neighboring Hubei, 17 (8.25%) in Yunnan, 20 (9.71%) in Hunan, 23 (11.17%) in Sichuan respectively. The DWs for mild impairment distance vision, mild anemia in other provinces except Henan were more than 3 times, the largest proportional difference was 4.33 times.
Ignoring the differences of their specific DWs and the ranking, most of the health states involving the top 15 and last 15 DWs overlapped among provinces. Nearly 56.67% of the top 15 DWs in these provinces belonged to the category of mental, behavioral, and substance use disorders. About 90% of the last 15 DWs belonged to other diseases. The DWs for mode rate to severe heroin and other opioid dependence ranked first in these provinces (Table 2).
Comparison with Asian countries and GBD 2013
Supplementary Figure SF2 showed the DWs of Anhui province were highly correlated (p < 0.001) with that of China (r = 0.989), GBD2013 (r = 0.871) and Japan (r = 0.861). Supplementary Figure SF3, SF4 and SF5 showed the comparison results between Anhui and other studies respectively. More details were shown in Supplementary Table S3.
Compared with Chinese DWs, none of Anhui's DW was 2 times or more. The DW for 8 (3.88%) health states in China was 2 times or more than that in Anhui, of which there are 2 states with a difference of more than 3 times, mild impairment of distance vision and mild anemia respectively(SF3).
Compared with Japanese DWs, 32 (19.88%) states showed twice as much differences or more, half of which were higher than in Japan, and 4 of them in Anhui province were more than 3 times. 6 were only 1/3 or even 1/6 of Japanese DW (SF4).The largest difference was the DW of herpes zoster(0.181[0.123–0.257]) in Japan which is 6.70 times higher than that of Anhui.
Compared with GBD 2013, 37 (21.51%) states showed twice as much differences or more, of which 24 (64.86%) had a higher DW in Anhui province, and the DW of 8 states of Anhui province were more than 3 times, the largest difference was the DW of mild Cannabis dependence of Anhui province (0.405[0.357–0.459]), which is 10.38 times higher than the value of GBD 2013(0.039[0.024–0.060]). In addition, the DWs of Anhui province in the headache of tension type, mild amphetamine dependence, and symptomatic internal nematode infections were more than 4 times that of GBD2013. That for bulimia nervosa, mild episode of major depressive disorder, moderate motor plus cognitive impairments, and attention deficit hyperactivity disorder were only 1/3 of GBD2013 DW (SF5).
In terms of ranking order, Anhui's ranking results were very similar to China, but far from GBD 2013 and Japan. The difference between their DWs still visible to the naked eye. Among the health states of the top 15 and the last 15 DWs in Anhui province, about 43% or 57% of them appeared in GBD 2013 and Japan respectively.The DW ranked first in GBD2013 or Japan was different from Anhui province and China. The DWs ranked last 2 in theses studies were the same (Table 3). 46.67% of the top 15 DWs in Japan belonged to the category of mental, behavioral, and substance use disorders, which was similar to the results in Anhui and China. Most of the top 15 DWs in GBD 2013 belonged to the categories of infectious disease and cancer.
Based on the sample data obtained from the voluntary participants of the general population in Anhui Province, this study analyzed the DWs of the general population in Anhui province and compared it with the results of other provinces in China, the whole country, GBD and Japan. In addition to the same conclusions of other studies which the DWs were highly correlated with GBD, we also had new findings.
The variation range of DWs for health states in Anhui was very close to that in Japan . According to the numbers of health states with large differences in DWs and the ranking of DWs, Anhui’s results were still less different from China and other provinces . The DWs for mild impairment distance vision and mild anemia in other provinces except Henan were more than 3 times. Considering the very similar composition in gender, age and education level of participants and cultural backgrounds, is there really such a big difference? After further comparison with Anhui’s DWs in the Chinese study, surprisingly, the DWs of those 2 health states were still 3 times, although the Chinese study included only the results of 6,258 participants in Anhui province . So can it be that different anchoring methods lead to different results? The Chinese study was anchored by the results of Chinese population health equivalence (PHE), not the results of GBD which both this study and the Japanese study adopted. We suspected that a higher DW was obtained by using PHE data to anchor. Consistent with the research results of four European countries and Japan, we also considered that the method of anchoring with PHE may not be applicable to the general population survey based on the network.Unfortunately, we cannot directly provide sufficient evidence to prove this, because we failed to obtain provincial PHE data.
Different from the results of domestic provinces and China, the health states numbers overlapped significantly less in the top 15 and the last 15 DWs in GBD and Japan and the ranking order was also quite different. The health state ranked first DW in Anhui, China and domestic provinces was severe heroin and other opioid dependence which ranked 3rd in GBD 2013 . The DWs of health states were generally high if the name of the health state contained the word “dependence” in Anhui, China and domestic provinces. The DW of mild cannabis dependence, mild amphetamine dependence, mild cocaine dependence of Anhui province was 10.38 times, 4.41 times and 2.81 times higher than that of GBD 2013  respectively. Different regions had different policies and cultural backgrounds, and people may have different perceptions for the same problem [12, 13, 16,17,18,19,20,21,22]. Take cannabis dependence, cannabis use was more common in high-income countries in recent years [23,24,25]. In Anhui, even in China, it was not widespread or even banned unless using for medical purposes. Take another example, both primary and secondary infertility in Anhui and China were 2 or 3 times than that in GBD2013. There is an old saying in China that there are three ways to be unfilial, the worst is not to produce off-spring. In the case of "hearing loss", the DWs were 0.019 for mild, 0.067 for moderate, 0.288 for severe, 0.245 for profound, and 0.166 for complete hearing loss. After reaching a certain degree, DW did not increased with the increase of severity, similar results were found only in Chinese study , this may be related to the concept, just as the meaning of "too much is not enough" in Chinese idiom.
WHO pointed out that the focus of the survey in developing countries was different from the developed countries . The health states involved in each study were not completely unified, the number of injury related health states was limited in this study. This may also be the reason for the inconsistency of DW ranking and the first health state in each study. Other differences may be attributed to different ethnicity, health states at the time of the survey or self-perception [21, 22, 27].
Interestingly, most of the health states with top 15 DWs in Asian countries or regions belonged to the category of mental, behavioral, and substance use disorders which was significantly different from GBD 2013. Compared with Anhui’s DWs, the numbers of health states showed greater differences and accounted for about 10% in China and domestic provinces, 19.88% in Japan and 21.51% in GBD 2103 respectively. The differences of DWs in neighboring provinces were smaller than other geographically distant provinces or countries. Is the geographical distance also related to the difference of DW? This deserves further study.This finding further confirmed that the differences among similar-cultural regions were smaller than cross-cultural regions.
There was no gold standard for measurement methods. The visual analogue scale, time trade-off, person trade-off, standard gamble, interpolation, rank-based valuation techniques, PC and mapping function methods [1, 6, 9, 13, 15, 17, 28,29,30] could be used to measure health states. Each of them has different properties which may be affect the preferences that are measured . The same health states in the same country had different values due to different methods [6, 13, 15].Considering that the gold standards were always absent in the measurement method,the contents of health states differed, the estimation criteria of comorbidity were loose, So, how can different valuation methods be equivalent? The main topic of the study on burden of disease is to obtain comprehensive, complete, consistent and comparable data. DW research was meant essentially to achieve this goal, but because of these problems, the final results may not be comparable. DW research methods have evolved over time, more and more DW research used PC techniques after 2011 . Because it integrates the advantages of the distribution of the DWs of the health states, the pearson correlation coefficient, simplicity of the analysis, and basic generic health state layer descriptions, PC techniques has been proved to be utility, fairly straight-forward and simple method [2, 3, 13,14,15,16, 28]. However, generic descriptions should be used in conjunction with disease-specific descriptions to enhance standardized systematic descriptions of health states . In addition, more evidence is needed to demonstrate the impact of anchoring methods on the results. Like the results of GBD2010  and GBD2013 , this study found that PC responses were largely consistent across very distinct settings, its’ robustness was further verified, but the exceptions do need to be faced squarely. More DWs data in different settings need to be collected, pooled use for the global DW set in the GBD study in future to obtain comparable DWs or burden of disease estimates for all countries or all regions.
There were differences in DWs in different countries or regions. The differences within the same countries and regions were smaller than those between different ones. And the differences of DWs in neighboring provinces were smaller than other geographically distant provinces or countries. Global or national DWs may not represent provincial levels, especially in the absence of gold standards. And there is an urgent need for relevant gold standards,which can at least eliminate the differences of the DWs caused by different measurement methods or research contents, even if there is no better way to solve the differences caused by cross-cultural backgrounds for the time being.
Strengths and limitations
The first strength is that this study has enough samples at provincial level. Secondly, the present study obtained high-quality PC data. Thirdly, through the detailed comparison with other studies, we got some meaningful results, which will provide a new basis for the measurement of DWs in the future. However, this study still had limitations such as poor-quality PHE data and too little health states related to injury. This study cannot avoid a common problem of all online surveys. Most of the participants were higher educated groups, which may affect the provincial representation. And this study had no municipal representative data and comorbidity was not considered. These deficiencies need further research and improvement.
Availability of data and materials
The data and materials that support the findings of this study are unavailable for public access because informed consent to share said data (beyond the research team) was not obtained from study participants, but are available from the corresponding author on reasonable request.
- DWs :
- GBD :
Global burden of disease
- DALYs :
Disability-adjusted life years
- PC :
- UI :
- PHE :
Population health equivalence
Üstün TB, Rehm J, Chatterji S, et al. Multiple-informant ranking of the disabling effects of different health states in 14 countries. LANCET. 1999;354:111–5.
Salomon JA, Vos T, Hogan DR, et al. Common values in assessing health outcomes from disease and injury: disability weights measurement study for the Global Burden of Disease Study 2010. Lancet. 2012;380:2129–43.
Haagsma JA, Noordhout CMD, Polinder S, et al. Assessing disability weights based on the responses of 30,660 people from four European countries. Popul Health Metrics. 2015;13:10. https://doi.org/10.1186/s12963-015-0042-4.
Voigt K, King NB. Out of Alignment? Limitations of the Global Burden of Disease in Assessing the Allocation of Global Health Aid. Public Health Ethics. 2017;10:244–56.
Voigt K, King NB. Disability weights in the global burden of disease 2010 study: two steps forward, one step back? Bull World Health Organ. 2014;92:226–8. https://doi.org/10.2471/BLT.13.126227.
Haagsma JA, Polinder S, Cassini A, Colzani E, Havelaar AH. Review of disability weight studies: comparison of methodological choices and values. Population Health Metrics 2014; 12:20 http://www.pophealthmetrics.com/content/12/1/20.
Burstein R, Fleming T, Haagsma J, Salomon JA, Vos T, Murray C. Estimating distributions of health state severity for the global burden of disease study. Popul Health Metrics. 2015;13:31. https://doi.org/10.1186/s12963-015-0064-y.
Essink-Bot ML, Pereira J, Packer C, et al. Cross-national comparability of burden of disease estimates: the European Disability Weights Project. Bull World Health Organ. 2002;80(8):644–52.
de Noordhout CM, Devleesschauwer B, Salomon JA, et al. Disability weights for infectious diseases in four European countries: comparison between countries and across respondent characteristics. Eur J Public Health. 2017;28(1):124–33.
Chatterjeea S, Gosselin RA. Estimating the effectiveness of a hospital’s interventions in India: impact of the choice of disability weights. Bull World Health Organ. 2015;93:476–82. https://doi.org/10.2471/BLT.14.147900.
Moon L, Gourley M, Goss J, et al. History and development of national burden of disease assessment in Australia. Arch Public Health. 2020;78:88. https://doi.org/10.1186/s13690-020-00467-2.
Salomon JA, Haagsma JA, Davis A, et al. Disability weights for the Global Burden of Disease 2013 study. Lancet Glob Health. 2015;3:e712–23.
Ock M, Ahn J, Yoon SJ, Jo MW. Estimation of Disability Weights in the General Population of South Korea Using a Paired Comparison. PLoS ONE. 2016;11(9): e0162478. https://doi.org/10.1371/journal.pone.0162478.
Ock M, Lee JY, Oh IH, Park H, Yoon SJ, Jo MW. Disability Weights Measurement for 228 Causes of Disease in the Korean Burden of Disease Study 2012. J Korean Med Sci. 2016;31:129–38. https://doi.org/10.3346/jkms.2016.31.S2.S129.
Kim YE, Jo MW, Park H, et al. Updating Disability Weights for Measurement of Healthy Life Expectancy and Disability-adjusted Life Year in Korea. J Korean Med Sci. 2020;35(27): e219. https://doi.org/10.3346/jkms.2020.35.e219.
Nomura S, Yamamoto Y, Yoneoka D, et al. How do Japanese rate the severity of different diseases and injuries? —an assessment of disability weights for 231 health states by 37,318 Japanese respondents. Popul Health Metrics. 2021;19:21. https://doi.org/10.1186/s12963-021-00253-4.
Liu XX, Wang F, Yu CH, et al. Eliciting national and subnational sets of disability weights in mainland China: findings from the Chinese disability weight measurement study. Lancet Reg Health West Pac. 2022;26:100520. https://doi.org/10.1016/j.lanwpc.2022.100520.
Manchaiah V, Danermark B, Ahmadi T, et al. Social representation of “hearing loss”: crosscultural exploratory study in India, Iran, Portugal, and the UK. Clin Interv Aging. 2015;10:1857–72. https://doi.org/10.2147/CIA.S91076.
Gerlinger C, Bamber L, Leverkus F, et al. Comparing the EQ-5D-5L utility index based on value sets of diferent countries: impact on the interpretation of clinical study results. BMC Res Notes. 2019;12:18. https://doi.org/10.1186/s13104-019-4067-9.
Germundsson P, Manchaiah V, Ratinaud P, Tympas A, Danermark B. Patterns in the social representation of “hearing loss” across countries: how do demographic factors influence this representation? Int J Audiol. 2019;57(12):931–8. https://doi.org/10.1080/14992027.2018.15168.
Komiyama O, Wang K, Svensson P, et al. Ethnic differences regarding sensory, pain, and reflex responses in the trigeminal region. Clin Neurophysiol. 2009;120(2):384–9.
Reynolds Losin EA, Woo CW, Medina NA, Andrews-Hanna J, Eisenbarth H, Wager T. Neural and Sociocultural Mediators of Ethnic Differences in Pain. Nat Hum Behav. 2020;4(5):517–30. https://doi.org/10.1038/s41562-020-0819-8.
Palamar JJ, Ompad DC, Petkova E. Correlates of Intentions to Use Cannabis among US High School Seniors in the Case of Cannabis Legalization. Int J Drug Policy. 2014;25(3):424–35. https://doi.org/10.1016/j.drugpo.2014.01.017.
Hall W, Lynskey M. Assessing the public health impacts of legalizing recreational cannabis use: the US experience. World Psychiatry. 2020;19:179–86.
Roberts BA. Legalized Cannabis in Colorado Emergency Departments: A Cautionary Review of Negative Health and Safety Effects. West J Emerg Med. 2019;20(4):557–72.
World Health Organization, THE World BANK. World report on disability 2011. World Health Organization, 2011, 377(9782):1977.
Barry DT, Glennc CP, Hoff RA, Potenza MN. Group differences in pain interference, psychiatric disorders, and general medical conditions among Hispanics and whites in the U.S. general population. Psychiatry Res 2017;258:337–343. https://doi.org/10.1016/j.psychres2017.08.049.
Rehm J, Frick U. Establishing disability weights from pairwise comparisons for a US burden of disease study. Int J Methods Psychiatr Res. 2013;22(2):144–54. https://doi.org/10.1002/mpr.1383.
Craig BM, Busschbach JJV, Salomon JA. Modeling Ranking, Time Trade-Off and Visual Analogue Scale Values for EQ-5D Health States: A Review and Comparison of Methods. Med Care. 2009;47(6):634–41. https://doi.org/10.1097/MLR.0b013e31819432ba.
Lokkerbol J, Wijnen BFM, Chatterji S, Kessler RC, Chisholm D. Mapping of the World Health Organization's Disability Assessment Schedule 2.0 to disability weights using the Multi‐Country Survey Study on Health and Responsiveness. Int J Methods Psychiatr Res. 2021;30: e1886. https://doi.org/10.1002/mpr.1886.
Charalampous P, Polinder S, Wothge J, Lippe EVD, Haagsma JA. A systematic literature review of disability weights measurement studies: evolution of methodological choices.Population Health Metrics. 2021; 19–21. https://doi.org/10.1186/s12963-021-00253-4.
The authors thank all participants, coordinators and managers for their support and help during this study, especially CL, YZ, FS, YW, YZ, JW, GJ, MC, HZ, XT, CY, FJ, XH, FH, MG, HC, RL, ZS, SZ. The author would also like to thank Dr. PY for his critical reading of the manuscript.
Funding for this study was supported by the central government's subsidy fund for mental health and chronic non communicable disease prevention and control projects.
Ethics approval and consent to participate
The questionnaire design and data collection of this study followed International guide to ethics and human body research and international ethical guide to human body research.This study was approved by the Ethics Committee of Medical Department of Wuhan University (2019YF2055), and all participants signed written informed consent before answering the questionnaire.
Consent for publication
The authors have no competing interests to report on this work.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
He, Q., Zhou, M., Yin, P. et al. Can global or national disability weights represent provincial level?. BMC Public Health 23, 461 (2023). https://doi.org/10.1186/s12889-022-14347-7
- Disability weight
- Health state
- Paired comparison
- Probit regression model
- Cross-cultural comparisons