- Research article
- Open Access
- Open Peer Review
This article has Open Peer Review reports available.
Household costs of illness during different phases of tuberculosis treatment in Central Asia: a patient survey in Tajikistan
© Ayé et al; licensee BioMed Central Ltd. 2010
Received: 15 March 2009
Accepted: 18 January 2010
Published: 18 January 2010
Illness-related costs incurred by patients constitute a severe economic burden for households especially in low-income countries. High household costs of illness lead to impoverishment; they impair affordability and equitable access to health care and consequently hamper tuberculosis (TB) control. So far, no study has investigated patient costs of TB in the former Soviet Union.
All adult new pulmonary TB cases enrolled into the DOTS program in 12 study districts during the study period were enrolled. Medical and non-medical expenditure as well as loss of income were quantified in two interviews covering separate time periods. Costs of different items were summed up to calculate total costs. For missing values, multiple imputation was applied.
A cohort of 204 patients under DOTS, 114 men and 90 women, participated in the questionnaire survey. Total illness costs of a TB episode averaged $1053 (c. $4900 purchasing power parity, PPP), of which $292, $338 and $422 were encountered before the start of treatment, during intensive phase and in continuation phase, respectively. Costs per month were highest before the start of treatment ($145) and during intensive phase ($153) and lower during continuation phase ($95). These differences were highly significant (paired t-test, p < 0.0005 for both comparisons).
The illness-related costs of an episode of TB exceed the per capita GDP of $1600 PPP about two-and-a-half times. Hence, these costs are catastrophic for concerned households and suggest a high risk for impoverishment. Costs are not equally spread over time, but peak in early stages of treatment, exacerbating the problem of affordability. Mitigation strategies are needed in order to control TB in Tajikistan and may include social support to the patients as well as changes in the management of TB cases. These mitigation strategies should be timed early in treatment when the cost burden is highest.
Illness often leads to substantial economic burden for the patients and their households, especially in resource-poor settings and in the absence of social support mechanisms [1, 2]. Economic costs of illness at the household level-below referred to as household costs-have strong implications for issues of poverty and equitable access to health care. A review found that ill-health, especially long-lasting disease including tuberculosis (TB), puts households at high risk of impoverishment . Poor households are deterred from the use of health care due to household costs, preventing the health services from reaching those most in need . In the case of TB, where the main control and prevention measure consists of treating active disease , it must therefore be expected, that high household costs have a negative impact on TB control.
There have been many studies on cost-effectiveness of different TB control strategies, mainly investigating the costs to the health system. Important findings of these studies include the feasibility and cost-effectiveness of short-course TB chemotherapy on a predominantly ambulatory basis [5–10]. Far fewer studies measured the costs to households [11–13]. One of the first studies to comprehensively measure household costs found that the costs of an episode of TB in Thailand amounted to 20% of annual household income in the poorest third of patients and thus were devastating . Other studies also found that household costs of an episode of TB were considerable-ranging from $186 to $1457 in Tanzania depending on treatment duration and around $920 in China [13, 14]. A number of studies investigated reasons for delay to treatment and consequently measured only the costs up to diagnosis and registration with the TB control program [15–17]. Also when studying the economic impact on households, this strategy may be tempting, based on the rationale that TB treatment should be free of charge afterwards. However, those studies that measured costs after enrolment in TB control programs invariably found considerable costs during treatment [1, 12–14]. In his review, Russell  found that "health expenditures tended to be lumpy, coming in peaks that intensified cost burdens over a few days or weeks". However, to our knowledge so far no study looked at the distribution of household costs of TB over time other than differentiating before diagnosis and after diagnosis.
Tajikistan is the poorest republic of the former Soviet Union with a per capita gross domestic product (GDP) of $1600 purchasing power parity (PPP, 2007 estimate) . The country's infrastructure was severely damaged during the civil war from 1992 to 1998. Many villages and even whole districts are cut off in winter from electricity supply and transportation axes in this mountainous country. The economy of Tajikistan is highly informal and it has been observed that the grey economy is more important in several respects than the official economy . Corruption is widespread and Transparency International ranks Tajikistan number 151 out of 180 countries in their perceived corruption index . Temporary labour migration to Russia is very common, especially among young men. The health system is still very specialised, hierarchical and at least nominally comprehensive as in the Soviet Union . Profound health sector and health financing reform is ongoing. Patients have to pay predefined fees for specific services above the primary care level, with different fee rates depending on referral or non-referral of the patient from primary care level. Informal payments are widespread also in the health system and out-of-pocket payments may constitute as much as 80% of total health funding . In a recent survey in two districts piloting a new primary care model, almost half of the patients who had visited a family doctor reported that they had made an informal payment . Involvement of the private sector in health care is very limited and virtually nonexistent outside the capital. The WHO estimate of TB incidence was 204 cases per 100,000 per year in 2006 , while the national TB control program estimated incidence between 160 and 180 cases per 100,000 per year. The country began DOTS implementation in 2002 and DOTS coverage reached 100% by the end of 2007. A study in ten districts of Tajikistan showed that the hospitalisation rate of TB patients was high (58%) and that a positive sputum smear result was the main predictor of hospitalisation . For patients who are not hospitalised, the program foresees facility-based observation of treatment.
In most settings in the former Soviet Union anti-TB drugs are provided for free, but patients have to pay for other services including additional medicines, x-rays and laboratory services and potentially face major financial barriers . In urban Ukraine, total illness-related costs of TB varied from $57 to $450 depending on the TB control strategy and on the locality . These costs were often outweighed by the social benefits provided to TB patients. Overall, very little is known about illness-related costs of TB in the former Soviet Union. This study aimed at quantifying household costs of an episode of TB in Tajikistan, with special attention to the time period, when costs were incurred.
Only districts with an established DOTS program were considered for inclusion in the study. Two regions were excluded because they were accessible only by air at the time of the study, Badakhshan and Sughd. In close collaboration with the Republican Centre for TB control of Tajikistan, 12 districts were purposefully selected to represent different urban, rural, lowland and mountainous settings. All pulmonary TB patients at least 15 years of age who were registered for the first time (i.e. "new" patients according to WHO definition) in the 12 study districts in the period from 1st December 2006 until 31st March 2007 were eligible for study participation. Patients who had defaulted from treatment before the first interview were excluded. Patients who defaulted during or at the end of the intensive phase were excluded from the second interview, as they could not possibly give information on the continuation phase.
Questionnaire and Interviews
Patients were visited in hospital or at home during the intensive phase of treatment, written informed consent was obtained and the first interview conducted. Three to four months later, the same patients were visited for a follow-up interview. The first interview covered the period before diagnosis, starting from the time point, when the patients experienced the first symptoms, and the intensive phase. The follow-up interview covered the continuation phase. The costs from the day of interview until the end of the respective phase were extrapolated, assuming a duration of four months for the continuation phase. The questionnaires were adapted from a questionnaire used for the quantification of costs of HIV/AIDS . They were piloted and finalised prior to the start of data collection. They are available from the authors upon request. Completing an interview took about one hour on average. The questionnaires included detailed questions about health care seeking, about the costs incurred by patients and their households and about household assets. Household costs included medical expenditure (for drugs, medical fees and laboratory tests), non-medical expenditure (for transport, special food items, modification of housing, traditional healers, self-treatment and any other reported expenditures), and loss of income due to the inability to work. To quantify loss of income, patients were asked about the actual reduction of income that they themselves and caretakers had experienced due to absence from their usual income-generating activities for a day. This daily income reduction was then multiplied with the number of days away from work. If patients did not know the income reduction for one day, but could give an estimate for the total income reduction until the day of interview, the total estimate was registered and divided by the number of days away from work during data entry. Where income was in-kind, like sometimes in agriculture, the monetary value was estimated. Information on treatment outcome could not be obtained, as treatment was not completed at the time of the interview.
Interviewers were trained for two weeks before conducting real interviews and supervised at least twice per month through interviews conducted together with the main researcher. Specific emphasis was given to support the patients in their recall work, including the use of a calendar of locally important events, to motivate the patient to provide accurate information on costs and to ask back if reported costs were unexpectedly high or low for local context. Data were entered by the main researcher in FileMaker (version 8.0v1, FileMaker Inc, USA, 1984-2005).
All analyses were conducted in Stata IC/10.1 for Macintosh (Stata Corporation, USA, 1985-2008). Individual cost items were summed up to the categories of medical costs, non-medical costs, and lost income-for each of the three phases of treatment separately (before start of treatment, intensive phase, and continuation phase). For conversion of Tajik Somonis to international US$ PPP, we used data provided by the World Bank .
In order to avoid the problems associated with complete case analysis and as recommended by Manca and Palmer  for similar problems, we used multiple imputation to deal with missing observations, implemented in the function "ice" in Stata [31, 32]. The algorithm in "ice" assumes normally distributed variables; this was assured through transformation to normal scores. The statistical comparisons of costs during the different phases were done with a paired t-test because we were mainly interested in the pair wise comparisons (n = 204 for all comparisons). To correct for the fact that we conducted three t-tests, we decreased our significance level to 1.67%. We plotted the residuals and checked visually whether the distributions were normal. In order to normalise the distribution of residuals, we used log-transformation.
We conducted a principal component analysis on 18 variables to construct a wealth index following the methodology described by Filmer and Pritchett . The 18 variables included 15 household assets and three housing characteristics. To improve validity of the wealth measure, we used continuous variables, where this was possible, and we did not recode any categorical variables as dummies . Subjects were allocated to five quintiles by their wealth index. To investigate the internal consistency of our measure of socio-economic status (SES), we plotted the mean of each asset variable against SES quintile and found that the asset variables were highly consistent with expectations.
The study has received ethical approval from the Ministry of Health of the Republic of Tajikistan.
Characteristics of study participants
Form of TB
Sputum smear positive
History of TB disease
Delay until start of treatment
Median patient delay
Median health system delay
Median total delay
Median duration of hospitalisation
Direct observation of treatment
Median number of visits for treatment observation
The mean total household costs of a TB episode amounted to $1053, corresponding to $4894 PPP. While a considerable part of the scientific discussion on household costs has concentrated on the period before diagnosis [15–17], this study found that costs during treatment account for three quarters of total costs-in a setting with free TB drugs. Monthly costs were most acute before treatment and during the intensive phase. The total costs presented here do not yet take into account that patients may incur costs after the end of treatment; especially further loss of income for those who have lost their jobs.
The study may seem only partly representative of Tajikistan, as two regions had to be excluded. One of the regions (Badakhshan) is remote and very sparsely populated. The other one (Sughd) had only eight districts with an established DOTS program at the time of the study (versus 34 districts with DOTS in the whole country) and was only accessible by air in winter. The study is representative of an area of Tajikistan, from where 80% of all cases under DOTS came in the year up to the study.
As other studies on household costs, this study also relied on self-reported costs-it is thus not excluded that patients forgot or overestimated some of the costs . By using two interviews for separate periods of treatment, this study limited recall time and recall bias. Further we tried to limit the potential problem of overestimation by training the interviewers for this situation.
Composition of household costs
Mean in US $
Percentage of direct costs
Costs of diagnostics and lab tests
Costs of medical fees
Self-treatment & traditional medicine
Variation of selected costs items over time [in US $]
Before start of treatment
Costs of diagnostic and lab tests
Costs of medical fees
Household costs were approximately three times higher than the per capita GDP and thus must be considered catastrophic for affected households. As TB disproportionately affects poorer segments of society this burden may be even more catastrophic in the typical TB-affected household than average GDP values indicate. Direct costs were relatively similar for all SES quintiles, while indirect costs, which depend on income, were three times higher in the wealthiest compared to the poorest quintile. This is suggestive of a regressive cost burden at least for direct costs.
Monthly costs were especially high before enrolment in DOTS treatment and during the intensive phase. Monthly costs were significantly higher during these two periods than during the continuation phase. This pattern was very clear for direct and total costs, but less so for lost income, which was more constant over the different phases of treatment. Patients who stopped working often did so for the duration of four months and more. This suggests that they did not only stop working during the time of acute illness, but for the whole treatment. Tajik doctors often recommend them not to work during treatment and for up to two years afterwards. Seasonality of income may have led to underestimation of lost income especially before the start of treatment, because patients were enrolled into the study in the winter months, which is the season when least work is available. Consequently, the observed pattern of higher costs during early stages of treatment may even be stronger in reality. The spread of costs over time affects the ability of households to cope with these costs , and our findings suggest that the problem of affordability of services is most acute in early stages of treatment.
The high costs found in this study suggest that sheer poverty and inability to cope with the economic burden of treatment negatively affect treatment adherence and treatment outcomes, as has been suspected in other contexts [12, 36]. Farmer et al  have shown that mitigation strategies including incentives for adherence can improve treatment adherence among TB patients. Donors and program managers need to implement effective mitigation strategies to enable patients to adhere to DOTS and to avoid severe impoverishment. The present study shows that these mitigation strategies must be timed early in treatment, when most of the costs are incurred. A recent study from Russia has identified types of incentives that are preferred by TB patients . Additional strategies to reduce the economic burden of TB disease on households might target the high costs for drugs, which are related to the prescription patterns including common use of symptomatic treatment. Studies from Russia have shown features in the management of TB that are also found in Tajikistan, including high hospitalisation rates and widespread use of symptomatic treatment [39, 40]. Furthermore similar factors influence hospitalisation in Russia and Tajikistan [25, 41]. These parallels suggest that some of our results may also apply to Russia and other post-Soviet countries.
During an episode of TB, patients and their households face costs of approximately $4900 PPP. Comparison with the per capita GDP of $1600 PPP shows that these costs are catastrophic for the households concerned and suggest a high risk for impoverishment. Costs are not equally spread over time, but are highest in early stages of treatment, exacerbating the problem of affordability. Free TB drugs do not lead to free treatment and non-TB drugs, transport, and loss of income are very important cost drivers for patients. Mitigation strategies are needed in order to control TB in Tajikistan and may include social support to the patients as well as changes in the management of TB cases. Mitigation strategies need to be timed early in treatment when the cost burden is highest. The provision of food supplements to TB patients may contribute to somewhat alleviate the economic burden. However, our results suggest that these should be delivered soon after a TB diagnosis. Reducing the use of non-TB drugs should be considered.
The authors thank all patients who participated and the health care staff who supported the team during the survey. The participation of Firuza Qurbonova and Zulfira Mengliboyeva in the data collection is gratefully acknowledged. Sandra Alba helped with the statistical analyses. Kate Molesworth, Don de Savigny, Josh Yukich and Manfred Stoermer commented on an earlier version of the manuscript.
- Russell S: The economic burden of illness for households in developing countries: a review of studies focusing on malaria, tuberculosis, and human immunodeficiency virus/acquired immunodeficiency syndrome. Am J Trop Med Hyg. 2004, 71 (Suppl 2): 147-155.PubMedGoogle Scholar
- McIntyre D, Thiede M, Dahlgren G, Whitehead M: What are the economic consequences for households of illness and of paying for health care in low- and middle-income country contexts?. Soc Sci Med. 2006, 62: 858-865. 10.1016/j.socscimed.2005.07.001.View ArticlePubMedGoogle Scholar
- Sauerborn R, Adams A, Hien M: Household strategies to cope with the economic costs of illness. Soc Sci Med. 1996, 43: 291-301. 10.1016/0277-9536(95)00375-4.View ArticlePubMedGoogle Scholar
- World Health Organisation: Treatment of tuberculosis-guidelines for national programmes. 2003, World Health Organisation, GenevaGoogle Scholar
- Murray C, DeJonghe E, Chum H, Nyangulu D, Salomao A, Styblo K: Cost effectiveness of chemotherapy for pulmonary tuberculosis in three sub-Saharan African countries. The Lancet. 1991, 338 (8778): 1305-1308. 10.1016/0140-6736(91)92600-7.View ArticleGoogle Scholar
- Fryatt R: Review of published cost-effectiveness studies on tuberculosis treatment programmes. Int J Tuberc Lung Dis. 1997, 1: 101-109.PubMedGoogle Scholar
- Saunderson P: An economic evaluation of alternative programme designs for tuberculosis control in rural Uganda. Soc Sci Med. 1995, 40: 1203-1212. 10.1016/0277-9536(94)00240-T.View ArticlePubMedGoogle Scholar
- Floyd K, Skeva J, Nyirenda T, Gausi F, Salaniponi F: Cost and cost-effectiveness of increased community and primary care facility involvement in tuberculosis care in Lilongwe District, Malawi. Int J Tuberc Lung Dis. 2003, 7 (Suppl): 29-37.Google Scholar
- Burman W, Dalton C, Cohn D, Butler J, Reves R: A cost-effectiveness analysis of directly observed therapy vs selfadministered therapy for treatment of tuberculosis. Chest. 1997, 112: 63-70. 10.1378/chest.112.1.63.View ArticlePubMedGoogle Scholar
- The World Bank: World Development Report 1993. Investing in Health. 1993, Oxford: Oxford University PressView ArticleGoogle Scholar
- Floyd K: Costs and effectiveness--the impact of economic studies on TB control. Tuberculosis. 2003, 83: 187-200. 10.1016/S1472-9792(02)00077-X.View ArticlePubMedGoogle Scholar
- Kamolratanakul P, Sawert H, Kongsin S, Lertmaharit S, Sriwongsa J, Na-Songkhla S, Wangmanee S, Jittimanee S, Payanandana V: Economic impact of tuberculosis at the household level. Int J Tuberc Lung Dis. 1999, 3: 596-602.PubMedGoogle Scholar
- Wyss K, Kilima P, Lorenz N: Costs of tuberculosis for households and health care providers in Dar es Salaam, Tanzania. Trop Med Int Health. 2001, 6: 60-68. 10.1046/j.1365-3156.2001.00677.x.View ArticlePubMedGoogle Scholar
- Jackson S, Sleigh A, Wang G-J, Liu X-L: Poverty and the economic effects of TB in rural China. Int J Tuberc Lung Dis. 2006, 10: 1104-1110.PubMedGoogle Scholar
- Lambert M, Delgado R, Michaux G, Volz A, Speybroeck N, Stuyft Van Der P: Delays to treatment and out-of-pocket medical expenditure for tuberculosis patients, in an urban area of South America. Ann Trop Med Parasitol. 2005, 99: 781-787. 10.1179/136485905X65152.View ArticlePubMedGoogle Scholar
- Needham D, Godfrey-Faussett P, Foster S: Barriers to tuberculosis control in urban Zambia: the economic impact and burden on patients prior to diagnosis. Int J Tuberc Lung Dis. 1998, 2: 811-817.PubMedGoogle Scholar
- Kemp J, Mann G, Simwaka B, Salaniponi F, Squire S: Can Malawi's poor afford free tuberculosis services? Patient and household costs associated with a tuberculosis diagnosis in Lilongwe. Bull World Health Org. 2007, 85: 580-585. 10.2471/BLT.06.033167.View ArticlePubMedPubMed CentralGoogle Scholar
- Central Intelligence Agency: The World Factbook. [https://www.cia.gov/library/publications/the-world-factbook/geos/ti.html]
- The World Bank: Republic of Tajikistan Poverty Assessment Update. 2005, Report No: 30853-TJ, DushanbeGoogle Scholar
- Transparency International: Perceived Corruption Index. [http://www.transparency.org/policy_research/surveys_indices/cpi/2008]
- The World Bank: Republic of Tajikistan Health Sector Note. 2004, Report, DushanbeGoogle Scholar
- Falkingham J: Poverty, out-of-pocket payments and access to health care: evidence from Tajikistan. Soc Sci Med. 2004, 58: 247-258. 10.1016/S0277-9536(03)00008-X.View ArticlePubMedGoogle Scholar
- Tediosi F, Ayé R, Ibodova Sh, Thompson R, Wyss K: Access to medicines and out of pocket payments for primary care: evidence from family medicine users in rural Tajikistan. BMC Health Serv Res. 2008Google Scholar
- World Health Organisation: Global Tuberculosis Control: Surveillance, Planning, Financing. WHO Report. 2008Google Scholar
- Thierfelder C, Makowiecka K, Vinichenko T, Ayé R, Edwards P, Wyss K: Management of pulmonary tuberculosis in Tajikistan: which factors determine hospitalisation?. Trop Med Int Health. 2008, 13: 1364-1371. 10.1111/j.1365-3156.2008.02149.x.View ArticlePubMedGoogle Scholar
- Mosneaga A, Yurasova E, Zaleskis R, Jakubowiak W: Enabling health systems in tuberculosis control: challenges and opportunities for the former Soviet Union countries. Health Systems and the Challenge of Communicable Disease Experiences from Europe and Latin America-European Observatory on Health Systems and Policies Series. Edited by: Coker R, Atun R, McKee M. 2008, New York: Open University Press, 171-192.Google Scholar
- Vassall A, Chechulin Y, Raykhert I, Osalenko N, Svetlichnaya S, Kovalyova A, Werf van der M, Turchenko L, Hasker E, Miskinis K, Veen J, Zaleskis R: Reforming tuberculosis control in Ukraine: results of pilot projects and implications for the national scale-up of DOTS. Health Policy Plan. 2008, 24: 55-62. 10.1093/heapol/czn045.View ArticlePubMedGoogle Scholar
- Wyss K, Hutton G, N'Diekhor Y: Costs attributable to AIDS at household level in Chad. AIDS Care. 2004, 16: 808-816. 10.1080/09540120412331290167.View ArticlePubMedGoogle Scholar
- The World Bank: The 2005 International Comparisons Program Report. Washington. 2008Google Scholar
- Manca A, Palmer S: Handling Missing Data in Patient-Level Cost-Effectiveness Analysis alongside Randomised Clinical Trials. Appl Health Econ Health Policy. 2005, 4: 65-75. 10.2165/00148365-200504020-00001.View ArticlePubMedGoogle Scholar
- Schafer J: Analysis of Incomplete Multivariate Data. 1997, London: Chapman & HallView ArticleGoogle Scholar
- Royston P: Multiple Imputation of Missing Values: update. The Stata Journal. 2005, 5: 188-201.Google Scholar
- Filmer D, Pritchett L: Estimating wealth effects without expenditure data-or tears: an application to educational enrollments in states of India. Demography. 2001, 38: 115-132.PubMedGoogle Scholar
- Kolenikov S, Angeles G: The Use of Discrete Data in PCA: Theory, Simulations and Applications to Socioeconomic Indices. Technical Report, Carolina Population Center, . 2004, University of North Carolina, Chapel Hill, 59-[http://www.cpc.unc.edu/measure/publications/pdf/wp-04-85.pdf]Google Scholar
- Ayé R, Wyss K, Abdualimova H, Saidaliev S: Patient's site of first access to health system influences length of delay for tuberculosis treatment in Tajikistan. BMC Health Serv Res. 2010, 10: 10-10.1186/1472-6963-10-10.View ArticlePubMedPubMed CentralGoogle Scholar
- Greene J: An ethnography of nonadherence: culture, poverty, and tuberculosis in urban Bolivia. Cult Med Psychiatry. 2004, 28: 401-425. 10.1023/B:MEDI.0000046429.55801.c8.View ArticlePubMedGoogle Scholar
- Farmer P, Robin S, Luc Ramilus St, Kim Yong J: Tuberculosis, poverty and "compliance": lessons from rural Haiti. Sem Resp Inf. 1991, 6: 254-260.Google Scholar
- Jakubowiak W, Bogorodskaya E, Borisov S, Danilova I, Lomakina O, Kourbatova E: Social support and incentives programme for patients with tuberculosis: experience from the Russian Federation. Int J Tuberc Lung Dis. 2007, 11: 1210-1215.PubMedGoogle Scholar
- Coker R, Dimitrova B, Drobniewski F, Samyshkin Y, Balabanova Y, Kuznetsov S, Fedorin I, Melentsiev A, Marchenko G, Zakharova S, Atun R: Tuberculosis control in Samara Oblast, Russia: institutional and regulatory environment. Int J Tuberc Lung Dis. 2003, 7: 920-932.PubMedGoogle Scholar
- Jacobs B, Clowes C, Wares F, Polivakho V, Lyagoshina T, Peremitin G, Banatvala N: Cost-effectiveness analysis of the Russian treatment scheme for tuberculosis versus short-course chemotherapy: results from Tomsk, Siberia. Int J Tuberc Lung Dis. 2002, 6: 396-405.PubMedGoogle Scholar
- Atun R, Samyshkin Y, Drobniewski F, Kuznetsov S, Fedorin I, Coker R: Social factors influencing hospital utilisation by tuberculosis patients in the Russian Federation: analysis of routinely collected data. Int J Tuberc Lung Dis. 2005, 9: 1140-1146.PubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/10/18/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.