Estimating the magnitude and direction of bias in tuberculosis drug resistance surveys conducted only in the public sector: a simulation study
© Cohen et al; licensee BioMed Central Ltd. 2010
Received: 4 November 2009
Accepted: 21 June 2010
Published: 21 June 2010
Accurate assessment of the burden of drug-resistant TB requires systematic efforts to quantify its magnitude and trend. In approximately half the countries where resistance has been reported, estimates are based on surveys conducted in public sector facilities. However, in locations where a substantial fraction of TB cases seek care with private providers, these surveys may not accurately measure resistance in the entire population.
We describe a mathematical model to investigate biases associated with sampling only from public sector cases in India, where TB treatment is offered in both public and private sectors. We then propose and demonstrate a weighted estimator as an efficient method for including small numbers of cases from the private sector as a way to recover valid estimates of resistance in the population under study.
We find that public sector surveys rarely provide valid estimates of drug-resistance among new and retreatment cases. Further, the magnitude and direction of the bias are sensitive to many parameters describing the health-seeking behaviours and treatment outcomes of tuberculosis patients, disallowing simple adjustments to recover accurate estimates.
In locations where large numbers of tuberculosis patients are diagnosed and treated by private sector practitioners who are not typically included in drug resistance surveys, targeted surveys for assessing drug resistance are required to validly estimate resistance.
M. tuberculosis resistant to the most potent available antibiotics threatens the success of global tuberculosis (TB) control strategies that rely on standard combinations of these drugs . Although resistance to even a single first-line agent can reduce the probability of successful treatment outcome, the most worrisome disease is resistant to both isoniazid and rifampin (multidrug-resistant; MDR). At least 50% of MDR TB patients treated with standard first-line therapy will fail treatment and suffer from relapse, chronic disease, or die . Furthermore, the detection of extensively drug resistant tuberculosis (XDR),[4, 5] defined as MDR with additional resistance to at least a fluoroquinolone and a second-line injectable antibiotic, heightens concern that increasingly resistant forms of tuberculosis will undermine the effectiveness of the current arsenal of treatment .
Determining the burden of MDR TB requires systematic efforts to assess the magnitude and trend of drug-resistant disease. Since 1994, under the leadership of the World Health Organization and the International Union Against Tuberculosis and Lung Diseases (WHO-IUATLD), four Anti-Tuberculosis Resistance in the World reports have been published [1, 7]. These reports include data from countries and regions that conduct either continuous surveillance or periodic surveys for drug resistant disease. The WHO-IUATLD provides guidance for the design of such surveys; the principle requirements are that they 1) include a representative sample of TB cases from the area under study; 2) differentiate between TB cases that have and have not been previously exposed to anti-TB antibiotics; and 3) use standard methods for determining drug resistance and utilize approved reference laboratories for quality assurance .
In some settings, obtaining representative samples of specimens from new and retreatment patients is a challenge. While drug resistance surveys are usually coordinated in the public sector by National Tuberculosis Programs (NTPs), in areas where a substantial fraction of patients are treated by private practitioners, many patients are neither observed in the public sector nor notified to public authorities. Consequently, surveys conducted exclusively on public sector cases may not reflect the local burden of resistance.
Here, we describe a simple mathematical model to investigate potential biases associated with sampling only from public sector cases in India. We use India as our motivating example to examine such biases for two reasons. First, India has the highest burden of TB patients in the world and most patients initially seek care through private practitioners [9, 10]. Second, previous work in India provides information on differences in diagnostic and treatment practices in the public and private sector (each of which are likely to affect the acquisition of drug resistance) and behavioral data suggesting how TB patients move between the private and public sectors . These data are have not been measured and are not generally available for other countries with both a high burden of TB and a sizable private sector in which diagnosis and treatment of TB is common. We present general conditions under which surveys conducted in the public sector provide valid estimates of drug resistance among new, retreatment, and combined cases and suggest methods to test whether these conditions are met. Finally, when exclusion of private sector cases from surveys results in biased estimates of the local burden of resistance, we describe an efficient method for including data from private sector samples to produce valid estimates of resistance.
Private sector providers and drug-resistant tuberculosis in India
India has the largest estimated number of incident (almost 2 million new cases per year) and prevalent TB cases (almost 3.5 million existing cases), representing more than 20% of the total worldwide burden of TB . India has invested heavily in extending free access to high-quality, standardized approaches for diagnosis and directly observed treatment of TB through public sector facilities and since 2006 has offered such services in all districts of the country. Still, surveys indicate that between 50%-90% of Indian patients first seek care through a private practitioner; this preference for care delivered in the private sector exists among patients with major and minor illnesses, patients residing in rural and urban areas, and patients from rich and poor households [9, 10, 13, 14]. Patients may first seek care with for-profit providers despite the fees associated with such services because private sector care is viewed as more accessible and for-profit providers are perceived as more empathic and better able to dispense high quality drugs .
While systematic steps toward improving the quality of TB diagnosis and treatment in the private sector through the WHO Public-Private Mix DOTS model (PPM-DOTS) in India appear promising,[15, 16] most care provided by private practitioners outside of PPM-DOTS programs remains sub-optimal. Specifically, private providers outside of PPM-DOTS do not routinely use sputum smear microscopy to diagnose TB, are more likely to prescribe non-standard regimens of antibiotics, and do not directly observe treatment to ensure adherence [13, 17, 18]. TB patients treated by these private sector providers are thus less likely to complete treatment and, if they survive and are not lost from the health care system entirely, will eventually present for retreatment in either the public or the private sector . TB patients treated with inappropriate drugs or for whom adherence is interrupted also have a higher probability of acquiring drug resistance while on treatment.
Based on eight surveys conducted between 1995 and 2006 that included a total of 3562 patients, 2.8% of new TB patients were estimated to be infected with MDR strains [1, 7]. Only one survey has provided data necessary to estimate the proportion of incident retreatment cases that are drug-resistant; in this 2006 survey of 1047 retreatment cases, 17.4% of cases were infected with MDR strains. Because these drug-resistance surveys are conducted in public sector facilities, it is not clear whether these surveys can be used to infer the level of drug resistance among all TB patients in India. Furthermore, because the risks of successful treatment outcome and acquired drug resistance differ between public and private patients and because patients failing therapy in one sector may switch to the other for retreatment, understanding the relationship between sector-specific treatment outcomes and the patterns in which patients navigate the healthcare system are important for determining biases associated with drug resistance surveys conducted only in the public sector.
Model details and equations
Model input parameters and outputs
Notes on values; (allowable ranges)
TB incidence rate (new cases)
Fixed incidence (value irrelevant)
Fraction of new cases that go to the public sector
Fraction of retreatment cases that go to the public sector
Fraction of new cases that are sensitive
1-qC Where q is the relative transmissibility of resistant compared with sensitive strains (range of q from 0-1; assumes no resistant "superbugs") and C (range of C from 0-1) is the proportion of all incident cases with MDR
Fraction of new sensitive cases that go to the public sector
Fraction of new resistant cases that go to the public sector
f R , f S
f •N , f •P
Fraction failing treatment among drug resistant and sensitive
Fraction failing treatment in public and private sectors
f S ≤ f R Failure more likely for resistant
f •N ≤ f •P Failure as or more likely in private sector
a N ,a P
Fraction of TB cases in the public and private sectors who acquire resistance
(0-1) a N ≤ a P Acquired resistance as or more likely in private sector
Fraction that are lost to follow-up or die
Fraction in the public sector that return to the public sector for next treatment episode
Retreatment cases more likely to present in public sector
Fraction in the private sector that return to the private sector for next treatment episode
Retreatment cases more likely to present in public sector
Proportion of new cases with resistant TB
Proportion of new cases in public sector with resistant TB
Proportion of new cases in private sector with resistant TB
Proportion of retreatment cases with resistant TB
Proportion of retreatment cases in public sector with resistant TB
Proportion of retreatment cases in private sector with resistant TB
Proportion of all cases with resistant TB
Proportion of all cases in public sector with resistant TB
Proportion of all cases in private sector with resistant TB
We model a constant incidence of tuberculosis (Λ) and assume that the fraction of incident new cases that is drug-resistant (1-y t ) reflects the relative frequency of drug-resistant disease in the most recent treatment generation and the relative transmissibility of drug-resistant compared with drug-sensitive M. tuberculosis. New tuberculosis cases that are not cured and do not die or get lost to follow-up, appear in the next time step (t+1) as retreatment cases. Previously treated cases can be repeatedly re-treated in either the same health sector or switch to the other health sector. Consistent with data from India [9–11], we specify that the majority of cases initially present for treatment in the private sector (y t (1-2x s ) + (1-y t ) (1-2x R ) > 0), but then move preferentially to the public sector for retreatment (r p ≤1/2 < r N ).
For simplicity, we model only two disease phenotypes: pan-sensitive TB and MDR TB. We model the acquisition of the MDR phenotype as occurring during a single course of treatment (a fraction a N or a P acquire resistance). While this does not realistically reflect the complexity of sequential acquisition of drug resistance, we gain qualitative insight into the effect of treatment and health seeking behavior on the frequency of drug resistance. Additionally, we ignore the possibility that those in retreatment categories may have been re-infected. We consider only the effects of standard first-line drug regimens; as such, the proportion of drug-sensitive cases that are cured exceeds that of drug-resistant cases in both public and private sectors. In all cases, we assume that a higher proportion of both drug-sensitive and drug-resistant cases are successfully treated in the public sector than in the private sector.
The fraction of new cases that is resistant reflects both the relative number of existing infectious cases of drug-resistant and drug-sensitive TB and the relative transmissibility of drug-resistant compared with drug-sensitive M. tuberculosis. The fraction of retreatment cases that is resistant reflects the past incidence of drug-resistance among new cases, the risk of acquired resistance among individuals initially infected with drug-sensitive strains, and the relative effectiveness of treatment for those with drug-resistant compared to drug-sensitive disease. To evaluate conditions under which surveys conducted in the public sector are biased, we report percent bias comparing measures of the proportion of disease that is resistant among new and retreatment cases in the public sector to these same measures in the counterfactual scenario where cases from both the public and private sector are sampled.
Resistance among new cases
It immediately follows that A N is equal to A, and therefore an unbiased estimator of A N based on a random sample from the public sector will unbiasedly estimate A, if and only if k FN = 1 and, or A N = A p . First, all new cases go to the public sector (i.e. k FN = 1) if and only if all new sensitive and all new drug-resistant cases go to the public sector (i.e. x S = x R = 1). Second, if all new cases do not go to the public sector (i.e. x S ≠ 1 or x R ≠ 1), then A N must equal A p for A N to also equal A. But A N = A P requires that the fraction of new sensitive cases that go to the public sector is the same as the fraction of new resistant cases that go to the public sector (i.e. x S = x R ), otherwise, if x S > x R , then A N < A P which leads to A N < A; and conversely, if x S < x R , then A N > A P , which leads to A N < A.
Resistance among retreatment cases
It immediately follows that B N equals B, and an unbiased estimator of B N based on a sample from the public sector unbiasedly estimates B, if and only if k RN = 1 and, or B N = B P . Relating this to the parameters in the mathematical model, k RN = 1 requires all individuals to attend public facilities for retreatment regardless of where previously treated (i.e. r N = 1 - r P = 1). In general, if an equal fraction of individuals treated in the public sector and individuals treated in the private sector seek retreatment in the public sector (0 ≤ r N = 1 -r P ≤ 1), then it follows immediately that B N = B P .
Resistance among combined cases
Let C represent the prevalence of MDR TB in all cases. We deconstruct C in terms of prevalence of drug resistance in new and retreatment cases, C = k N A + (1 - k N )B, where k N is the proportion of all cases that are new. Presuming that we unbiasedly estimate A and B, these estimates must be weighted by the proportion of new and retreatment cases in order to unbiasedly estimate C.
Investigating the role of small targeted surveys
In the event that the conditions required for public sectors surveys to return unbiased estimates of resistance among new or retreatment cases are unlikely to be met, we recommend a weighted estimator to combine data from public and private sectors to unbiasedly estimate MDR TB prevalence in new and retreatment cases. We assume that the relative burden of cases for public and private sector are known. Presuming that the MDR TB surveys are routine activities in the public sector, the data from these surveys can be augmented with data from targeted surveys in the private sector to efficiently provide unbiased estimates of the MDR TB prevalence (Case 1) . In Case 1, n N , the number of samples collected in the public sector, is fixed by the routine activity, and n P , the number of samples collected from the private sector, is determined by either resources or desired level of accuracy. For example, suppose on a routine basis, 200 samples are tested in the public sector (for each type of case) to estimate the prevalence of MDR TB. To obtain an unbiased estimate of the fraction of all cases with resistance, we can sample an additional 50 individuals in the private sector. The choice of 50 additional samples is arbitrary and was chosen only for illustrative purposes. When surveys are not routine in the public sector, we propose an activity that divides the sample between the two sectors (Case 2). In Case 2, the total sample size, n, will be determined by resource availability and will then be divided between the public and private sector, so that n = n N + n P . For example, if resources only allow testing of 50 samples, we may divide the samples evenly between the public and private sectors.
Resistance among new cases
Resistance among retreatment cases
For any fixed value of the RR of failure, there is a greater fraction of resistance among retreatment cases in the private sector as the RR of acquired resistance increases within the private sector (trend toward darker red moving up the y-axis in Figure 3). Furthermore, this bias is amplified when individuals failing therapy within either sector remain within that sector for retreatment. Additional files provide results where we do not assume equal MDR TB prevalence in new cases (Additional File 1) and when risk of amplified resistance in the public sector exceeds that in the private sector (Additional File 2).
Resistance among combined cases
Use of small targeted surveys to eliminate bias
We consider two scenarios to illustrate the potential applications of this approach. First, in locations where there is ongoing surveillance for drug resistance in the public sector, we propose a simple extension to estimate prevalence of drug resistance in the private sector (Case 1). Second, in areas where public sectors surveys are not routinely conducted, we propose an efficient sampling strategy that returns unbiased estimates of resistance for a fixed sample size (Case 2).
A similar estimator and variance can be constructed for retreatment cases. The variances of the estimators differ between the two scenarios based on the sample sizes determination.
Accurate estimates of the burden of MDR tuberculosis are required to mobilize appropriate resources to combat this threat to global tuberculosis control. In locations where patients can access care in various settings, patterns of care-seeking behavior may be related to drug-resistance. For example, there may be socio-economic factors that are associated with both infection with drug-resistant strains and preference for private sector care or, in areas where second-line treatment for MDR-TB is only offered in one sector, patients with suspected MDR may preferentially move to that sector for treatment. There are few data to suggest how care-seeking behavior may be associated with drug-resistance within any particular location, and, even if these associations were well-measured it would be difficult to generalize these findings to new settings where resistance and treatment may have a very different association.
Our model presents a simplified picture of the mechanisms by which drug resistance appears and spreads within populations and provides insight into the conditions when surveys of drug-resistant disease will be biased in settings where treatment is offered by both private and public practitioners. We find that stringent conditions must be met to assure that resistance among new and retreatment cases is unbiasedly estimated with public sector surveys (Figures 2 and 3). Furthermore, especially when estimating resistance among retreatment cases, it is difficult to predict either the magnitude or the direction of this bias. As the data necessary for trying to infer direction and magnitude of bias is rarely known (and probably never known with sufficient certainty), we propose a simple method for using weighted estimators that requires additional information from the private sector to be combined with estimates from public sector surveys to efficiently and validly estimate resistance in the entire population. In countries with ongoing or routine MDR TB surveys in the public sector, targeted surveys in the private sector provide an efficient mechanism to remove bias. However, even in locations without existing surveillance in the public sector, the stratified sampling estimator can be used to initiate public/private sector surveys (Figures 5 and 6).
In order to obtain this unbiased estimator of MDR TB in the new and retreatment cases using the stratified sampling estimator, we assumed that the burden of disease for each sector is known, in order to directly calculate k FN and k RN (the fraction of new and retreatment cases, respectively, that go to the public sector). Knowledge of the burden of TB by sector also would allow for proportionate division of our sample under fixed resource assumptions (Case 2) in order to minimize the variance of the estimator. In practical terms, this will require national TB programs to strengthen relations with private sector and create mechanisms for reliable notification (or estimation) of new and retreatment disease in both sectors; this remains a immense challenge, especially in areas with many types of private providers and hidden populations of patients receiving informal care. However, as has been noted previously [15, 16], an additional benefit of increasing communication and contact with private providers promises to improve the standard of care for patients being treated outside of the public system.
Our conclusions are based on a crude model for the health seeking behavior of new and retreatment TB cases. Since we were focused on understanding the relationship between health seeking behavior and treatment outcomes of patients on the performance of tuberculosis drug resistance surveys, the model does not include details of the complex natural history of tuberculosis. For example, we do not include age-specific differences in the rates of progression to disease or the risks of reinfection. We also assume a fixed time-step over which cases present for retreatment and that this time-step does not depend on drug resistance phenotype. Omitting these details allows us to gain insight into the potential biases associated with surveys limited to the subset of patients attended to in the public sector, but limits our ability to assess the dynamic behavior of this system. Accordingly, we focus our conclusions on the system in equilibrium. We note that when levels of drug resistance are rising or falling, the potential biases associated with public-sector only surveys will be even more unpredictable, further supporting our recommendation that small surveys in the private sector will be important for understanding the true burden of resistance in locations where patients seek care outside the public system.
Recent recommendations of the WHO and IUATLD recognize the need to assess drug resistance within the private sector in countries where care is offered in settings where surveys are not easily implemented, however these recommendations currently emphasize the need to first expand and improve surveys within the public sector. While we agree that public sector surveys remain a central activity, we suggest that even small additional surveys targeted to the private sector would substantially improve validity of these surveys. In the absence of adequate scale-up of second-line treatment programs for MDR care in the public sector, we expect that suspected MDR cases will increasingly seek care with private providers offering non-standardized care; this would worsen bias and increase the likelihood that resistance among retreatment cases would be underestimated if only measured within the public sector.
TC was funded by NIH grant U54 GM088558-01 (MIDAS) and BH was funded by NIH grant T32 1007358. MP was funded by NIH grant R01 EB006195.
- Anti-tuberculosis drug resistance in the world: report 4. 2008, Geneva: World Health Organization
- Lew W, Pai M, Oxlade O, Martin D, Menzies D: Initial drug resistance and tuberculosis treatment outcomes: systematic review and meta-analysis. Ann Intern Med. 2008, 149: 123-34.PubMedView Article
- Espinal MA, Kim SJ, Suarez PG, Kam KM, Khomenko AG, Migliori GB, Baéz J, Kochi A, Dye C, Raviglione MC: Standard short-course chemotherapy for drug-resistant tuberculosis: treatment outcomes in 6 countries. JAMA. 2000, 283: 2537-45. 10.1001/jama.283.19.2537.PubMedView Article
- Shah NS, Wright A, Bai GH, Barrera L, Boulahbal F, Martín-Casabona N, Drobniewski F, Gilpin C, Havelková M, Lepe R, Lumb R, Metchock B, Portaels F, Rodrigues MF, Rüsch-Gerdes S, Van Deun A, Vincent V, Laserson K, Wells C, Cegielski JP: Worldwide emergence of extensively drug-resistant tuberculosis. EID. 2007, 13: 380-7. 10.3201/eid1303.061400.
- Gandhi NR, Moll A, Sturm AW, Pawinski R, Govender T, Lalloo U, Zeller K, Andrews J, Friedland G: Extensively drug-resistant tuberculosis as a cause of death in patients co-infected with tuberculosis and HIV in a rural area of South Africa. Lancet. 2006, 368: 1575-80. 10.1016/S0140-6736(06)69573-1.PubMedView Article
- Raviglione M: XDR-TB: entering the post-antibiotic era?. Int J Tuberc Lung Dis. 2006, 10: 1185-7.PubMed
- Wright A, Zignol M, Van Deun A, Falzon D, Gerdes SR, Feldman K, Hoffner S, Drobniewski F, Barrera L, van Soolingen D, Boulabhal F, Paramasivan CN, Kam KM, Mitarai S, Nunn P, Raviglione M, Global Project on Anti-Tuberculosis Drug Resistance Surveillance: Epidemiology of antituberculosis drug resistance 2002-07: an updated analysis of the Global Project on Anti-Tuberculosis Drug Resistance Surveillance. Lancet. 2009, e-published Apr 15
- Guidelines for surveillance of drug resistance in tuberculosis. 2003, Geneva: World Health Organization, second
- Uplekar M, Juvekar S, Morankar S, Rangan S, Nunn P: Tuberculosis patients and practitioners in private clinics in India. Int J Tuberc Lung Dis. 1998, 2: 324-9.PubMed
- Uplekar M, Pathania V, Raviglione M: Private practitioners and public health: weak links in tuberculosis control. Lancet. 2001, 358: 912-6. 10.1016/S0140-6736(01)06076-7.PubMedView Article
- TB patients and Private For-Profit Health Care Providers in India. 1997, Geneva: World Health Organization
- Global tuberculosis control: surveillance, planning, financing. 2008, Geneva: World Health Organization
- Fochsen G, Deshpande K, Diwan V, Mishra A, Diwan V, Thorson A: Health care seeking among individuals with cough and tuberculosis: a population-based study from rural India. Int J Tuberc Lung Dis. 2006, 10: 995-1000.PubMed
- Suganthi P, Chadha VK, Ahmed J, Umadevi G, Kumar P, Srivastava R, Magesh V, Gupta J, Sharda MA: Health seeking and knowledge about tuberculosis among persons with pulmonary symptoms and tuberculosis cases in Bangalore slums. Int J Tuberc Lung Dis. 2008, 12: 1268-73.PubMed
- Floyd K, Arora VK, Murthy KJ, Lonnroth K, Singla N, Akbar Y, Zignol M, Uplekar M: Cost and cost-effectiveness of PPM-DOTS for tuberculosis control: evidence from India. Bull World Health Organ. 2006, 84: 437-45. 10.2471/BLT.05.024109.PubMedPubMed CentralView Article
- Dewan PK, Lal SS, Lonnroth K, Wares F, Uplekar M, Sahu S, Granich R, Chauhan LS: Improving tuberculosis control through public-private collaboration in India: literature review. BMJ. 2006, 332: 574-8. 10.1136/bmj.38738.473252.7C.PubMedPubMed CentralView Article
- Prasad R, Nautiyal R, Mukherji P, Jain A, Singh K, Ahuja R: Diagnostic evaluation of pulmonary tuberculosis: what do doctors of modern medicine do in India?. Int J Tuberc Lung Dis. 2003, 7: 52-7.PubMed
- Greaves F, Ouyang H, Pefole M, MacCarthy S, Cash R: Compliance with DOTS diagnosis and treatment recommendations by private practitioners in Kerala, India. Int J Tuberc Lung Dis. 2007, 11: 110-2.PubMed
- Sisodia R, Wares D, Sahu S, Chauhan L, Zignol M: Source of retreatment cases under the revised national TB control programme in Rajasthan, India, 2003. Int J Tuberc Lung Dis. 2006, 10: 1373-9.PubMed
- Hedt BL, Pagano M: Health Indicators: Eliminating bias from convenience sampling indicators. Statistics in Medicine.
- Yule G: Notes on the Theory of Association of Attributes in Statistics. Biometrika. 1903, 2: 121-134. 10.1093/biomet/2.2.121.View Article
- Simpson E: The Interpretation of Interaction in Contingency Tables. Journal of the Royal Statistical Society. Series B. 1951, 13: 238-241.
- Guidelines for the surveillance of drug resistance in tuberculosis. 2009, Geneva: World Health Organization, fourth
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/10/355/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.