Implications of the HIV testing protocol for refusal bias in seroprevalence surveys
© Reniers et al. 2009
Received: 19 May 2008
Accepted: 28 May 2009
Published: 28 May 2009
Skip to main content
© Reniers et al. 2009
Received: 19 May 2008
Accepted: 28 May 2009
Published: 28 May 2009
HIV serosurveys have become important sources of HIV prevalence estimates, but these estimates may be biased because of refusals and other forms of non-response. We investigate the effect of the post-test counseling study protocol on bias due to the refusal to be tested.
Data come from a nine-month prospective study of hospital admissions in Addis Ababa during which patients were approached for an HIV test. Patients had the choice between three consent levels: testing and post-test counseling (including the return of HIV test results), testing without post-test counseling, and total refusal. For all patients, information was collected on basic sociodemographic background characteristics as well as admission diagnosis. The three consent levels are used to mimic refusal bias in serosurveys with different post-test counseling study protocols. We first investigate the covariates of consent for testing. Second, we quantify refusal bias in HIV prevalence estimates using Heckman regression models that account for sample selection.
Refusal to be tested positively correlates with admission diagnosis (and thus HIV status), but the magnitude of refusal bias in HIV prevalence surveys depends on the study protocol. Bias is larger when post-test counseling and the return of HIV test results is a prerequisite of study participation (compared to a protocol where test results are not returned to study participants, or, where there is an explicit provision for respondents to forego post-test counseling). We also find that consent for testing increased following the introduction of antiretroviral therapy in Ethiopia. Other covariates of refusal are age (non-linear effect), gender (higher refusal rates in men), marital status (lowest refusal rates in singles), educational status (refusal rate increases with educational attainment), and counselor.
The protocol for post-test counseling and the return of HIV test results to study participants is an important consideration in HIV prevalence surveys that wish to minimize refusal bias. The availability of ART is likely to reduce refusal rates.
Progress in medical technology has brought rapid HIV testing within reach of nationally representative surveys. This has generated new prospects for resolving bias in HIV prevalence estimates based on antenatal clinic (ANC) sentinel surveillance data, or, for providing a new gold standard for HIV prevalence estimates altogether [1–5]. However, data from population-based surveys are also subject to bias due to the exclusion of high risk groups from the sampling frame, and non-response because of population mobility and refusal. The association between mobility and HIV infection has been documented extensively [6–9]. In comparison, relatively little is known about the relationship between refusal and HIV infection in nationally representative surveys [2, 3, 5, 10]. Several small-scale studies in STD and antenatal clinics have concluded that refusals are positively associated with HIV infection [11–19]. Three studies remain inconclusive about the nature of the relationship or suggest the opposite pattern [20–22]. On aggregate, population-based surveys are believed to underestimate true HIV prevalence, but most studies have not been able to identify significant bias due to testing refusal [5, 9, 23–27]. Two studies challenge that optimism [28, 29].
A study design feature that may contribute to differences in refusal bias is the protocol for post-test counseling and the return of test results. Agreeing to post-test counseling and the return of test results is often a prerequisite of study participation in health facility-based studies. This contrasts with most population-based surveys that follow a protocol in which respondents or clients do not receive their HIV test results (e.g., Demographic and Health Surveys). Instead, they are given a voucher for retesting at the nearest Voluntary Counseling and Testing (VCT) center at no cost (if the service is not already free of charge) . In early studies involving testing for HIV, the return of test results was not usually an option because samples had to be shipped to an off-site lab for analysis. With the increasing availability and reliability of rapid tests, the return of test results is now feasible in the same session in which the specimens are collected. Because of the ethical prescription that study participants should share in the benefits of research , the pressure to provide post-test counseling including the return of HIV test results in HIV prevalence surveys is likely to increase in the future. Therefore, it is important to assess how to accommodate this guideline in the testing protocol while preserving or maximizing the external validity of the ensuing HIV prevalence estimates.
Using data from a health facility in Addis Ababa, we first present covariates of testing refusal. Second, we quantify refusal bias in HIV prevalence estimates under different post-test counseling study protocols via regression models that account for sample selection. A final noteworthy feature of our study is that antiretroviral therapy (ART) was introduced in Ethiopia during the course of data collection, and that allows us to evaluate its impact on refusal rates.
The data for this study come from prospective monitoring of hospital admissions and outpatient visits which was initiated at Zewditu Memorial Hospital in May 2003 and continued for nine months. Zewditu Memorial Hospital is a government facility in the inner city of Addis Ababa and was one of the few hospitals with a VCT center of sufficient capacity to accommodate our study. Initially, the study covered the TB-HIV clinic (TB, ambulatory patients), the medical emergency (ER), internal medicine (IM), gynaecology (GY), and pediatric wards (PE). For each patient, a ward nurse collected basic background characteristics (age and sex) as well as the admission and discharge diagnosis. One month into the study, the surgical ward (SU) was included and we added educational status, religion, birthplace and marital status as background variables on the data collection forms. After new patients were identified, a VCT-nurse did pre-test counseling and asked for written consent of the patient. For minors, consent was obtained from the parent or guardian along with the assent of the patient him- or herself.
Following pre-test counseling, patients had the option to participate in the study with the return of HIV test results and post-test counseling (consent level A), to participate in the study without the return of test results or post-test counseling (consent level B), or, to decline testing and counseling altogether (consent level C). In the remainder of this article, we refer to post-test counseling as the interaction between counselor and client that includes the return of the test result and a counseling session tailored to the HIV status of the client. The three consent levels help us mimic refusal bias under different post-test counseling study protocols: we consider consent level C to represent refusals in a protocol where post-test counseling is not offered to respondents (or where an explicit provision exists to test without post-test counseling), and we combine consent levels B and C to represent refusals in a protocol where post-test counseling is a requirement of study participation.
After consent was obtained, the VCT-nurse administered a Determine Rapid HIV1-2 test. Capillus™ HIV-1/HIV-2 confirmatory tests were done on positive samples, and if the outcomes of these tests were discrepant a Uni-Gold™ HIV test was done as a tie breaker. Tests were offered free of charge. Nine VCT nurses carried out counseling and anywhere between two and four nurses covered each ward. All but one of the counselors were female.
To study whether refusals are more common among patients with a higher likelihood of infection, we rely on the admission diagnosis because (1) it is correlated with HIV status, and (2) also observed for patients who were not tested. We use admission rather than discharge diagnosis because it is less likely to be influenced by the test result itself. The availability of information on the medical condition of respondents constitutes an important advantage of a medical facility-based sample. In contrast, most measured traits correlate weakly with HIV status in community-based studies, and that renders assessments of refusal bias in HIV prevalence estimates questionable. The downside of a medical facility-based study is that it is not necessarily representative of the determinants of participation in population-based surveys (e.g., levels of -undisclosed- prior knowledge of one's HIV status may be higher in a health facility sample, and prior knowledge of HIV positive status has been identified as a source of bias in HIV prevalence estimates ). Therefore, our estimates of the degree of refusal bias cannot be extrapolated to general population surveys, but a health facility-based sample is probably satisfying to identify the type of post-test counseling protocol that minimizes bias (i.e., to identify the relative magnitude of refusal bias under different study protocols).
Admission diagnoses and likelihood of infection. Zewditu Memorial Hospital, Addis Ababa (2003–04, age 16 and above)
Diarrhoea and GE of presumed infectious origin
Herpes zoster, oral candiasis, toxoplasmosis and PCP
B02, B37, B58–59
Other infectious and parasitic diseases
A01, A03, A07, A30, A35, A41, A63–64, A68, A75, A82, B45
Neoplasm's of breast, cervix, uterus and leiomyoma
C50, C53–55, D25–26
Other neoplasms (benign and malignant)
C0, C2–4, C51–52, C56–58, C6–9, D0, D22–24, D3–4
Diabetes and hypoglycemia
Diseases of the nervous system (mainly meningitis)
G00, G03–04, G25, G40, G54
Other diseases of the circulatory system
I05, I09, I15, I21, I31, I38, I49–51, I61, I63–64, I80, I83–I84, I86, I88
Other diseases of the respiratory system
J11, J44–46, J86, J90, J93–94, J98
Gastritis and other diseases of the oesophagus, stomach and duodenum
Diseases of the appendix
Hernia and intestinal obstruction
K40, K42–43, K46, K56
Cholelithiasis and diseases of the pancreas
K80, K82, K85–K86
Other diseases of the digestive system
K04, K12, K60, K62–63, K65–66, K72–73, K75–76, K83, K91–93
Diseases of the skin and subcutaneous tissue
Glomerular diseases and diseases of the urinary system
Diseases of male genital organs
Inflammatory diseases of female pelvic organs and disorders of the female genital tract
Complications of pregnancy and delivery
Fever of unknown origin
Symptoms signs and abnormal clinical findings not elsewhere specified
R0–4, R56–58, R62
External causes and injuries
S, T, X
Other and unknown admission diagnoses
A80, B19, B56, D5–8, E15, E40–42, E55, E83, E86, E88, K36, P07, Q43, Q53, U, Z4
The likelihood of infection as measured by the admission diagnosis is first used as a predictor in logistic regression models with the consent level as the outcome. In these models, we verify whether the effect of the infection likelihood persists while controlling for other characteristics of the respondent. In the next step, we estimate refusal bias in HIV prevalence via a comparison of observed HIV prevalence estimates and predicted values generated by Heckman probit models that account for sample selection [32–34]. The Heckman sample selection model corrects for the possibility that HIV prevalence is different in respondents who refuse testing. More formally, the Heckman sample selection model is a two-equation model consisting of a regression and a selection equation that are simultaneously estimated. The regression equation predicts HIV status: y = Xβ + u 1 where X is a vector of covariates. The selection equation specifies that HIV status is only observed if Zγ + u 2 > 0. In this equation Z stands for a vector of characteristics that affect consent for HIV testing. The error terms in both equations are assumed to be normally distributed. Ordinary probit estimates of the parameters in the regression equation are biased when ρ (the correlation between u 1 and u 2) is not zero. The Heckman selection model lets us use information for patients who refused the HIV test (e.g., counselor, admission diagnosis, and other sociodemographic background characteristics) to improve estimates of parameters in the regression model, and thus improve estimates of HIV prevalence (i.e., the mean predicted value).
We limit the study population in four respects. The first set of excluded cases is multiple admissions of the same individual. We only consider first admissions because higher order admission diagnoses might be influenced by the test outcome at first visit, and thus introduce problems of reverse causality. For the same reason, we exclude individuals who volunteered their HIV status. The third excluded category is patients under 16 years old, primarily because we wish to restrict our study population to an age range that is common in seroprevalence surveys. The TB/HIV clinic constitutes another special case. HIV testing is standard practice in diagnosing patients of the TB/HIV clinic and some are referred to it precisely for that reason. The TB/HIV clinic of Zewditu Memorial Hospital was also one of the pioneering ART facilities in Ethiopia, which contributes to the (self-) selection of patients.
The study protocol was approved by the Research and Publications Committee of the Addis Ababa University Faculty of Medicine and received ethics clearance from the Ethiopian Science and Technology Agency, and the Institutional Review Board of the University of Pennsylvania. Written informed consent was obtained for administering and using the HIV test results for research purposes. No individual informed consent was requested for using (anonimized) background characteristics and the admission diagnosis of patients who refused the HIV test.
Consent for testing and HIV status (Zewditu Memorial Hospital, Addis Ababa, 2003–04)
Study participants (column %)
Consent level A (testing & post-test counseling)
Consent level B (testing only)
Consent level C (total refusal)
Known HIV status
Discharged/expired prior to testing
Covariates of consent for HIV testing (Zewditu Memorial Hospital, Addis Ababa, 2003–04)
Consent level (row %)
Consent level (row %)
Pearson Chi2(10) = 28.99 p <.01
Pearson Chi2(14) = 276.27, p <.01
Pearson Chi2(8) = 37.46 p <.01
Pearson Chi2(6) = 134.44, p <.01
Study month a
Prior to ART
Pearson Chi2(2) = 44.72, p <.01
Pearson Chi2(4) = 14.49, p <.01
Likelihood of infection a
Pearson chi2(2) = 1.86, p = 0.40
7.5 – 14.9
15.0 – 29.9
Pearson Chi2(6) = 44.05, p <.01
Particularly relevant for the analysis of bias in HIV-prevalence estimates is the association between the likelihood of infection and refusal: consent for testing and post-test counseling (consent level A) drops from over 83% in patients with the lowest likelihood of infection to just under 70% among those with the highest likelihood of infection. This is partly compensated by an increasing share of patients who consented to testing without post-test counseling (consent level B) as the likelihood of infection increased.
Binary and multinomial logistic regressions predicting refusal of testing for HIV (Zewditu Memorial Hospital, Addis Ababa, 2003–04)
Binary logistic regression predicting refusal
(exp(b) or odds ratios)
Multinomial logistic regression predicting refusal
(exp(b) or relative risk ratios)
B & C versus A
B versus A
C versus A
B versus A
C versus A
Likelihood of infection
Counselor (vs #1)
Study month (vs period prior to ART)
Ward (vs ER)
Education (vs no schooling)
> 12th grade
Marital status (vs never married)
LR chi2 (df)
Prob > chi2
Breaking down the outcome by level of consent (models 3 and 4) changes little in terms of the substantive conclusions compared to the binary logistic regression models. The most noteworthy differences are that age is a weak predictor of total refusal (consent level C versus A), and that educational status does not have an effect in the equation predicting testing without versus testing with post-test counseling (consent level B versus A). The parameters for marital status point in the same direction as in the binomial model but vary in their significance level.
Comparison of HIV seroprevalence estimates based on standard probit models and models accounting for sample selection under various scenarios (Zewditu Memorial Hospital, Addis Ababa, 2003–04)
Test of Heckman modela
Post-test counseling is required
Post-test counseling is optional
E(HIV% – Probit)
(16.4 – 19.1)
E(HIV% – Heckman)
(21.7 – 24.4)
(19.9 – 24.4)
Consent groups A and B
All consent groups
All consent groups
HIV status in consent group B is unobserved
HIV status in consent groups B and C is unobserved
HIV status in consent group C is unobserved
LR test H0:ρ = 0
In the first column we present a simple empirical test of the Heckman model. Here we assume that we do not have information on HIV status for those who agreed to test but declined post-test counseling (consent level B), and we predict HIV prevalence in the sample consisting of individuals in consent levels A and B only using an ordinary probit model and a probit model that accounts for sample selection. Because we know the HIV prevalence in the total sample, we can compare the probit estimates with observed HIV prevalence. The ordinary probit estimate of HIV prevalence is 17.7%, the selection model establishes HIV prevalence at 23.1%, and the true or observed value is 22.2%. Heckman estimates are thus more accurate than standard probit estimates of HIV prevalence. The LR test confirms that selection bias is significant.
It is noteworthy that ρ = 0 for a Heckman model that only includes basic sociodemographic background characteristics (sex, age, marital status, and education) in the selection equation (not shown). That model also underestimates HIV prevalence. Adding counselor to the selection equation renders ρ ≠ 0, but significantly overestimates HIV prevalence. Inclusion of information on the health status of patients – an indicator that correlates well with HIV status and consent for testing – thus considerably improves Heckman predictions of HIV prevalence. This sensitivity analysis confirms an earlier finding that the validity of Heckman estimates are subject to the specification of the selection equation . The specification used in this application, however, produces good estimates of HIV prevalence.
The last two columns compare HIV estimates for two plausible study protocols. Bias in prevalence estimates is substantial if response is dichotomized into refusal or full participation without the option of testing without post-test counseling (column 2). This scenario is most typical for clinical intervention studies. Bias is much smaller, and only marginally statistically significant, when the study protocol explicitly allows participants to opt out of post-test counseling and the return of HIV test results. This is shown in the third column. This scenario is more typical for population-based serosurveys.
Our analyses establish that consent for testing is correlated with the likelihood of HIV infection (assessed in terms of the diagnosis at admission): patients who agree to testing with or without post-test counseling (consent levels A and B) are less likely to be infected than those who refuse an HIV test (consent level C). This relationship implies that testing refusal constitutes a potential source of bias in HIV prevalence estimates. Regression methods that account for sample selection confirm this, but qualification is required in two respects. First, our study is based on a hospital population and demands confirmation in a more general sample. Second, much seems to depend on the study protocol and informed consent procedures. In this sample, bias is limited if respondents are offered the opportunity to opt out of post-test counseling and the return of test results.
Because most population-based surveys utilize a testing protocol that does not necessarily involve post-test-counseling and the return of test results, they are less likely to be affected by refusal bias than studies where post-test counseling is a requirement for study participation. This does not mean, however, that HIV prevalence estimates from population-based serosurveys are free of bias. First, we identified marginally significant bias under the assumptions of a protocol whereby test results are not returned to respondents. Second, bias may result from other sources than those studied here (e.g., limitations of the sampling frame and other forms of non-response).
Although this paper has focused on the relationship between the likelihood of infection and consent for testing, it is not the most important predictor of consent. The largest variation in consent is produced by the counselors, which suggests that studies interested in minimizing non-response must be careful in the selection and training of their fieldwork team. Unfortunately our study was not designed to assess the possible reasons for the variable study enrollment rates by counselor (e.g., via the randomization of counselors across wards). We have no reason, however, to suspect significant bias in HIV prevalence estimates due to variability in consent attributable to counselors. Another covariate of consent is the availability of ART. In our study, the odds to consent for testing and counseling increased by about 20% per month following the launch of a governmental ART program. The absence of a control group, however, does not allow us to exclude other factors that may be responsible for this association. The finding that patients are more likely to agree to testing once treatment becomes available is nonetheless plausible, and confirms findings from another observational study .
The protocol for post-test counseling and the return of HIV test results to study participants is an important determinant of consent for testing, and should be carefully evaluated in studies that wish to minimize refusal bias in HIV prevalence surveys. For the sake of scientific accuracy, it is recommended to provide a modality to test without post-test counseling when introducing the study protocol to respondents. In studies where there is a long wait between testing and the availability of test results, this is often a de-facto option. As technological advances in rapid testing methods reduce the waiting time, however, this will become a consideration of increasing importance. To date, most population-based serosurveys have followed a protocol that did not involve the return of HIV test results. To the extent that our findings can be extrapolated to non health facility-based settings, this study suggests that in doing so, these surveys have avoided a potentially important source of bias. Finally, we find that the availability of ART is likely to reduce refusal rates, and thus the potential for refusal bias.
This study has been made possibly with financial support from the AIDS Foundation of Amsterdam (grant #7022), the World Health Organization (OD/TS-07-00275/A21-181-6), and a Hewlett Foundation grant to the University of Colorado at Boulder for the African Population Studies Research and Training Program. We wish to thank the following institutions for their support: the Population Studies Center (University of Pennsylvania), the Public Health Service of Amsterdam (GGD Amsterdam), the Medical Faculty of Addis Ababa University, and the Zewditu Memorial Hospital. We wish to thank the nurses and VCT team for their dedicated work. Admission diagnoses were coded with the assistance of Dr. Abiy Arefeayine and Nurse Misganaw Getaw. Tariqua Tadesse and Yeshi G/Wold oversaw data collection and data entry. We acknowledge the thoughtful comments and suggestions from Jimi Adams, Derek Briggs, Doug Ewbank, Richard Rogers, Christie Sennott, Rania Tfaily, the late Etienne van de Walle, Susan Watkins, and the journal's reviewers. The content of this publication is the sole responsibility of the authors and does not represent the views of the supporting institutions or funding agencies.
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.