- Research article
- Open Access
Pharmacy refill adherence outperforms self-reported methods in predicting HIV therapy outcome in resource-limited settings
BMC Public Health volume 14, Article number: 1035 (2014)
Optimal adherence to antiretroviral therapy is critical to prevent HIV drug resistance (HIVDR) epidemic. The objective of the study was to investigate the best performing adherence assessment method for predicting virological failure in resource-limited settings (RLS).
This study was a single-centre prospective cohort, enrolling 220 HIV-infected adult patients attending an HIV/AIDS Care and Treatment Centre in Dar es Salaam, Tanzania, in 2010. Pharmacy refill, self-report (via visual analog scale [VAS] and the Swiss HIV Cohort study-adherence questionnaire), pill count, and appointment keeping adherence measurements were taken.
Univariate logistic regression (LR) was done to explore a cut-off that gives a better trade-off between sensitivity and specificity, and a higher area under the curve (AUC) based on receiver operating characteristic curve in predicting virological failure. Additionally, the adherence models were evaluated by fitting multivariate LR with stepwise functions, decision trees, and random forests models, assessing 10-fold multiple cross validation (MCV). Patient factors associated with virological failure were determined using LR.
Viral load measurements at baseline and one year after recruitment were available for 162 patients, of whom 55 (34%) had detectable viral load and 17 (10.5%) had immunological failure at one year after recruitment. The optimal cut-off points significantly predictive of virological failure were 95%, 80%, 95% and 90% for VAS, appointment keeping, pharmacy refill, and pill count adherence respectively. The AUC for these methods ranged from 0.52 to 0.61, with pharmacy refill giving the best performance at AUC 0.61.
Multivariate logistic regression with boost stepwise MCV had higher AUC (0.64) compared to all univariate adherence models, except pharmacy refill adherence univariate model, which was comparable to the multivariate model (AUC = 0.64). Decision trees and random forests models were inferior to boost stepwise model.
Pharmacy refill adherence (<95%) emerged as the best method for predicting virological failure. Other significant predictors in multivariate LR were having a baseline CD4 T lymphocytes count < 200 cells/μl, being unable to recall the diagnosis date, and a higher weight.
Pharmacy refill has the potential to predict virological failure and to identify patients to be considered for viral load monitoring and HIVDR testing in RLS.
Widespread antiretroviral scale-up programs are taking place in resource-limited settings (RLS) . Where resources are available, monitoring response to combination antiretroviral therapy (ART) by virological outcome is recommended because of its strong correlation with therapy success in HIV patients. At virological failure, therapy changes are then guided by genotypic resistance testing. However, sophisticated laboratory tests, such as virological monitoring and genotyping, are very expensive and not used in RLS [1–3]. Therefore, in RLS, clinical or immunological criteria are used to evaluate ART outcome. It has been shown that relying on such clinical and immunological criteria to change therapy can cause high levels of HIV drug resistance (HIVDR), compromising the next line of therapy [2, 3]. It is, therefore, important to identify affordable proxy markers for response to therapy in RLS. There is sufficient international evidence to support that adherence to combination ART is a major predictor of viral suppression [4, 5], HIVDR [6, 7], CD4 T lymphocytes count recovery  and survival [5, 8, 9].
Strategies to measure adherence to ART include the Medication Event Monitoring System (MEMS), pharmacy refill records, pill count, appointment keeping, adherence diaries, and interviewer-administered or self-report questionnaires [10–13]. Current objective methods of measuring adherence include MEMS, biologic measures, and pill counts [14, 15]. Each of these assessment methods is associated with certain strengths and drawbacks. MEMS is widely used in clinical studies, whereas pill count, self-report, and pharmacy refill are widely used in the context of HIV and AIDS clinical-care settings.
Self-report utilizing structured questionnaires is popular in RLS, due to its ease of use in busy settings, affordability, and low staff requirements. Self-report has consistently been correlated with viral load response to therapy and has been proposed as a robust and appropriate indicator of adherence [16, 17]. However, other documented studies showed that self-reports have low sensitivity and positive predictive value . While self-reported adherence may be consistently correlated with important clinical outcomes, these relationships are generally modest at best, and the method continues to lack accuracy and precision. Self-reported adherence is known to overestimate adherence, due to recall bias and social desirability. Furthermore, the inaccuracy of self-reports is highlighted by the need to often dichotomize the highly-skewed data with cut-off points of 90 – 100% in most adherence analyses . A combination of adherence strategies is recommended to improve the accuracy of this measurement .
In sub-Saharan Africa, adherence to ART has been touted to be equal to or even higher than levels in resource-rich settings (RRS) . However, many adherence studies are based on non-objective self-report measurements and are not well validated against viral load measurement . Therefore, it is still arguable whether these higher adherence reports reflect the actual situation. Moreover, treatment failures have been shown, even with self-reported high adherence . It is important to acknowledge that self-reported adherence measures are far from robust. Its wide use and acceptance need further justification than the few selected studies that show correlation between this measure with other objective measures and with virological, clinical immunological, and survival outcome .
Pill count and appointment keeping adherence are among the early warning indicators (EWIs) recommended by the World Health Organization (WHO) as surrogate markers for emergence of HIVDR . Manual pill count is a cheap and easy alternative to estimate dosages taken. However, it is limited by the intensive need of staff and the possibility of pill sharing or dumping by the patients. Appointment keeping can be easily abstracted from attendance records, but is only a subjective measure.
Another objective method that has higher potential is pharmacy refill adherence, which is commonly obtained in clinical care. It has been validated as a measure of ART adherence related to viral load . Although pharmacy refill method is not constrained by the recall bias seen with self-report, its utility for predicting patient outcomes is poorly validated in RLS [23, 24].
Apparently, there is no single consensus tool to capture adherence to ART and no accepted gold standard to assess adherence especially in RLS [14, 15, 25, 26]. Moreover, studies show low or moderate correlation among methods, which can be attributed to the fact that they measure separate dimensions of adherence behaviour, that different non-adherence cut-off points have been established, or even to the limitations of the methods themselves. These issues result in varied non-adherence frequency measures throughout time and among diverse settings.
Validating adherence measurement against viral load is of utmost importance. An accurate and sensitive measure of adherence should give clinicians objective data to use during discussions on individuals’ non-adherence risk factors without causing shame or stigma to a patient . The fear of stigma is a problem in non-adherence disclosure and can lead to over-reporting of adherence. The objective of the current study was to validate an instrument capable of measuring adherence to ART in HIV-infected adults by evaluating how such measurements can be used to predict virological outcome in Tanzania. Such an instrument can be used to select patients for viral load measurement and HIVDR testing, as a lower-cost alternative to generalized viral load follow-up.
This was a single-centre prospective cohort study enrolling HIV-infected adult patients attending an HIV/AIDS Care and Treatment Centre (CTC) that provides ART at Amana District Hospital in Dar es Salaam, Tanzania, in 2010. Patients on ART are normally scheduled to return for antiretroviral (ARV) pick-ups at least once per month. During the visit, they also consult with the clinician after receiving adherence counselling from the nurses. CD4 T lymphocytes count measurements are taken at least once every six months. Two hundred and fifty-four patients were invited into the study during the months of May to July, 2010. Selection criteria were either starting ART or being on ART. Patients were conveniently sampled, as the study researcher continuously recruited patients who were referred by nurse counsellors, the first contacts at the clinic. Each day, 10-15 unselected patients were recruited into the study. Exclusion criteria were being younger than 18 years of age, pregnant, having opportunistic infections, or malignancy. Of the 254 patients recruited into the study, 34 (13.4%) were excluded from analysis for various reasons (see Figure 1). The remaining 220 provided adherence information and were followed for a period of one year. The herein presented study included 162 patients who completed a one-year follow-up and had their adherence, immunological, and virological outcome monitored over the entire period.
Issues pertaining to patient confidentiality, benefits and risks to participating patients, justice, rights and respect that the patients deserve were addressed by ethical clearance and informed consent. The study was approved by the Muhimbili University of Health and Allied Sciences (MUHAS) Research Ethics Committee. Only patients who were willing to participate in the study and who signed an informed consent were recruited into this study. Patient codes were used to de-link the patient data in databases. Patients did not receive any payments to motivate them to participate in the study.
Virological and immunological outcomes
Virological and immunological outcomes were assessed at the one-year follow-up. These outcomes were dichotomized according to the following definitions: Virological failure was defined as having a viral load above the detection limit of 400 copies/mL; and immunological failure was defined according to the WHO guidelines as having (i) a CD4 T lymphocytes count of 100 cells/μl at six months post ART, (ii) a CD4 T lymphocytes count of equal to or less than CD4 pre-ART at six months on ART, or (iii) 50% reduction from the on-ART peak CD4 T lymphocytes count [28, 29].
Adherence was measured using pharmacy refill, self-report, pill count, and appointment keeping methods, as described for each method. Adherences measurements were taken at four time points during a one-year follow-up, including at recruitment (zero), one, two, and 12 months after recruitment. Overall adherence for each method was the mean of the measurements taken at the four time points and this mean was considered in subsequent analyses.
Each refill period was identified as the interval between the last visit date and the scheduled refill date. Refill adherence was 100% if all pills during the scheduled refill period had been picked up on time. Refill percent values above 100 for patients who refilled earlier than scheduled were rounded to 100 percent. Refill adherence was not calculated on a monthly basis to reduce an error of a few additional pills left over at the end of each refill period. The calculation was based on cumulative sum of the days that a patient was late for ARV pick-up appointments in each month over the year, divided by the total number of days over all periods between pick-up periods in the year of study, resulting in the percentage of time the patient was without medication over the whole year.
The researcher for this study administered a self-report questionnaire to patients to assess missed dosages over the previous one month. During a one-year follow-up, self-report was used to determine adherence at recruitment (zero), one, two, and 12 months after recruitment. For patients who started ART at recruitment, the measurement at month zero was not available. Patient self-reported adherence was investigated using a validated study tool . This validated study tool consists of two major sections: 1) the visual analog scale (VAS), which probed the percentage of doses taken in the previous month; and 2) it probes two questions from the Swiss HIV Cohort Study Adherence Questionnaire (SHCS-AQ) regarding frequency of missed doses and if a patient ever missed two consecutive doses (drug holiday) in the previous month. By definition, reporting of having missed at least one ARV dose or two consecutive doses in the month preceding the interview was scored as non-adherent [26, 30].
At each visit, pills remaining in bottles were counted and the proportion of these pills to the dispensed pills during the previous visit was calculated based on the dose and the number of days dispensed. The pill count adherence percent was obtained by dividing the number of pills consumed by the total number of pills at the beginning of the given interval and multiplied by 100.
At this CTC, appointments were issued based on the period that the dispensed ARV stock would last, which usually this amounts to one month. Appointment keeping was deduced from the medical records and database. A patient visiting on the scheduled appointment was scored as 100% appointment adherent, whereas if an appointment was skipped entirely, it was scored as 0% adherence. For a delayed appointment, the proportions of days late (in% with respect to the total number of days between two visits) were subtracted from 100.
Univariate analysis was done to explore a cut-off that gives a better trade-off between sensitivity and specificity, and a higher area under the curve (AUC), based on receiver operating characteristic (ROC) curve in predicting virological failure. For each method, we used a threshold cut-off beginning with 100% and decreasing to 50% in steps of 5. At the selected cut-off for pharmacy refill, VAS, pill count, and appointment keeping (a binary variable-expressed adherence, such that patients with adherence below the cut-off were classified as non-adherent, whereas the others were classified as adherent). The trade-off for sensitivity and specificity was analysed upon changing the adherence cut-off to predict virological failure at 12 months, which was dichotomized using two different viral load cut-off points: > 400 copies/mL or > 1,000 copies/mL. The 400 copies/ml virological cut-off was the detection limit of this study’s viral load measurement, while 1,000 represented a virologic cut-off widely used to perform genotyping. A model was also tested combining adherence and immunological outcome to predict virological failure. The performance criteria for all adherence measures were calculated with two-by-two tables and ROC analysis. Univariate logistic regression (LR) was used to determine patient baseline factors associated with virological outcome. Also, evaluation of the adherence models was done by fitting multivariate LR with stepwise functions (Akaike information criterion [AIC] and boost), decision trees, and random forests assessing extra-sample error via multiple 10-fold cross validation (MCV), repeated 50 times with random seeds. All models were analysed in terms of different performance (goodness-of-fit functions), specifically AUC, accuracy, true positive rate (sensitivity), and true negative rate (specificity). Performance distributions were compared pair-wise with a t-test adjusted for sample overlap (due to the MCV procedure) and multiple comparisons.
Kaplan Meier survival analysis was used to determine the difference between the time taken to increase the CD4 T lymphocytes count by 50 cells/μl or by 100 cells/μl between adherent and non-adherent patients. Furthermore, Cox regression analysis was done to adjust for confounding factors such as age, gender, and CD4 T lymphocytes count at baseline. Descriptive analyses, including median interquartile range (IQR) for numerical variables, frequencies, and proportions for categorical variables, and tested for association using Fisher and Chi-square tests for categorical variables, and Wilcoxon signed rank test for continuous values. The level of significance for all analyses was p < 0.05. All statistical analyses were performed using the R statistical package version 2.15.1  and Weka open source software .
A total of 162 (73.6%) out of the 220 patients followed for a one-year period had a virological outcome, an immunological outcome, and adherence data at the one-year time point and were included in this study (see Figure 1). The median IQR follow-up duration was 13 (11 - 13) months. Of the 220 patients, 49 (22.3%) were excluded because they could not be interviewed at the one-year follow-up. Four (1.8%) had died before the one-year end point; eight (3.6%) were transferred to other centres; 20 (9.1%) visited in the last month, but could not meet the interviewer; and 17 (7.7%) were likely to be lost to follow-up, as they had missed their appointments for at least three months. An additional nine (4.1%) patients were excluded from the study, due to lack of viral load measurement. There was no significant difference in terms of socio-demographic and social support characteristics, adherence measured by different methods, and immunological status between included and excluded patients (see Additional file 1). They also did not differ in duration of therapy or support for treatment. However, included patients were more likely to receive atripla or a once-daily single tablet regimen than the excluded group, while other therapy regimen distribution was not different. Also, included patients had a CD4 T lymphocytes count measurement performed significantly more often than the excluded group; eight (6 - 11) times compared to seven (4 - 11) times (p = 0.03).
Among the patients included in the study, the median IQR of ART exposure by the end of the observation period was 37 (30 - 48) months. This information was only available for patients with follow-up. With regard to duration of therapy at recruitment, there was no significance difference between included and excluded patients (p = 0.36).
Performance of adherence models
With regard to adherence measurements, performance for various cut-off points to predict a viral load above 400 or above 1,000 copies/ml is shown in Table 1, Additional file 2, and Additional file 3. The 400 copies/ml virological cut-off was the study’s detection limit of viral load measurement, while 1,000 represents virologic cut-off widely used to perform genotyping. For SHCS-AQ, there are no cut-off points, as non-adherence is defined as in the methods. Evaluation of prediction by adherence measurements taken at different time points of follow-up and by overall adherence was similar (see Additional file 3). For the remainder of this paper, only overall adherence is considered. Prediction performance was better when predicting a viral load above 1,000 copies/ml (compared to above 400 copies/ml; see Additional file 2), but this viral load was not the clinically relevant one and was not analysed further. The optimal trade-off between highest AUC, acceptable sensitivity and specificity, and still being significantly predictive for virological failure (VL > 400 copies/ml) was 95%, 80%, 95% and 90% for VAS, appointment keeping, pharmacy refill, and pill count adherence respectively (see Table 1). At these adherence cut-off values, the proportion of adherent patients were 87.0%, 78.9%, 24.5%, and 54.3% for VAS, appointment keeping, pill count, and pharmacy refill adherence respectively. Whereas, for SHCS-AQ 71.6% of patients were adherent. For VAS, appointment keeping, pharmacy refill, and pill count adherence, the median IQR was 100 (98.33 - 100), 100 (84.33 - 100), 89.55 (86.23 - 100), and 78.83 (74.04 - 89.90) respectively. Combining adherence and immunological outcome improved the accuracy and sensitivity to predict virological failure, while specificity was not much worse (see Table 1).
Factors significantly associated with virological and immunological outcome
At recruitment, 13% of patients who were on treatment for more than four months had detectable viral load. This rate increased to 34% of all patients after the one-year follow-up. Two of 15 patients who started ART at recruitment still had a detectable viral load after one year. The odds of virological failure for various predictors are shown in Table 2. Patients who had a detectable viral load did not differ significantly from those with an undetectable viral load in terms of socio-demographic characteristics, except with respect to distance travelled to the centre: those with undetectable viral load lived closer, six (2 - 8) km as compared to seven (4.5 - 8.5) km in the subset with detectable viral load (p = 0.04). Patients with detectable viral load were more significantly taking triomune, a fixed combination of stavudine, lamivudine, and nevirapine, than those with undetectable viral load (p = 0.05). Detectable viral load at study entry, immunological failure at one year, and poor pharmacy refill adherence were other significant predictors of detectable viral load at the end of the study. Having a drug holiday and baseline CD4 T lymphocytes count had a borderline significance trend (see Additional file 4). The median IQR CD4 T lymphocytes count at last visit was 379 (255 - 541) cells/μl. Twenty-eight (17.3%) patients had CD4 T lymphocytes count less than 200 cells/μl.
In univariate LR analysis, odds of virological failure were determined. The study suggests that, being able to recall the diagnosis date, being on an efavirenz-containing regimen, having a pharmacy refill adherence of > 95% and having an undetectable viral load at baseline were significantly associated with lower likelihood of virological failure. On the contrary, having a baseline CD4 T lymphocytes count less than 200 cells/μl, being on a triomune, nevirapine, or stavudine regimen, having a higher body weight or body mass index (BMI), longer therapy duration, and having drug holidays were predictors of poor virological outcome (see Table 2). Adherence measured by the self-report methods VAS (>95%) and SHCS-AQ, appointment keeping (>80%) and pill count (>90%) (each time the optimum% cut-off was from Table 1) were not predictive of virological outcome. Older age, undetectable viral load at baseline, and VAS adherence were associated with lower likelihood of immunological failure, whereas having a baseline CD4 T lymphocytes count of less than 200 cells/μl was associated with immunological failure. According to the WHO definition of immunological failure, 17 of 162 (10.5%) patients had immunological failure at one year after recruitment.
Pharmacy refill was the only adherence measurement that was significantly associated with 12-month virological outcome (see Table 2 and Table 3). The proportion of patients with undetectable viral load increased with increasing pharmacy refill adherence, and was highest in patients with over 95% adherence (see Figure 2). Moreover, patients with detectable viral load had significantly lower pharmacy refill adherence with 91.9% (80 - 99.4%), as compared to 98.8% (91.1 – 100%) in patients with undetectable viral load (p = 0. 003). Of the 17 patients with immunological failure, 10 had a pharmacy refill adherence of > 95%.
The sensitivity results indicated that the pharmacy refill adherence has the highest AUC and best balance between sensitivity and specificity, as compared with other adherence measurements (see Table 1 and Additional file 5). At a cut-off of 95%, refill adherence covers the highest AUC in the ROC curve and significantly predicts virological failure. The proportion decreased in a dose response fashion in lower adherence cut-off groups (see Figure 2). Increasing adherence was associated with increased rates of undetectable viral load.
Multiple cross validation models to predict virological failure
Fifty independent runs of 10-fold MCV were executed, using different input sets and feature selection methods (see Table 4 and Additional file 6). MCV univariate models using a dichotomized adherence variable gave AUC and accuracy that were not much different than the univariate evaluation in Table 1 and Additional file 3, although sensitivity was consistently lower. A univariate logistic regression fitted on actual numeric (instead of optimized cut-off) pharmacy refill adherence yielded an AUC of 0.64 and accuracy of 68.73 (see Additional file 6). This was slightly better than with the optimal cut-off analysis shown in Table 1, but sensitivity was worse. A full LR model combining all variables in boost stepwise gave AUC of 0.64, which was higher than all the univariate adherence models, except the pharmacy refill adherence using continuous numeric values. The non-linear methods (decision trees and random forests) did not show improvement over the LR (see Table 4).
In multivariate analysis, having pharmacy refill adherence of < 95%, having a baseline CD4 T lymphocytes count < 200 cells/μl, being unable to recall the diagnosis date, and weight per 1 kg higher remained significant in predicting virological failure (see Table 3). Only pharmacy refill adherence and being able to recall diagnosis were significant predictors of immunological outcome.
Kaplan-Meier and Cox model for immunological recovery
Kaplan Meier survival and Cox regression analysis was performed with months as the time unit, in order to assess the time taken by patients to recover CD4 T lymphocytes count by at least 50 or 100 cells/μl (see Figure 3). Patients who had a pharmacy refill adherence of < 95% recovered 50 and 100 cells/μl in a median (95% confidence interval) of five (4 - 8) and eight (5 - 10) weeks respectively. Whereas, patients who were > 95% refill adherent recovered in 4.5 (4 - 6) and seven (6 - 9) weeks respectively. Although the median time to reach the increment was longer in the non-adherent group, this difference did not reach significance (log rank p = 0.15 and 0.23 for increment of 50 and 100 cells/μl respectively). However, when pharmacy refill adherence was combined with age, gender, and CD4 T lymphocytes count at baseline, a Cox regression analysis resulted in significant differences (Wald test p < 0.01).
The aims of this study were to evaluate the correlation and validation of adherence with virological outcome in RLS. In RLS, viral load monitoring, the preferred method for evaluating treatment outcome, is hindered by financial and technical constraints, and the widely-used indicators of response to therapy, as well as clinical and immunological outcome monitoring, are confounded by poor sensitivity. The best measurement of adherence, along with other affordable and easily-obtained variables, could be used in RLS as a proxy for virological failure. The best model using such variables to predict virological failure may be useful to flag patients for viral load measurements, as a cheaper alternative than universal viral load follow-up. This study was carried out as an exploratory study using convenient sampling, and can, therefore, serve to inform larger studies on pharmacy refill adherence as a predictor of virological treatment response.
Four measures of adherence used in a RLS were validated against a one-year virological response as a “gold standard”. Although it is widely agreed that adherence plays an important role in the ART outcome, there is no standardized measure of adherence agreed upon in RLS. Each adherent measurement method has inherent weaknesses in various settings.
There was a high proportion of patients with virological failure (34%) in the study, despite high self-reporting adherence of > 95% in up to 87% of patients. Factors significantly associated with virological failure at the one-year of follow-up were having pharmacy refill adherence of < 95%, having baseline CD4 T lymphocytes count < 200 cells/μl, being on triomune-, stavudine-, or nevirapine-containing regimen, not being able to recall HIV diagnosis date, a unit increase in body weight even after correcting for age, and a baseline viral load. The association between the uses of triomune may be related to the negative correlation of both stavudine and nevirapine individually, rather than the fixed-dose therapy itself. These two drug components are both independently associated with virological failure after one year of therapy.
Of the adherence measurements, only pharmacy refill adherence was positively correlated with the virological response in a dose response relationship per each increasing adherence rate stratified category. The dose response relationship between pharmacy refill adherence and virological outcome has also been observed in previous studies [10, 23, 33]. Increasing the virological failure threshold to a viral load > 1,000 copies/mL led to an increase of sensitivity and specificity, confirming previous observation . While this is to be expected, given that lack of adherence will result in lack of virus control, the more relevant clinical cut-off remains the lower cut-off of 400 copies/ml, which was more related to residual replication of partially-resistant virus under drug selective pressure. Similar factors predicted the immunological outcome. The correlation of pharmacy refill adherence and virological outcome has been shown previously . Moreover, pharmacy refill showed a promising applicability, given its high performance in multiple validation regression models, balanced sensitivity and specificity compared to self-reports, appointment keeping, or pill count methods. Superiority of pharmacy refill adherence over self-report has been shown in a number of studies in RRS [10, 33, 36] and RLS [23, 37]. Generally, lower sensitivity and higher specificity values were noted for adherence measurement to predict virological failure, as compared to predicting virological success. This indicates that adherence measurement is more likely to detect adherent persons who are likely to have a good virologic outcome, as compared to identifying failing patients. Combining all variables in a multiple cross validation-full logistic regression model did not improve much over the pharmacy refill univariate model. The non-linear models test for interactions among variables, and especially random forests perform better if the variables are interdependent. Since the linear regression model performed better than the non-linear models, this indicates that the variables analysed are behaving independently with respect to the outcome. Thus, the power of pharmacy refill to predict virological failure is independent of other patient variables.
Even though in this study the optimal cut-off for pharmacy refill was 95%, the dose response relationship shown in the comparison of virological outcome and grouped pharmacy refill adherence suggests that ARV regimens suppress viral load even at moderate adherence of > 70%. This may be related to current regimens that are more potent, and, thus, may sufficiently sustain complete viral suppression and avoid HIVDR, especially among patients initiating ART [7, 17, 38].
Pharmacy refill adherence measurement has a potential to be used in RLS [24, 39]. In most RLS undertaking special ART scale-up programs, patients receiving free ARV supplies are obliged to return to the same centre on a monthly basis to obtain their medication refills, which allows implementation of this method. Furthermore, where the pharmacies have a computerized database, the pharmacy refill adherence measurement becomes feasible, practical, and guaranteed .
There are some limitations to the application of pharmacy refill adherence. First is the assumption that the drugs refilled are always consumed. There is no way to tell if the patient sold, shared, or dumped the medications after a refill. For that matter, this method is a mere proxy for adherence. A second drawback is inability to detect or predict all patients with viral rebound, given the relatively-low sensitivity for predicting virological failure (around 60%). Moreover, with this method, it is not possible to discriminate patterns of adherence, such as treatment interruption, which may be more risky for non-nucleoside reverse transcriptase inhibitor resistance than a run of occasionally missed doses [6, 40]. An additional drawback is when an individual can acquire medication refills from multiple pharmacies outside the regular clinic. However, in most RLS, patients have to return to the same clinic for their monthly ARV refills , and, thus, pharmacy refill is a feasible adherence measurement.
In this study, self-report measures did not predict virological outcome. Self-reported adherence has not been consistently validated in RLS . In some studies that evaluated self-report, it was found to be less sensitive [37, 41]. The unreliable nature of self-report stems from participants’ beliefs and fears about how the information provided to clinicians or adherence counsellors would be used. Thus, accuracy of self-reported adherence may be undermined if patients fear that reporting non-adherence could have negative consequences in their healthcare provisions, which are more relevant in RLS. In addition, clinic-based pill counts could not predict virologic outcome. Generally, clinic-based pill counts performed poorly in other studies, also . Unannounced pill counts conducted in homes have been closely associated with virological suppression [7, 43], the development of HIVDR , and progression to AIDS , but are less feasible. This study found no association between appointment keeping and viral load. Authors who found a better correlation  suggested that monitoring of clinic attendance may be an objective and effective measure, and could be a useful adjunct to an adherence measure, such as pill counting in RLS. Human resource constraints in these settings do not allow pill counts or other time-consuming measures to be taken. Therefore, monitoring clinic attendance and acting on missed appointments may be an effective proxy measure .
Furthermore, in a Cox regression model, after accounting for age, gender, and immunological status, the groups of patients having pharmacy refill > 95% were more likely to recover immunologically in a relatively-shorter period than the non-adherent group. The cross validation after fitting the adherence in logistic regression indicate that adherence estimates combined with other variables performs better in predicting the virological outcome than adherence alone. These results indicated that a model combining only adherence and immunological response performs equally well, even better than using adherence alone. This model is important in routine clinical practice because immunological outcome is easily available, whereas other variables may not be so easily and routinely available to use for predicting therapy response.
The study findings are limited by a small sample size obtained by convenient sampling of individuals who were initiating or continuing ART within the three months period of recruitment at the study site.
Of the adherence methods investigated in this study, pharmacy refill method had the best performance in predicting virological failure. The performance improved when pharmacy refill adherence was combined with immunological response. The study found, this combination to have a reasonable sensitivity (around 70%) and specificity (around 60%) to predict virological failure. This combination could be useful to flag patients at risk for virological failure. It appeared to be more reliable than immunological and clinical response alone and was cost-saving. Thus, it had the potential for a wider application in ART follow-up in RLS. When combined with even more variables, such as treatment and demographic characteristics in the full logistic regression model, the prediction still improved, however this may become less straightforward and may, thus, lose applicability in RLS where doctors have very little time to devote to individual patients.
Akaike information criterion
Acquired immunodeficiency syndrome
Area under the curve
Cluster of differentiation 4
Care and Treatment Centre
Human immunodeficiency virus
HIV drug resistance
Medication event monitoring system
Multiple cross validation
receiver operating characteristic
Swiss HIV Cohort study-adherence questionnaire
Visual analog scale.
Chaiwarith R, Wachirakaphan C, Kotarathititum W, Praparatanaphan J, Sirisanthana T, Supparatpinyo K: Sensitivity and specificity of using CD4+ measurement and clinical evaluation to determine antiretroviral treatment failure in Thailand. Int J Infect Dis. 2007, 11: 413-416. 10.1016/j.ijid.2006.11.003.
Rawizza HE, Chaplin B, Meloni ST, Eisen G, Rao T, Sankalé J-L, Dieng-Sarr A, Agbaji O, Onwujekwe DI, Gashau W, Nkado R, Ekong E, Okonkwo P, Murphy RL, Kanki PJ: Immunologic criteria are poor predictors of virologic outcome: implications for HIV treatment monitoring in resource-limited settings. Clin Infect Dis. 2011, 53: 1283-1290. 10.1093/cid/cir729.
De Luca A, Marazzi MC, Mancinelli S, Ceffa S, Altan AMD, Buonomo E, Prosperi MCF, Pedruzzi B, Noorjehan AM, Scarcella P, Liotta G, Palombi L: Prognostic value of virological and immunological responses after 6 months of antiretroviral treatment in adults with HIV-1 infection in sub-Saharan Africa. J Acquir Immune Defic Syndr. 2012, 59: 236-244. 10.1097/QAI.0b013e31824276e9.
Nachega JB, Hislop M, Dowdy DW, Chaisson RE, Regensberg L, Maartens G: Adherence to nonnucleoside reverse transcriptase inhibitor-based HIV therapy and virologic outcomes. Ann Intern Med. 2007, 146: 564-573. 10.7326/0003-4819-146-8-200704170-00007.
Chi BH, Cantrell RA, Zulu I, Mulenga LB, Levy JW, Tambatamba BC, Reid S, Mwango A, Mwinga A, Bulterys M, Saag MS, Stringer JSA: Adherence to first-line antiretroviral therapy affects non-virologic outcomes among patients on treatment for more than 12 months in Lusaka, Zambia. Int J Epidemiol. 2009, 38: 746-756. 10.1093/ije/dyp004.
Oyugi JH, Byakika-Tusiime J, Ragland K, Laeyendecker O, Mugerwa R, Kityo C, Mugyenyi P, Quinn TC, Bangsberg DR: Treatment interruptions predict resistance in HIV-positive individuals purchasing fixed-dose combination antiretroviral therapy in Kampala, Uganda. AIDS. 2007, 21: 965-971. 10.1097/QAD.0b013e32802e6bfa.
Bangsberg DR: Preventing HIV antiretroviral resistance through better monitoring of treatment adherence. J Infect Dis. 2008, 197 (Suppl 3): S272-S278.
Kiwuwa-Muyingo S, Walker AS, Oja H, Levin J, Miiro G, Katabira E, Kityo C, Hakim J, Todd J: The impact of first year adherence to antiretroviral therapy on long-term clinical and immunological outcomes in the DART trial in Uganda and Zimbabwe. Trop Med Int Health. 2012, 17: 584-594. 10.1111/j.1365-3156.2012.02974.x.
Nachega JB, Hislop M, Dowdy DW, Lo M, Omer SB, Regensberg L, Chaisson RE, Maartens G: Adherence to highly active antiretroviral therapy assessed by pharmacy claims predicts survival in HIV-infected South African adults. J Acquir Immune Defic Syndr. 2006, 43: 78-84. 10.1097/01.qai.0000225015.43266.46.
Grossberg R, Zhang Y, Gross R: A time-to-prescription-refill measure of antiretroviral adherence predicted changes in viral load in HIV. J Clin Epidemiol. 2004, 57: 1107-1110. 10.1016/j.jclinepi.2004.04.002.
Nachega JB, Mills EJ, Schechter M: Antiretroviral therapy adherence and retention in care in middle-income and low-income countries: current status of knowledge and research priorities. Curr Opin HIV AIDS. 2010, 5: 70-77. 10.1097/COH.0b013e328333ad61.
Bärnighausen T, Chaiyachati K, Chimbindi N, Peoples A, Haberer J, Newell M-L: Interventions to increase antiretroviral adherence in sub-Saharan Africa: a systematic review of evaluation studies. Lancet Infect Dis. 2011, 11: 942-951. 10.1016/S1473-3099(11)70181-5.
Grossberg R, Gross R: Use of pharmacy refill data as a measure of antiretroviral adherence. Curr HIV/AIDS Rep. 2007, 4: 187-191. 10.1007/s11904-007-0027-4.
Chesney MA, Ickovics JR, Chambers DB: Self-reported adherence to antiretroviral medications among participants in HIV clinical trials: The AACTG Adherence Instruments. Patient Care Committee & Adherence Working Group of the Outcomes Committee of the Adult AIDS Clinical Trials Group (AACTG). AIDS Care. 2000, 12: 255-266. 10.1080/09540120050042891.
Berg KM, Arnsten JH: Practical and conceptual challenges in measuring antiretroviral adherence. J Acquir Immune Defic Syndr. 2006, 43 (Suppl 1): S79-S87.
Simoni JM, Kurth AE, Pearson CR, Pantalone DW, Merrill JO, Frick PA: Self-report measures of antiretroviral therapy adherence: A review with recommendations for HIV research and clinical management. AIDS Behav. 2006, 10: 227-245. 10.1007/s10461-006-9078-6.
Wagner GJ: Utility of self-reported antiretroviral adherence: Comment on Simoni et al. (2006). AIDS Behav. 2006, 10: 247-248. 10.1007/s10461-006-9122-6.
Santa Helena ET, Nemes MI, Eluf-Neto J: Development and validation of a multidimensional questionnaire assessing non-adherence to medicines. Rev Saude Publica. 2008, 42: 764-767. 10.1590/S0034-89102008000400025.
Mills EJ, Nachega JB, Buchan I, Orbinski J, Attaran A, Singh S, Rachlis B, Wu P, Cooper C, Thabane L, Wilson K, Guyatt GH, Bangsberg DR: Adherence to antiretroviral therapy in sub-Saharan Africa and North America: a meta-analysis. JAMA. 2006, 296: 679-690. 10.1001/jama.296.6.679.
Watt MH, Maman S, Golin CE, Earp JA, Eng E, Bangdiwala SI, Jacobson M: Factors associated with self-reported adherence to antiretroviral therapy in a Tanzanian setting. AIDS Care. 2010, 22: 381-389. 10.1080/09540120903193708.
Bisson GP, Rowh A, Weinstein R, Gaolathe T, Frank I, Gross R: Antiretroviral failure despite high levels of adherence: discordant adherence-response relationship in Botswana. J Acquir Immune Defic Syndr. 2008, 49: 107-110. 10.1097/QAI.0b013e3181820141.
WHO HIV Drug resistance report 2012. [http://apps.who.int/iris/bitstream/10665/75183/1/9789241503938_eng.pdf]
Goldman JD, Cantrell RA, Mulenga LB, Tambatamba BC, Reid SE, Levy JW, Limbada M, Taylor A, Saag MS, Vermund SH, Stringer JSA, Chi BH: Simple adherence assessments to predict virologic failure among HIV-infected adults with discordant immunologic and clinical responses to antiretroviral therapy. AIDS Res Hum Retroviruses. 2008, 24: 1031-1035. 10.1089/aid.2008.0035.
McMahon JH, Jordan MR, Kelley K, Bertagnolio S, Hong SY, Wanke CA, Lewin SR, Elliott JH: Pharmacy adherence measures to assess adherence to antiretroviral therapy: review of the literature and implications for treatment monitoring. Clin Infect Dis. 2011, 52: 493-506. 10.1093/cid/ciq167.
Hess LM, Raebel MA, Conner DA, Malone DC: Measurement of adherence in pharmacy administrative databases: a proposal for standard definitions and preferred measures. Ann Pharmacother. 2006, 40: 1280-1288. 10.1345/aph.1H018.
Deschamps AE, De Geest S, Vandamme A-M, Bobbaers H, Peetermans WE, Van Wijngaerden E: Diagnostic value of different adherence measures using electronic monitoring and virologic failure as reference standards. AIDS Patient Care STDS. 2008, 22: 735-743. 10.1089/apc.2007.0229.
Acri TL, TenHave TR, Chapman JC, Bogner HR, Gross R: Lack of association between retrospectively collected pharmacy refill data and electronic drug monitoring of antiretroviral adherence. AIDS Behav. 2010, 14: 748-754. 10.1007/s10461-008-9502-1.
El-Khatib Z, Ekstrom AM, Ledwaba J, Mohapi L, Laher F, Karstaedt A, Charalambous S, Petzold M, Katzenstein D, Morris L, Ekström AM: Viremia and drug resistance among HIV-1 patients on antiretroviral treatment: a cross-sectional study in Soweto, South Africa. AIDS. 2010, 24: 1679-1687. 10.1097/QAD.0b013e32833a097b.
Mee P, Fielding KL, Charalambous S, Churchyard GJ, Grant AD: Evaluation of the WHO criteria for antiretroviral treatment failure among adults in South Africa. AIDS. 2008, 22: 1971-1977. 10.1097/QAD.0b013e32830e4cd8.
Glass TR, De Geest S, Weber R, Vernazza PL, Rickenbach M, Furrer H, Bernasconi E, Cavassini M, Hirschel B, Battegay M, Bucher HC: Correlates of self-reported nonadherence to antiretroviral therapy in HIV-infected patients: the Swiss HIV Cohort Study. J Acquir Immune Defic Syndr. 2006, 41: 385-392. 10.1097/01.qai.0000186371.95301.52.
R Development Core Team: R: A Language and Environment for Statistical Computing. 2012
Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH: The WEKA data mining software. ACM SIGKDD Explor Newsl. 2009, 11: 10-10.1145/1656274.1656278.
Fairley CK, Permana A, Read TRH: Long-term utility of measuring adherence by self-report compared with pharmacy record in a routine clinic setting. HIV Med. 2005, 6: 366-369. 10.1111/j.1468-1293.2005.00322.x.
Kerr SJ, Avihingsanon A, Putcharoen O, Chetchotisakd P, Layton M, Ubolyam S, Ruxrungtham K, Cooper DA, Phanuphak P, Duncombe C: Assessing adherence in Thai patients taking combination antiretroviral therapy. Int J STD AIDS. 2012, 23: 160-165. 10.1258/ijsa.2009.009152.
Gross R, Yip B, Lo Re V, Wood E, Alexander CS, Harrigan PR, Bangsberg DR, Montaner JSG, Hogg RS: A simple, dynamic measure of antiretroviral therapy adherence predicts failure to maintain HIV-1 suppression. J Infect Dis. 2006, 194: 1108-1114. 10.1086/507680.
Gutierrez EB, Sartori AMC, Schmidt AL, Piloto BM, França BB, de Oliveira AS, Pouza AR, Moreno RV, de Melo PC, de Almeida Ribeiro MCS: Measuring adherence to antiretroviral treatment: the role of pharmacy records of drug withdrawals. AIDS Behav. 2012, 16: 1482-1490. 10.1007/s10461-012-0168-3.
Rougemont M, Stoll BE, Elia N, Ngang P: Antiretroviral treatment adherence and its determinants in Sub-Saharan Africa: a prospective study at Yaounde Central Hospital Cameroon. AIDS Res Ther. 2009, 6: 21-10.1186/1742-6405-6-21.
Bangsberg DR: Less than 95% adherence to nonnucleoside reverse-transcriptase inhibitor therapy can lead to viral suppression. Clin Infect Dis. 2006, 43: 939-941. 10.1086/507526.
Chalker J, Wagner A, Tomson G, Laing R, Johnson K, Wahlström R, Ross-Degnan D: Urgent need for coordination in adopting standardized antiretroviral adherence performance indicators. J Acquir Immune Defic Syndr. 2010, 53: 159-161. 10.1097/QAI.0b013e3181befa12.
Luebbert J, Tweya H, Phiri S, Chaweza T, Mwafilaso J, Hosseinipour MC, Ramroth H, Schnitzler P, Neuhann F: Virological failure and drug resistance in patients on antiretroviral therapy after treatment interruption in Lilongwe, Malawi. Clin Infect Dis. 2012, 55: 441-448. 10.1093/cid/cis438.
Balikuddembe R, Kayiwa J, Musoke D, Ntale M, Baveewo S, Waako P, Obua C: Plasma drug level validates self-reported adherence but predicts limited specificity for nonadherence to antiretroviral therapy. ISRN Pharmacol. 2012, 2012: 274978-
Minzi OMS, Naazneen AS: Validation of self-report and hospital pill count using unannounced home pill count as methods for determination of adherence to antiretroviral therapy. Tanzan J Health Res. 2008, 10: 84-88.
Kalichman SC, Amaral CM, Stearns H, White D, Flanagan J, Pope H, Cherry C, Cain D, Eaton L, Kalichman MO: Adherence to antiretroviral therapy assessed by unannounced pill counts conducted by telephone. J Gen Intern Med. 2007, 22: 1003-1006. 10.1007/s11606-007-0171-y.
Bangsberg DR, Perry S, Charlebois ED, Clark RA, Roberston M, Zolopa AR, Moss A: Non-adherence to highly active antiretroviral therapy predicts progression to AIDS. AIDS. 2001, 15: 1181-1183. 10.1097/00002030-200106150-00015.
Kunutsor S, Walley J, Katabira E, Muchuro S, Balidawa H, Namagala E, Ikoona E: Clinic Attendance for Medication Refills and Medication Adherence amongst an Antiretroviral Treatment Cohort in Uganda: A Prospective Study. AIDS Res Treat. 2010, 2010: 872396-
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/14/1035/prepub
RZS acknowledges the support of the Belgian Technical Cooperation (BTC) for funding part of this study. A-MV was supported by the Fonds voor Wetenschappelijk Onderzoek Vlaanderen (grant K8.012.12 N). MP was funded by MRC grant MR/K006665/1 in association with University of Manchester’s Health Research Centre (HeRC). FM, SA, JV, RJC, EF and EVW were funded by their respective institutions. Part of this study was funded by Sida, under Muhimbili University of Health and Allied Sciences (MUHAS) small grants scheme and by the Fonds voor Wetenschappelijk Onderzoek Vlaanderen (grant G.0.692.14 N). We are thankful to the patients for cooperating and participating in the study and to the CTC manager, doctors, nurses, councillors, tracking officers, and auxiliary support staff who made the meetings with patients much easier.
The authors declare that they have no competing interests.
RZS designed the study, conducted the interviews, performed data analysis, and wrote the first draft of the manuscript; FM, SA, and RJC supervised laboratory work and manuscript review; MP and JV assisted with statistical analysis and manuscript review; EFL, EVW, and AMV supervised the overall study implementation and manuscript development process. All authors read and approved the final manuscript.
Electronic supplementary material
Additional file 2: Values of sensitivity, specificity, ROC curve area under the curve (AUC) by adherence assessment method at different cut-off points predicting virological failure (viral load > 1,000 copies/ml) and immunological failure, using adherence measurement. Values of sensitivity, specificity and AUC by predicting virological failure or immunological failure using adherence measurements. (DOCX 24 KB)
Additional file 3: Values of sensitivity, specificity, ROC curve AUC by adherence assessment method at different cut-off points predicting virological failure (viral load < 400 copies/ml) at individual time points of adherence follow-up measurement. Description of data: Four time points 1 - 4 and the overall adherence outcome is shown per adherence measurement method. Key: SHCS-AQ = Swiss HIV Cohort Study Adherence Questionnaire; VAS = Visual Analog Scale; AUC = Area Under the Curve; ROC = Receiver Operating Characteristic; Sen = Sensitivity; Spe = Specificity; CI = confidence interval; PPV = positive predictive value; NPV = negative predictive value; DR = detection rate; DP = detection prevalence. AUC noted as “NA” indicated a situation when one of the two groups (adherent or non-adherent) was too small to calculate sensitivity and specificity. (XLSX 21 KB)
Additional file 5: ROC curve using sensitivity and specificity for predicting virological failure at different pharmacy refill adherence threshold cut-off points between 50 and 100. Note an increase in the area under the curve with increasing cut-off until optimum at 95%. ROC curve for predicting virological failure at different pharmacy refill adherence cut-off points. (EPS 5 KB)
Additional file 6: Performance of and adherence univariate models versus full variable-null model using over 10-fold cross validation, repeated 50 times to predict virological failure (>400 copies/ml). Good-ness of fit of predicting virological failure using univariate versus full variable model. (DOCX 19 KB)
About this article
Cite this article
Sangeda, R.Z., Mosha, F., Prosperi, M. et al. Pharmacy refill adherence outperforms self-reported methods in predicting HIV therapy outcome in resource-limited settings. BMC Public Health 14, 1035 (2014). https://doi.org/10.1186/1471-2458-14-1035
- Adherence validation
- Questionnaires, Sensitivity and specificity, HIV-1