Recruitment and retention of participants in a pragmatic randomized intervention trial at three community health clinics: Results and lessons learned

Background

Obesity and hypertension and their associated health complications disproportionately affect communities of color and people of lower socioeconomic status. Recruitment and retention of these populations in research trials, and retention in weight loss trials has been an ongoing challenge.

Methods

Be Fit, Be Well was a pragmatic randomized weight loss and hypertension management trial of patients attending one of three community health centers in Boston, Massachusetts. Participants were asked to complete follow-up assessments every 6-months for two years. We describe challenges encountered and strategies implemented to recruit and retain trial participants over the 24-month intervention. We also identify baseline participant characteristics associated with retention status. Retention strategies included financial incentives, contact between assessment visits, building relationships with health center primary care providers (PCPs) and staff, and putting participant convenience first.

Results

Active refusal rates were low with 130 of 2,631 patients refusing participation (4.9%). Of 474 eligible persons completing telephone screening, 365 (77.0%) completed their baseline visit and were randomized into the study. The study population was predominantly non-Hispanic Black (71.2%), female (68.5%) and reported annual household income of less than $35,000 (70.1%). Recruitment strategies included use of passive approval of potential participants by PCPs, use of part-time staff, and outsourcing calls to a call center. A total of 314 (86.0%) people completed the 24-month visit. Retention levels varied across study visits and intervention condition. Most participants completed three or more visits (69.6%), with 205 (56.2%) completing all four. At 24-months, lower retention was observed for males and the intervention condition. Retention strategies included building strong relationships with clinic staff, flexibility in overcoming participant barriers through use of taxi vouchers, night and weekend appointments, and keeping participants engaged via newsletters and social gatherings.

Conclusion

We were able to retain 86.0% of participants at 24-months. Recruitment and retention of high percentages of racial/ethnic minorities and lower income samples is possible with planning, coordination with a trusted community setting and staff (e.g. community health centers and RAs), adaptability and building strong relationships.

Trial registration

Clinicaltrials.gov Identifier: NCT00661817

Recruitment and retention of racial/ethnic minorities, particularly non-Hispanic Blacks (Blacks) and Hispanics, into clinical trials continues to be a challenge. A distrust of the medical community due to historical injustices such as the Tuskegee Syphilis Study is an oft cited reason for low participation [1–4]. Yet other factors may lead to exclusion from efficacy trials, such as a lack of trials occurring in racial/ethnic minority serving institutions, limited awareness of ongoing trials, work/family and insurance barriers, trial referral practices of PCPs, and the prevalence of co-morbid conditions [5–7]. Regardless of the challenges, inclusion of these groups in clinical research is important to ensure that results are relevant, and generalizable [8]. Many of the health conditions under study in trials disproportionately affect communities of color, and thus their participation is critical [9].

Weight loss interventions have generally been less successful among Blacks. For example, the Trial of Non-pharmacologic Interventions in the Elderly found that Blacks lost on average 2.9 kilograms less than Whites over the course of the intervention [10]. Similar disparities were seen in the lifestyle treatment arm of the Diabetes Prevention Program. At 24-months weight loss was lower among black men (−5.9%) and women (−3.2%) than White men (−7.2%) and women (−5.9%) or Hispanic men (−7.8%) and women (−7.1%) [11]. Yet, rates of obesity and obesity-related conditions are very high among Black and Hispanic populations. According to data from the National Health and Nutrition Examination Study, 44.1% of Blacks were obese compared with 38.7% of Hispanics and 32.4% of non-Hispanic Whites (Whites) [12]. Rates of obesity-related conditions such as diabetes, high blood pressure, and asthma are higher among Blacks and Hispanics than among Whites [13]. There is clearly a need for interventions that are effective at weight loss and weight maintenance among Black and Hispanic populations. In order to design and implement interventions that will work for this population, we need well-designed trials with high representation of these at-risk groups. Thus increasing participation and effectiveness among Black and Hispanic populations in weight loss studies is of great importance.

Long-term weight loss trials have traditionally had low to moderate retention levels. For example, the Louisiana Obese Subjects Study reported retention rates at 24-months of 51% in their intervention group and 46% in usual care [14]. While several studies such as the Trials of Hypertension Prevention have had high levels of retention [15], a review of 16 weight loss trials showed that nine had relatively high (31–64%) levels of loss to follow-up [16]. Study interpretability declines with increasing levels of loss to follow-up in intervention trials [16–18]. Therefore, it behooves us to understand strategies that yield high retention levels.

This report describes the recruitment and retention of participants in a pragmatic randomized weight loss and hypertension management trial, ‘Be Fit, Be Well’ [19, 20]. The purposes of this paper are to: (1) describe recruitment and retention activities; (2) discuss challenges encountered and strategies used to overcome them; (3) analyze study retention and patterns of retention and its association with baseline participant characteristics, and (4) describe lessons learned and directions for future research.

Be Fit, Be Well (BFBW) was a randomized, controlled, intervention trial funded by the National Heart, Lung and Blood Institute under a U01 grant mechanism. Participants were recruited from three community health clinics in the Boston neighborhoods of Roxbury and Dorchester. Primary eligibility criteria were age 21 or older, a body mass index (BMI) between 30 and 50 kg/m², and diagnosed hypertension with prescribed anti-hypertensive medication.

Exclusion criteria have been described elsewhere but, included: a history of myocardial infarction, stroke, or an atherosclerotic cardiovascular disease procedure ≤ 6 months before study entry, prior or planned bariatric surgery, recent or planned pregnancy, use of Food and Drug Administration (FDA) approved weight loss medications or medications known to increase weight [21, 22].

At baseline all participants had their weight, height and blood pressure measured, received the National Heart Lung and Blood Institute’s Aim for a Healthy Weight brochure, and completed an audio computer assisted self-interview survey which collected data on demographics, health behaviors, mental health and neighborhood characteristics. Participants randomly assigned to the usual care group received no further educational materials or instruction throughout the study period, but returned for follow-up assessments every 6 months. The study intervention is described elsewhere in greater detail [22]. Briefly, intervention participants received materials for behavior self-monitoring including a pedometer, 18 attempted calls from a community health educator (monthly in year 1, bi-monthly in year 2), were invited to attend optional monthly group sessions and received information about community resources. In concert with their assigned community health worker (CHW), intervention participants set behavioral goals, tracked them via their selected modality (website or interactive voice response system), received tailored feedback on their progress and had the opportunity to modify their goals every 13 weeks. All study procedures, materials and modifications were approved by the Institutional Review Board at Harvard School of Public Health (protocol #14455).

All participants were asked for their preferred language and study materials were available in English and Spanish. A BFBW study logo was on all printed materials, including letterhead, envelopes and other mailings. The study had a dedicated email address and two dedicated landlines. All participants were asked to complete follow-up data collection visits every six months for two years. At follow-up visits participants had their weight and blood pressure measured and completed a computer-assisted survey. Baseline and follow-up visits were conducted by trained, bilingual RAs (RAs) and were generally less than one hour in length. As there were two RAs and three health clinics, each RA had primary responsibility for one health clinic and shared responsibility for participants at the third. A part-time project director oversaw the work of the RAs and coordinated with health clinic staff and medical directors.

Recruitment protocol

We used primarily active recruitment methods [23]. Potentially eligible patients were identified largely through a two-step process of preliminary medical record searches using the primary eligibility criteria of obesity and hypertension, followed by more thorough medical record review. At the two clinics with electronic medical records (EMR), clinic staff provided lists of patients with diagnosed hypertension (using ICD-9 codes) and obesity (using measured height and weight). The clinic without an EMR was able to provide a list of people with ICD-9 codes for hypertension using their claims billing system. Trained RAs performed medical record reviews on each identified patient to assess study eligibility.

Lists of potentially eligible patients were sent to their PCP for confirmation and permission to contact. RAs contacted approved patients by mail. Letters explained the study, told patients to expect a call in the next five days, and were printed on clinic letterhead with the signature of the patient’s PCP. RAs called and administered a brief eligibility questionnaire and answered any questions. Calls were tracked using call logs. Up to 10 phone calls were allocated per potential participant. Eligible and interested patients that could be reached were scheduled for a baseline study visit at their clinic.

Appointments were monitored using Google calendar using a unique ID for each potential participant. The project director generated weekly recruitment status reports which aggregated data on the number of medical records reviewed, potentially eligible patients identified, names submitted to PCPs for approval, intro letters mailed, eligibility calls made/completed, number of appointments scheduled and number of participants randomized. Study staff discussed these weekly reports and developed improvement strategies throughout the recruitment period as challenges arose. We aimed to enroll 360 participants over the course of one year. We anticipated enrollment to be approximately 66% black and 30% Hispanic.

Retention protocol

Participants scheduled their next follow-up visit appointment at the end of each completed follow-up visit (i.e. 6-month visit scheduled at end of baseline visit). If a visit was missed, the RAs called by phone and/or sent letters and emails to schedule their next appointment. Each participant appointment was identified using their participant identification number and appointment outcomes were updated daily via Google calendar. Visit outcomes were classified as: completed, rescheduled, or no show. Distinction was made between appointments rescheduled for study or personnel reasons versus appointments that were rescheduled by the participant’s request. Completed follow-up appointments were classified as within or out of window. The visit window was calculated as six weeks before and after the visit date as calculated based on their baseline visit.

At the baseline visit we asked for home, work and cell phone numbers of each participant and the name and number of two people who we could contact in the event that we could not reach them at any of the other numbers provided [24]. Reminder letters were sent two weeks before each scheduled visit and reminder calls were made one week prior and the day of each visit. In the event of a missed appointment, we attempted to reschedule the visit via phone, email (where available) and/or mailed letter.

We offered flexible visit scheduling; both in time (evening and Saturday visits) and location (could come to any of the three health clinics). We mailed birthday cards to all participants in their preferred language. Participants received incentives to compensate them for their time at baseline and follow-up visits [25]. Upon completion of baseline, 6-, 12, and 18-month visits participants received a $50 grocery card and at 24-months they received a $75 grocery card. Intervention participants also received a scale at their 12-month visit and a blood pressure monitor at 18-months to support their behavioral self-monitoring goals.

For each participant and each follow-up visit we calculated expected visit date, date visit window opened, date visit window closed, logged whether the visit was completed, visit completion date, and made notes about the participant’s status with the study and our attempts at contact. This information was organized into spreadsheets that were updated on a weekly basis, cross-referenced with the Google calendar and reviewed at weekly team meetings to assure accuracy. Participants that were significantly out of window for a given follow-up visit were classified as missed for that visit and contacted for completion of their next visit. Retention percentages were monitored weekly and new strategies developed and implemented as needed. IRB amendments were submitted Participants did not directly participate in strategy development, but they were informally queried about reasons for participation and barriers. The information they provided did inform our strategies.

Statistical analyses

Logistic regression was used to generate multivariable adjusted p-values comparing the probability of retention at each follow-up visit (6-, 12-, 18- or 24-months) across categories of baseline characteristics including randomization group, age, gender, race, income, primary language, depression score, education, work status and self-rated health. All p-values are two-sided with an alpha level 0.05. All analyses were conducted using SAS 9.2.

Participant recruitment

We reviewed over 2500 medical records, mailed 860 introductory letters, and made over 4100 eligibility screening call attempts (Figure 1). Less than 5% of potentially eligible participants actively refused study enrollment (130 of 2631). Few people that completed our eligibility screening call (33 of 507 (6.5%)) were deemed ineligible; 365 (77%) of those eligible and with scheduled baseline visit appointments completed their baseline visit and were randomized. Of the 109 people that did not complete their baseline visit, 48 (44.0%) were male, 28 (25.7%) had a Spanish surname and mean age was 51.5 years (data not shown). We enrolled our last participant on April 30, 2009 for a total recruitment period of 15 months.

CONSORT flow diagram of health center selection, participant screening, eligibility and retention: Be Fit, Be Well Study.

Full size image

Figure 2 displays the number screening calls completed and participants recruited per week and includes when strategies were implemented. Most strategies were implemented during the first six months of the recruitment period, making it difficult to disentangle their effects. Passive PCP approval’s implementation one month after the start of recruitment increased the pool of potentially eligible participants we could contact. Of our other strategies, hiring additional short-term part-time staff and use of the call center greatly increased our capacity complete screening calls and randomize participants. One month after implementing the call center, our weekly number of screening calls completed was nearly 7 times greater than the week they started (Figure 1). However, the reply card mailing, PCP and patient self-referral, and newspaper ads were not incrementally effective (Table 1). PCPs rarely used the clinic drop boxes to submit patients for the study. Only one person that responded to our newspaper ads went on to be randomized.

Weekly number of screening calls completed and randomized participants by implementation date of recruitment strategies.

Full size image

Participant retention

Table 2 summarizes retention strategies employed and their results. The most useful activities in terms of response from participants, and our administrative assessment, were regular communication with clinic staff and PCPs, newsletters and social gatherings for participants and extra incentives to participants with previous missed visits. CHWs were also helpful in scheduling appointments with active intervention participants.

Completion percentages varied by visit, with the highest retention observed at 24- (86.0%) and 6-months (74.5%) and the lowest at 12- (69.3%) and 18-months (67.1%) (Table 3). At 6-months lower retention was observed for those with income less than $10,000 (64.2%) compared to those with income of $35,000 or more (80.7%). Retention was consistently, but non-significantly, higher at Health Center A as compared to the other two sites. At 12 months, retention at Health Center B (62.1%) was significantly lower than that observed Health Center A (75.3%; p=0.01). There was also higher retention of the usual care group compared to those in the intervention which was statistically significant at 12- (75.1% vs. 63.3%; p=.00039) 18-(71.9% vs. 62.2%; p=0.01) and 24-months (89.7% vs. 82.2%; p=0.03), but not at 6 months (p=0.19). Persons under age 50 at baseline had lower retention than older persons at 6-, 12- and 18-month visits. At 18 months, women (71.2%) had higher retention than men (58.3%; p=0.04). A similar pattern for gender was seen at 24-months. At the 18-month visit, we observed lower retention of Hispanic participants (47.9%) as compared to Blacks (70.4%; p=0.03). The other characteristics examined including educational attainment, depression score, self-rated health, and language were not associated with retention at any study visit.

Recruitment and retention of participants, especially low income and minority members, in longitudinal weight loss trials has not proven an easy task in previous studies. Using an adaptive and comprehensive approach in BFBW, we were able to recruit a predominantly low-income minority population from local community health centers and have 86.0% complete their 24-month follow-up visit. This combination of reasonably high recruitment and high retention in this population group is unusual and we attribute our success to multiple factors [26]. First, we developed and maintained partnership relationships with health center staff, PCPs and participants which kept them connected to the study. Second, we created systems for timely tracking of recruitment and retention and monitored activities closely. Third, we used the data to generate new ideas and strategies and implemented them quickly. Specific strategies such as use of flexible part-time staff, outsourcing recruitment calls to a call center, passive PCP approval, and a willingness to meet the needs of individual participants in a variety of ways (taxi vouchers, evening and weekend appointments, allowing children at appointments, offering home visits) were most effective in achieving strong recruitment and retention numbers. Our results are consistent with Davis et al. (2002), which found that things such as providing meaningful incentives, using a participant-tracking database, maintaining between-assessment contacts, and establishing a project identity were associated with increased retention [27].

Recruitment took three months longer than our initial goal of one year. We anticipated that our study population would be approximately 66% Black and 30% Hispanic. Actual enrollment was 71.2% Black and 13.2% Hispanic. Enrollment of Blacks was as expected, but we recruited fewer Hispanics than anticipated. This is, at least in part, attributable to two administrative delays. First, Health Center C had the largest Hispanic patient population and we began enrolling participants there nearly three months after beginning at health center C, due to administrative delays. Second, we had numerous delays in translation and production of Spanish screening and study materials and did not start enrolling Spanish speakers until mid-summer 2008.

With 86.0% retention at 24-months, we exceeded our goal of 80.0%. However, retention varied by study visit. Retention was high at 6-months (74.5%), declined at 12- (69.3%) and 18-months (65.5%) and was highest at 24-months. It is unlikely that seasonal variation could explain differences in retention by visit. Participants were recruited over a 15-month period and there were participants completing each visit in every season. Additionally, 69.6% completed at least three and the majority (56.2%) attended every visit (data not shown). Differences by gender, age, race, income and randomization group may be due to a variety of factors including a predominately female staff, competing life activities for younger participants including balancing children and work, and travel. For example, it is possible that the focus of the study on weight loss and hypertension was less salient for younger participants for whom chronic disease consequences may still have been many years away [28]. We found that our Hispanic participants were more likely to leave Boston for long periods of time to visit family either in other parts of the United States or in Latin America. This may explain their lower retention at 18-months. Intervention participants were less likely to complete the 24-month visit than those in usual care. This may be due to dissatisfaction with the intervention (and its results on their weight) or fatigue (intervention participants received more contacts than usual care).

There have been few long-term weight loss interventions among low-income minority participants conducted in the United States. In a 2011 review of 38 obesity management interventions commissioned by the Agency for Healthcare Research and Quality, only four studies were conducted in predominately Black or Hispanic populations [29]. Of those studies, there were only two studies that went as long as 18-months with retention of 89.2% and 63.2% respectively. Our 24-month retention result is a major accomplishment given our setting in community health centers, and the fact that this was a pragmatic trial—we had no run in period, and had minimal behavioral inclusion criteria.

As has been noted in other studies, relationships with PCPs, administrators and staff at the health centers were key in recruiting participants [30] and maintaining high levels of retention. Nicholson et al. (2011) found that administrative support in the form of desk space, access to patient schedules and medical records, and coordination with PCPs and nurses was important for participant retention [31]. Coordination with health centers allowed us to be flexible in our strategies for recruitment and retention. For example, we could not have met with participants before or after their primary care visits if we didn’t have access to the scheduling systems, and we couldn’t have gotten some of the updates to contact information without working with the health center administrators. While there were some challenges, ultimately, having follow-up visits at the health centers was an important advantage in retaining participants and also linking the intervention to patient’s primary care.

Many attempts were needed to reach some participants, both during recruitment and retention. What worked best was spreading the calls out over time. We might call a person 10 times in the first two weeks of April, not get a response and try 10 more calls in mid-May. This strategy was effective for some participants because of out of town travel or illness. However, for others, namely the 50 people lost to follow-up, this was not an effective strategy. Our findings are consistent with work by Cotter et al. (2005) that showed that limiting contact leads to lower retention and the additional costs associated with more contact are cost-effective [32]. Kleschinsky et al. (2009) found that in a study of repeat driving under the influence (DUI) offenders that increasing calls up to 40 calls per person yielded additional completions [33]. We found that a high number of call attempts spread out over a period of months, specifically including multiple weeks in which no calls were placed, did lead to additional visit completions. We did this with approval of our IRB and with careful consideration of participant burden and ethical concerns.

It is important to put our results into context. BFBW took place during a recession. We began with an already low-income population and many faced additional hardships, including loss of jobs and homelessness during the study [34, 35]. An important part of our success was the recognition that our participants led complicated lives and our study was a small, but hopefully, important part of it. We approached our participants with cultural and emotional sensitivity.

A limitation of this investigation is that we lack specific controlled information about the distinct effect of specific individual strategies implemented on recruitment and retention. Additionally we lack information on factors that have been associated with recruitment and retention in other studies such as marital status, number and age of children or other dependents, and perceived stress [8, 31, 36]. Future studies to more comprehensively analyze such factors might include variables such as health literacy/numeracy, social capital, satisfaction with the clinic, comorbidity, and contacts with the PCP.

BFBW successfully recruited and retained predominantly minority and low-income participants at three community health centers in Boston. We learned that trials of this nature take significant and directed resources to maximize recruitment and retention. Careful tracking of recruitment strategies, with flexibility to add new strategies as required, is often needed to meet goals. Our success, along with those of several other studies shows that conducting longitudinal research in this setting is not impossible even during trying economic and political times. These are populations that disproportionately bear the burden of obesity and chronic disease and are in need of research attention. With dedication, planning, stakeholder engagement, flexibility and sensitivity, high recruitment and retention levels are achievable.

BFBW:

Be Fit, Be Well

BMI:

Body mass index

FDA:

Food and drug administration

CHW:

Community health worker

RA:

Research assistant

EMR:

Electronic medical record

PCP:

Primary care provider

DUI:

Driving under the influence

The authors wish to thank the study RAs (Noemi Lopez, Danielle Rivas, Argelis Rivera and Andrea Vasquez), community health workers (Erica Levine, Charlette Steed and Andrea Vasquez) and project director (Julie Goldman). We wish to thank the health center staff, administrators and PCPs for their support and assistance. We also wish to thank the participants of this study for their dedication.

This work was supported in part by grant U01-HL087071 from the National Heart, Lung, and Blood Institute and grants K22CA126992 (Dr. Bennett), K05CA124415-04 (Dr. Emmons), 5T32CA009001-36 (Dr. Warner), and 5R25GM055353-14 (Dr. Warner).

The opinions expressed are those of the authors and do not necessarily reflect those of the National Heart Lung and Blood Institute, the National Cancer Institute or the National Institutes of Health.

Affiliations

1. Department of Epidemiology, Harvard School of Public Health, 677 Huntington Ave, Boston, MA, USA

   • Erica T Warner

2. Center for Community-Based Research, Division of Population Sciences, Dana Farber Cancer Institute, 375 Longwood Ave, Boston, MA, USA

   • Erica T Warner
   • , Karen M Emmons
   •  & Gary G Bennett

3. Channing Division of Network Medicine, Brigham and Women’s Hospital, 181 Longwood Ave, Boston, MA, USA

   • Erica T Warner
   •  & Bernard Rosner

4. Division of Cancer Control and Population Sciences, National Cancer Institute, 6130 Executive Boulevard, Executive Plaza North, Rockville, MD, USA

   • Russell E Glasgow

5. Department of Society, Human Development, and Health, Harvard School of Public Health, 677 Huntington Ave, Boston, MA, USA

   • Karen M Emmons

6. Department of Psychology and Neuroscience, Duke University, Box 90086, Durham, NC, USA

   • Gary G Bennett

7. Duke Global Health Institute, Duke University, Box 90086, Durham, NC, USA

   • Gary G Bennett
   •  & Sandy Askew

8. Department of Biostatistics, Harvard School of Public Health, 677 Huntington Ave., Boston, MA, USA

   • Bernard Rosner

9. Department of Medicine, Harvard Medical School, 25 Shattuck St., Boston, MA, USA

   • Bernard Rosner

10. Department of Surgery, Alvin J. Siteman Cancer Center, Washington University School of Medicine, Saint Louis, MO, USA

    • Graham A Colditz

Corresponding author

Correspondence to Erica T Warner.

Competing interests

The authors have no competing interests to report.

Authors’ contributions

GGB, ETW, KME, BAR, and GAC had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: GGB, ETW, REG, KME, and GAC. Acquisition of data: GGB, ETW, and GAC. Analysis and interpretation of data: REG, SA, KME, BAR, and GAC. Drafting of the manuscript: ETW, GGB, REG, KME and GAC. Critical revision of the manuscript for important intellectual content: GGB, ETW, REG, KME, BAR, and GAC. Statistical analysis: ETW and REG. Obtained funding: GGB, REG, KME, and GAC. Administrative, technical, and material support: GGB, ETW, SA, KME, and GAC. Study supervision: GGB and KME. All authors read and approved the final manuscript.

Below are the links to the authors’ original submitted files for images.

Reprints and Permissions