This article has Open Peer Review reports available.
Utility of telephone survey methods in population-based health studies of older adults: an example from the Alberta Older Adult Health Behavior (ALERT) study
© Vallance et al.; licensee BioMed Central Ltd. 2014
Received: 22 November 2013
Accepted: 6 May 2014
Published: 22 May 2014
Random digit dialing is often used in public health research initiatives to accrue and establish a study sample; however few studies have fully described the utility of this approach. The primary objective of this paper was to describe the implementation and utility of using random digit dialing and Computer Assisted Telephone Interviewing (CATI) for sampling, recruitment and data collection in a large population-based study of older adults [Alberta Older Adult Health Behavior (ALERT) study].
Using random digit dialing, older adults (> = 55 years) completed health behavior and outcome and demographic measures via CATI. After completing the CATI, participants were invited to receive a step pedometer and waist circumference tape measure via mail to gather objectively derived ambulatory activity and waist circumference assessments.
Overall, 36,000 telephone numbers were called of which 7,013 were deemed eligible for the study. Of those, 4,913 (70.1%) refused to participate in the study and 804 (11.4%) participants were not included due to a variety of call dispositions (e.g., difficult to reach, full quota for region). A total of 1,296 participants completed telephone interviews (18.5% of those eligible and 3.6% of all individuals approached). Overall, 22.8% of households did not have an age 55+ resident and 13.6% of individuals refused to participate, Average age was 66.5 years, and 43% were male. A total of 1,081 participants (83.4%) also submitted self-measured ambulatory activity (i.e., via step pedometer) and anthropometric data (i.e., waist circumference). With the exception of income (18.7%), the rate of missing data for demographics, health behaviors, and health measures was minimal (<1%).
Older adults are willing to participate in telephone-based health surveys when randomly contacted. Researchers can use this information to evaluate the feasibility and the logistics of planned studies using a similar population and study design.
In Canada, few efforts have been established to monitor and track the health behaviors (and related health outcomes) of older adults. Older adults have some of the poorest health behavior indices which are often associated with an increased risk of chronic disease [1, 2]. Monitoring these behaviors, and gaining a deeper understanding of the factors associated with various health behaviors in this population can: a) provide relevant information for stakeholders upon which evidence-based interventions and programs can be developed and implemented, and b) facilitate the health and well-being of older adults.
Researchers have utilized a variety of sampling/recruitment strategies including social media, health registry/database/chart review, print media, and public service announcements . In the context of older adults, several studies have used random digit dialing and subsequent Computer-Assisted Telephone Interview (CATI) methods to establish a study sample. However, details regarding the recruitment procedures, outcomes (e.g., data completeness, reasons for refusal, number of call attempts) and call dispositions (e.g., average number of call attempts, reasons for refusal) are limited [4–7]. Providing these details (and challenges) related to characteristics of telephone recruitment using random digit dialing and CATI survey methods for study recruitment may assist other researchers in their planning and execution of older-adult health-related initiatives utilizing these methods.
We recently completed participant recruitment and data collection for the population-based Alberta Older Adult Health Behavior (ALERT) study. The primary objective of this paper was to describe the implementation and utility of using random digit dialing and CATI for sampling, recruitment and data collection in the ALERT study. Secondary objectives were to: a) examine the utility of using objective measures in telephone-based population studies, and b) describe follow-up completion and data completeness rates using these approaches.
The ALERT study was a large population-based study of the health behaviours and objective outcomes of older adults (≥55 years of age) conducted using random digit dialing and CATI. All subject recruitment and data collection were conducted through a centralized research unit - the Population Research Laboratory (PRL) at the University of Alberta (Edmonton, Alberta, Canada). The PRL is a member of the Association of Academic Survey Research Organizations (AASRO) and assists academic researchers and policy makers in designing and implementing applied social science research projects. The PRL specializes in the gathering, analysis, and presentation of data about demographic, social and public issues and has particular expertise in administering CATI surveys. The PRL maintains a complement of highly skilled telephone interviewers and supervisors who are thoroughly trained in respondent selection procedures, questionnaire instructions, and neutral probing.
The ALERT Study was approved by the University of Alberta’s Health Research Ethics Board as well as the Athabasca University Research Ethics Board. Prior to initiating the survey portion of the telephone interview, participants were assured that their participation is voluntary and that their answers are confidential and stored in a secured database and used only for study purposes. Participants were also informed they had have the right to end the interview at any time.
Random digit dialing
A random digit dialing approach was used to ensure that households in each health zone had an equal chance to be contacted whether or not their household was listed in a telephone directory. The sampling frame of telephone numbers updated to 2012 was based on landlines and excluded businesses, unlisted cellular phone exchanges, and government exchanges. The typical structure of a telephone number in the province of Alberta (and across Canada) is: Area code (NPA)-Prefix/Exchange (NXX)- Phone Number (XXXX). Since 1986, the PRL has maintained a set of eight digit working banks (NPA-NAA-XX) for the province of Alberta. The PRL works with a vendor to update the list regularly to add in new area codes (3 digits), new exchanges (3 digits assigned to certain geographies in the province) and observed densities of residential and business telephone numbers assigned at the two-digit level. The last two digits in the phone number are then randomly seeded and generated in the range 00–99 and appended with replacement to the working bank file based on the number of telephone numbers projected to reach the sample size desired. Within the 10-digit numbers generated the PRL purges the duplicate phone numbers. However, even at the two-digit level of generation, a substantial number of not-in service numbers and business numbers are still generated. The sample of numbers is sent out to a vendor for pre-dialing (screening) to filter out these numbers before data collection projects that require interviewer dialing. The main strength of random digit dialing is the ability to sample all telephone households in the province, not simply households on a particular directory or commercial list .
Data collection consisted of two stages. In stage I, data were collected via CATI. Physical activity was assessed using the leisure score index (LSI) of the Godin Leisure-Time Exercise Questionnaire (GLTEQ) ; Sitting (i.e., sedentary behavior) was assessed using a newly developed tool designed to assess total past year sitting behaviors and time in the following domains; sleeping and napping, transportation, work and volunteering, and light leisure and relaxing. It was designed for use in epidemiologic studies, and can be administered by interviewer of by written questionnaire. ; Health-related quality of life was measured using the four-week version of the RAND-12 Health Status Inventory (RAND-12) ; Depressive symptoms were assessed using the validated Patient Health Questionnaire-9 (PHQ-9) ; Health literacy was assessed using three health literacy screening questions developed and validated by Chew and colleagues ; Medication adherence was assessed using the Moriskey 8-item Medication Adherence Questionnaire . Chronic health conditions, demographic information (e.g., age, income, education, employment), and health information (e.g., anthropometrics, smoking status) were assessed via self-report.
After confirming eligibility, and prior to the start of the interview, participants were asked if they were willing to wear a step pedometer for three days, and measure their waist size. In stage II of data collection, the PRL mailed a step pedometer (StepsCount SC-01, Deep River, Ontario, Canada), and a 60” waist circumference tape measure (Almedic, Stevens, Ontario, Canada) to each participant who consented and completed the survey interview, and agreed to receive the pedometer and tape measure. Participants were asked to start tracking their daily steps over a three-day period the following morning after receiving the package in the mail. Participants were encouraged to wear the pedometer on one weekend day. The package included detailed instructions on how to use the pedometer, how to record the number of steps for each day on a log sheet, and how to measure their waist circumference. To obtain the most precise waist measurement, participants were asked to complete the measure three times and perform all three measures at one time. After one week of sending the pedometer, the interviewers contacted the participants and recorded their three-day step count total and waist circumference. Participants were also able to e-mail their data to the project coordinator at the PRL. When there was no option to leave a message, the PRL sent a letter to the participants reminding them to submit their step count and waist circumference information.
Prior to data collection, interviewers received training from the study principal investigator (JV) on survey content and administration of the survey procedures. Pre-testing commenced on a sample of 20 respondents and was initiated via random digit dialing to test the adequacy of the survey instrument including routing patterns, question wording and flow, adequacy of transitional statements, and to estimate the length of the interviews. In addition, the PRL assessed the mail out process and follow-up to pretest participants to obtain step counts and waist measurement information.
The target population consisted of people who at the time of the survey were living in Alberta and could be contacted by direct phone dialing. Individuals were eligible to participate if they were at least 55 years of age, were able to walk unassisted (without the aid of a wheelchair or walking aid), community dwelling (not living in an aged care setting) and were able to complete a computer-assisted telephone survey in English. Sampling by gender, equally weighted across the zones based on estimates of population maintained by the province of Alberta, was conducted.
The PRL collected the main data using a CATI system. The PRL research coordinator formatted the ALERT instrument electronically and loaded a sample database of randomly generated telephone numbers onto the CATI system, which allocates telephone numbers to the interviewing stations after which the interviewer dials the number. To eliminate effects of seasonality on health and health behaviors (i.e., extreme hot and cold), the interviewing began on September 28, 2012 and was completed on November 20, 2012. This sampling time frame was chosen as all weather patterns could likely be experienced within these months in Alberta (e.g., ranging from warm summer heat in late September to potential snowfall in late November). All data collection was conducted from the centralized CATI facilities of the PRL at the University of Alberta. To obtain a greater response rate, telephone numbers allocated by the CATI system were dialed at least eight times or until a final disposition (such as a completed interview or a final refusal) was obtained. Telephone numbers were called on various days over a one-week period and at different times of day. After making contact, interviewers identified themselves, verified the telephone number, and then asked the screening questions for selecting the respondent. Before administering the questionnaire, the interviewer reviewed consent and confidentiality with the participants.
The PRL team took steps to ensure high quality data including initial extensive training, regular reporting and communication with the principal investigator (JV) on issues and concerns, periodic information updates and refresher trainings for the interviewers, continuous review of the data for missing values and inconsistencies, checking call dispositions, and conducting back-up interviewing by supervisors when required. Data were accumulated on a regular basis using the features of the CATI system and then imported into the IBM SPSS for Windows (version 20.0). The data were checked regularly. The data cleaning process included wild code, discrepant value, and consistency checks. Each variable was reviewed and evaluated. Missing data was defined as any data point where the participant indicated either 1) no response, or 2) don’t know/not sure.
Number of call attempts for completed interviews and for all call records for main study
Number of attempts
All call records
Final call dispositions for main study
Final disposition number
Final outcome of call attempts
Not in service
Permanent no contact3
Claimed to be on the National Do Not Call List4
Ineligible - No 55+ in household
Ineligible – Unable to walk unassisted
Will call lab
Cell phone, do not want to be disturbed
8 or more call attempts
Total telephone allocated
Participation rate (%)
Demographic details of study participants
Men N = 555
Women N = 741
66.5 ± 8.2
66.2 ± 8.1
66.6 ± 8.3
27.2 ± 5.1
27.2 ± 4.2
27.1 ± 5.7
Some high School
Not working, not looking
Retired but working
Of the 1,296 participants who completed the CATI survey, only 19 (1%) of people declined the pedometer and waist circumference follow-up for various reasons (e.g., going on extended vacation, emergency surgery). Eighty-five percent (n = 1,081) of the 1,277 participants who received the pedometer and waist circumference tape measure submitted their data to the PRL. Overall, 100% of participants provided step data on all three days and three sequential waist circumference measurements (i.e., there was no missing data). Of the sample, 625 participants voluntarily submitted this data via e-mail. The remaining 454 participants provided their data when they received a follow-up phone call from the interviewer. Total cost to complete the study (e.g., staff, pre test, data collection, postage, follow-up, supplies) was $115.90 (CDN) per participant.
Our study is one of the first to fully report on the implementation and outcomes associated with CATI survey with follow-up objectively measured data. We found that among older adults, random digit dialing is a feasible recruitment strategy in obtaining a population-based sample of older adults. Although the overall yield per telephone call is relatively low (3.6%), participation rates were relatively high among eligible individuals despite the inclusion of objectively measured data (19.3%). Indeed, of the final sample, 85% of participants (n = 1,081) completed and returned the objective assessments (step pedometer and waist circumference). Of these participants, data quality was excellent as 100% of participants provided step data on all three days and three waist circumference measurements.
Our participation rate of 19.3% among eligible individual is higher than previous reports where the average random digit dialing telephone survey response rates in the U.S. was approximately 9% (out of 761 surveys) . While we were able to achieve a high participation rate, the general decline in participation rates for random digit dial telephone surveys has been well documented [17, 18]. Refusing participation can be explained by a range of factors including time constraints (decreasing volunteerism), a general cynicism about research, the research may present as a burden, or disinterest in the survey topic . Moreover, households are increasingly subject to telemarketing solicitation and automated telephone calls (i.e., ‘robocalls’), which may reduce people’s willingness to participate in legitimate surveys. However, in our study of older adults, call dispositions did not indicate an overwhelming sense of cynicism, burden, or lack of interest in the topic. Indeed, our data cannot take into account individuals screening or blocking unknown numbers using telephone technologies such as caller ID or call blocking (or a general refusal to answer the phone). While a high participation rate may lead to a higher degree of validity in the resulting data, research has demonstrated that low participation rates do not necessarily compromise the validity of the data .
Our data also suggests that the majority of participants in the study were contacted after the second attempt and virtually no benefits in recruitment were gained after 6 attempts. However, it should be noted that the overall yield per call is relatively low at 3.6%. We enrolled 96% of respondents with 85% of calls and 88% with only 72% of calls. Future researchers would experience cost savings by potentially stopping recruitment early. Although our data suggest that random digit dialing is highly feasible in relatively large populations, the utility of this approach in smaller and more restricted populations would be questionable. This has important implications for researchers who are conducting population-based studies where a large number of restrictions are included or where highly select and specific participants are required. Although one would have anticipated that inclusion of objective measures would have significantly hindered patient enrollment, our data suggest that older adults are willing and able to complete objective assessments beyond the telephone survey context. It is possible that participants perceived the step pedometer and waist circumference tape measure as an incentive to participating. Best practice guidelines for improving respondent cooperation for telephone surveys suggest that issuing incentives (monetory or non-monetary) has a strong impact on telephone survey participation rates [21, 22]. Regardless, the use of health behavior assessment tools that have consumer appeal and usability appear to provide a mutual benefit for both the researcher and the participant.
Despite the strengths of our study including random digit dialing procedures, the use of trained interviewers, a geographically representative sample of older adults, and the use of objective physical activity ambulatory behavior (pedometers) and health assessments (waist circumference), there are several limitations that deserve mention. First, there is a growing trend towards the use of mobile phones. The generalizability of our sample, and studies that use similar methods, may become more limited as these individuals (who have no landline) are not represented in this survey design. As a result, researchers will need to develop additional techniques to reach these individuals. Second, the data provided may be biased towards socially acceptable response as opposed to the true feelings of the individual. There is no information regarding study non-responders to compare with study participants. However, using Statistics Canada 2012 census data we compared our sample to the general population on education and employment status. Our sample was representative of the population based on these variables as census data indicated 50.8% of Albertans over the age of 55 years had completed college or university , 44.9% were currently employed part or full time , and 59% were classified as overweight/obese (compared to 63.6% in our sample) . We were unable to compare the samples on income as our study asked participants for combined household income as opposed to census data that collects individual respondent income. Selection bias may be occurring where only those individuals who were interested in health may have participated in the study.
Our study suggests that older adults are interested in and willing to participate in telephone-based health surveys using random digit dialing and CATI. Further, our study suggests that these methods can be expanded to include not only traditional telephone-based collection of information but also objective measures of health behaviors and anthropometrics including walking and waist circumference via post. The majority of completed interviews (60.3%) were achieved on the first two call attempts. Researchers utilizing similar approaches to data collection may consider making only two call attempts for possible cost savings given the diminished benefits of making >2 call attempts. Based on comparison to national census data, older adults that participated in our study were fairly representative of the general population of older adults living in the province of Alberta. Researchers seeking to conduct studies of a similar design with older adults can expect an ~19% response rate of those eligible and ~4% of all random numbers called.
Random digit dialing and CATI are indeed common data collection tools in market research. However, with respect to the health context, random digit dialing and CATI use among samples of older adults is limited. Our manuscript provides readers with information on logistics and feasibility of recruiting older adults into similarly designed studies. Our data provides information in which readers can use to plan and budget for related research initiatives. Our study suggests that it is feasible to ask older adults to complete objective measures (i.e., pedometer steps and waist circumference) via post, after the telephone interview is complete. The older adults in our sample certainly demonstrated they were able to conduct these two assessments without any difficulties. Future research should continue to engage older adults in telephone-based health surveys with the addition of more objective and valid measures of health and health behaviour, such as the use of accelerometers to objectively measure physical activity and sedentary behavior.
This study was supported by an Alberta Innovates – Health Solutions Establishment Grant awarded to Dr. Jeff Vallance. Dr. Jeff Vallance is supported by the Canada Research Chairs program and a Population Health Investigator Award from Alberta Innovates – Health Solutions. Dr. Dean Eurich is supported by a Population Health Investigator Award from Alberta Innovates – Health Solutions and a New Investigator Award from the Canadian Institutes for Health Research.
- Artaud F, Dugravot A, Sabia S, Singh-Manoux A, Tzourio C, Elbaz A: Unhealthy behaviours and disability in older adults: Three-city Dijon cohort study. BMJ. 2013, 347: 1-15.View ArticleGoogle Scholar
- Shankar A, McMunn A, Steptoe A: Health-related behaviors in older adults relationships with socioeconomic status. Am J Prev Med. 2010, 38: 39-46. 10.1016/j.amepre.2009.08.026.View ArticlePubMedGoogle Scholar
- UyBico SJ, Pavel S, Gross CP: Recruiting vulnerable populations into research: a systematic review of recruitment interventions. J Gen Intern Med. 2007, 22: 852-863. 10.1007/s11606-007-0126-3.View ArticlePubMedPubMed CentralGoogle Scholar
- Ho SY, Lam TH, Chung SF, Lam TP: Cross-sectional and prospective associations between passive smoking and respiratory symptoms at the workplace. Ann Epidemiol. 2007, 17: 126-131. 10.1016/j.annepidem.2006.06.010.View ArticlePubMedGoogle Scholar
- Levine MA, Xu S, Gaebel K, Brazier N, Bedard M, Brazil K, Lohfeld L, MacLeod SM: Self-reported use of natural health products: a cross-sectional telephone survey in older Ontarians. Am J Geriatr Pharmacother. 2009, 7: 383-392. 10.1016/j.amjopharm.2009.12.003.View ArticlePubMedGoogle Scholar
- Wilson-Genderson M, Pruchno R: Effects of neighborhood violence and perceptions of neighborhood safety on depressive symptoms of older adults. Soc Sci Med. 2013, 85: 43-49.View ArticlePubMedGoogle Scholar
- Zhao G, Ford ES, Li C, Balluz LS: Physical activity in U.S. older adults with diabetes mellitus: prevalence and correlates of meeting physical activity recommendations. J Am Geriatr Soc. 2011, 59: 132-137. 10.1111/j.1532-5415.2010.03236.x.View ArticlePubMedGoogle Scholar
- Brick JM: Random-digit dialing. Encyclopedia of survey research methods. Edited by: Lavrakas PJ. 2008, Thousand Oaks, CA: SageGoogle Scholar
- Godin G, Shephard RJ: A simple method to assess exercise behavior in the community. Can J Appl Sport Sci. 1985, 10: 141-146.PubMedGoogle Scholar
- Lynch BM, Friedenreich CM, Csizmadi I: Adult perception of sedentary behavior self-report assessment: Cognitive interviewing the SIT-Q. J Sci Med Sport. 2013, 15: 295-Google Scholar
- Ware J, Kosinski M, Keller SD: A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996, 34: 220-233. 10.1097/00005650-199603000-00003.View ArticlePubMedGoogle Scholar
- Kroenke K, Spitzer RL, Williams JB: The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001, 16: 606-613. 10.1046/j.1525-1497.2001.016009606.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Chew LD, Bradley KA, Boyko EJ: Brief questions to identify patients with inadequate health literacy. Fam Med. 2004, 36: 588-594.PubMedGoogle Scholar
- Morisky DE, Green LW, Levine DM: Concurrent and predictive validity of a self-reported measure of medication adherence. Med Care. 1986, 24: 67-74. 10.1097/00005650-198601000-00007.View ArticlePubMedGoogle Scholar
- Brick JM, Kalton G: Handling missing data in survey research. Stat Method Med Res. 1996, 5: 215-238. 10.1177/096228029600500302.View ArticleGoogle Scholar
- CMOR tracking system - cooperation, refusal and response rates. [http://www.mra-net.org/resources/respondent_cooperation/coop_rates_avg.cfm]
- Curtin R, Presser S, Singer E: Changes in telephone survey nonresponse over the past quarter century. Pub Op Quarterly. 2005, 69: 87-98. 10.1093/poq/nfi002.View ArticleGoogle Scholar
- Keeter S, Kennedy C, Dimock M, Best J, Craighill P: Gauging the impact of growing nonresponse on estimates from a national RDD telephone survey. Pub Op Quarterly. 2006, 70: 759-779. 10.1093/poq/nfl035.View ArticleGoogle Scholar
- Galea S, Tracy M: Participation rates in epidemiologic studies. Ann Epidemiol. 2007, 17: 643-653. 10.1016/j.annepidem.2007.03.013.View ArticlePubMedGoogle Scholar
- Keeter S, Miller C, Kohut A, Groves RM, Presser S: Consequences of re-ducing non-response in a national telephone survey. Pub Op Quarterly. 2000, 64: 125-148. 10.1086/317759.View ArticleGoogle Scholar
- Canada Public Works and Government Services: Best practices in public opinion research. 2007, Ottawa, ON: Improving respondent cooperation for telephone surveysGoogle Scholar
- Singer E, Van Hoewyk N, Gebler T, Raghunathan T, McGonagle K: The effect of incentives on response rates in interviewer-mediated surveys. J Off Stat. 1999, 15: 217-230.Google Scholar
- Table 282–0070: Labour force survey estimates (LFS), wages of employees by type of work, National Occupational Classification for Statistics (NOC-S), sex and age group, annual (current dollars unless otherwise noted), CANSIM (database). [http://www5.statcan.gc.ca/cansim/a26]
- Table 282–0004: Labour force survey estimates (LFS), by educational attainment, sex and age group, annual (persons unless otherwise noted), CANSIM (database). http://www5.statcan.gc.ca/cansim/a26]
- Table 105–0501: Health indicator profile, annual estimates, by age group and sex, Canada, provinces, territories, health regions (2012 boundaries) and peer groups, occasional, CANSIM (database). 2012, [http://www5.statcan.gc.ca/cansim/a47]Google Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/14/486/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.