Skip to main content
  • Research article
  • Open access
  • Published:

Non-response bias in physical activity trend estimates

Abstract

Background

Increases in reported leisure time physical activity (PA) and obesity have been observed in several countries. One hypothesis for these apparently contradictory trends is differential bias in estimates over time. The purpose of this short report is to examine the potential impact of changes in response rates over time on the prevalence of adequate PA in Canadian adults.

Methods

Participants were recruited in representative national telephone surveys of PA from 1995-2007. Differences in PA prevalence estimates between participants and those hard to reach were assessed using Student's t tests adjusted for multiple comparisons.

Results

The number of telephone calls required to reach and speak with someone in the household increased over time, as did the percentage of selected participants who initially refused during the first interview attempt. A higher prevalence of adequate PA was observed with 5-9 attempts to reach anyone in the household in 1999-2002, but this was not significant after adjustment for multiple comparisons.

Conclusion

No significant impact on PA trend estimates was observed due to differential non response rates. It is important for health policy makers to understand potential biases and how these may affect secular trends in all aspects of the energy balance equation.

Peer Review reports

Background

The Global Strategy for Diet and Physical Activity [1] emerged in response to the recognition of the growing global burden posed by non communicable disease (NCD). Many national health surveys monitor NCD risk factors using self-reported data to assess trends in these risk factors over time. [2–5].

The physical activity levels of Canadian have been tracked since 1981 [6]. These data have been essential in setting national physical activity goals, identifying which groups are less active than others and determining whether differences in physical activity levels between groups have been increasing, decreasing or remaining stable over time. In Canada, population levels of leisure-time physical activity (PA) based on self-reports have increased over a twenty-year period. [6] However, during this same period the prevalence of overweight and obesity has also increased at alarming rates. This co-occurrence of increases in reported leisure time PA and obesity has also been observed in Finland [7, 8], Spain [9], Australia [10, 11] and elsewhere. These apparently contradictory trends may be explained by changes in dietary patterns, the limitation of physical activity measures based on only leisure-time rather than total activity across all domains, or of other changing patterns. Other explanations include differences in response bias, such as changes in social desirability bias regarding active lifestyles, or decreasing response rates (for example due to the increase in telemarketing) leading to differential non-response bias in estimates over time [12]. Differentials in the magnitude of bias over time may lead to erroneous conclusions regarding the changes in the monitored prevalence estimates and relationships with other variables of interest.

The purpose of this short report is to examine potential non-response biases (e.g., in terms of number of attempted contacts) in self-reported data introduced during the recruitment to random digit dial surveys and how these may influence trend estimates, using the example of PA.

Methods

Study participants 18 years and older were recruited to the Physical Activity Monitor (PAM) a nationally representative Canadian survey of adults launched in 1995, repeated in 1997 and run continuously since 1998 employing a monthly sample. The sample is selected in two steps. Households are selected using random digit dialing and then the individual with the nearest birth date to the interview date is selected. During the recruitment process, the following information is recorded 1) number of attempts made to call the household, 2) number of times someone answered in trying to reach the selected individual, 3) the number of contacts with the selected individual and completion status (initial agreement, refusal converted to participation). Verbal informed consent was obtained according to the procedures of the Ethics Committee, York University.

An adapted version of the Minnesota Leisure Time Physical Activity questions was administered from 1995-2002 and then again in 2007. The PA score was calculated by summing the product of the number of times each type of PA was performed in the previous 12 months, the duration in minutes and the intensity of the activity expressed in METs [13]. 'Adequate' PA was defined as 5294 kilojoules (1260 MET-minutes) per week and equivalent to at least 60 minutes of moderate-intensity activity daily [14]. Differences in the prevalence of adequate PA by attempts, answers, contacts, and completion status were tested across and within survey periods using Student's t tests (SPSS Complex Samples, version15) and adjusting significance levels for multiple comparisons by Holm's procedure.

Results

In total, 26,496 individuals were involved in the study between 1995 and 2007. Overall, the number of calls required to reach someone in the selected households increased over the period (Table 1). In 1995-1998, 10 or more such calls were required to reach 13.2% of respondents and this percentage increased to 23.3% by 2007. Similarly, the number of times that the calls were answered by someone in the household (to select individual respondents and then contact them) increased, with 3 or more calls being required in 41.6% of households in 1995-1998 compared to 59.8% in 2007. After selecting the respondent, at least three contacts with that respondent were made to complete the interview (e.g., due to inconvenient time of call) in 15.3% of households in 2007, which was higher than the percentages required in 1999-2002. Agreeing to participate in the survey during the first attempt to interview the respondent decreased over the twelve year span, with initial refusals roughly twice as high in 2007 compared to the early and middle years.

Table 1 Distribution of adults responding to Canadian Physical Activity Monitor by stages in participant recruitment, 1995-2007

Prevalence of 'adequate PA' was unrelated to the number of attempts made to contact someone in the household in 1995-1998 and 2007. In 1999-2002 only, a higher prevalence of adequate PA was observed with 5-9 attempts than in fewer attempts (and compared to 5-9 attempts in 1995-98), but this was not significant after adjusting for multiple comparisons (Table 2). At all time periods, adequate PA was unrelated to the number of times someone in the household answered the telephone, the number of times the selected individual was contacted, and whether that individual initially agreed or was converted from a refusal to a participant.

Table 2 Adequate physical activity (PA) among Canadian adults by stages in participant recruitment, 1995-2007

Discussion

This study found no evidence of a differential bias in levels of adequate PA according to the various stages at which non-response could occur during recruitment. Therefore, there is no support for the hypothesis that increase in physical activity over time were related to changes in the underlying representivity of the samples. As a result, differences in non-response rates do not explain the apparent contradictory trends of increases in both sufficient leisure time physical activity and obesity over time. More investigation is required to understand changes in the multiple factors influencing the energy balance equation. In particular, a better understanding is required of changes in sedentary behaviours, total energy expenditure through physical activity, dietary patterns, and total energy intake, and how these changes relate to the rising rates of overweight and obesity.

One limitation of this study is that comparisons were made between individuals who were easier and harder to recruit, but was unable to make comparisons with those not recruited at all due to a lack of information external to the surveys. External data has been used to quantify non-response bias in other studies with equivocal results. Using proxy reports from more cooperative or readily accessible family members for PA and other key variables, Vink [15] determined that although the non responding family members generally had less favorable estimates than those responding, the impact on estimates was small. In the Netherlands, the prevalence of healthy lifestyle behaviours was over estimated, but there was no bias in associations [16]. In contrast, Macera et al [17] found limited impact on estimates of key variables comparing an initial face-to-face contact (as opposed to telephone contact herein) to subsequent follow-up. This was also the case in a different Canadian study using the same PA measure as in the current study [18]. However, trend estimates may also be influenced by response bias including changes in question order, inclusion of sensitive subject areas in surveys, and changes in social desirability (for example, higher over-reporting of PA as low levels of participation become more widely recognized as a health issue). Accounting for bias is important to those involved in health policy to ensure that any observed changes are indeed due to secular trends and not an artifact of temporal differences in bias.

Conclusion

Although it was not possible to quantify differential impacts of response bias over time, the data do not demonstrate a significant effect on trend estimates due to differential response rates over the twelve year span.

References

  1. World Health Assembly 57.17: Global strategy on diet and physical activity. 2004, Geneva, World Health Organisation, Accessed April 27, 2009, [http://apps.who.int/gb/ebwha/pdf_files/WHA57/A57_R17-en.pdf]

    Google Scholar 

  2. Armstrong T, Bonita R: Capacity building for an integrated noncommunicable disease risk factor surveillance system in developing countries. Ethn Dis. 2003, 13 (2 Suppl 2): S13-18.

    PubMed  Google Scholar 

  3. Puska P, Helasoja V, Prättälä R, Kasmel A, Klumbiene J: Health behaviour in Estonia, Finland and Lithuania 1994-1998. Standardized comparison. Eur J Public Health. 2003, 13 (1): 11-17. 10.1093/eurpub/13.1.11.

    Article  PubMed  Google Scholar 

  4. Ramsey F, Ussery-Hall A, Garcia D, McDonald G, Easton A, Kambon M, Balluz L, Garvin W, Vigeant J: Prevalence of selected risk behaviors and chronic diseases--Behavioral Risk Factor Surveillance System (BRFSS), 39 steps communities, United States, 2005. MMWR Surveill. 2008, 57 (11): 1-20.

    Google Scholar 

  5. Tanuseputro P, Manuel DG, Leung M, Nguyen K, Johansen H: Risk factors for cardiovascular disease in Canada. Can J Cardiol. 2003, 19 (11): 1249-1259.

    PubMed  Google Scholar 

  6. Craig CL, Russell SJ, Cameron C, Bauman A: Twenty Year Trends of Physical Activity among Canadian Adults. Cdn J Public Health. 2004, 95 (1): 59-63.

    Google Scholar 

  7. Barengo NC, Nissinen A, Tuomilehto J, Pekkarinen H: Twenty-five-year trends in physical activity of 30- to 59-year-old populations in eastern Finland. Med Sci Sports Exerc. 2002, 34 (8): 1302-1307. 10.1097/00005768-200208000-00011.

    Article  PubMed  Google Scholar 

  8. Silventoinen K, Sans S, Tolonen H, Monterde D, Kuulasmaa K, Kesteloot H, Tuomilehto J: Trends in obesity and energy supply in the WHO MONICA Project. Int J Obes Relat Metab Disord. 2004, 28 (5): 710-718. 10.1038/sj.ijo.0802614.

    Article  CAS  PubMed  Google Scholar 

  9. Rodríguez Artalejo F, López García E, Gutiérrez-Fisac JL, Banegas Banegas JR, Lafuente Urdinguio PJ, Domínguez Rojas V: Changes in the prevalence of overweight and obesity and their risk factors in Spain, 1987-1997. Prev Med. 2002, 34 (1): 72-81. 10.1006/pmed.2001.0962.

    Article  PubMed  Google Scholar 

  10. Australian Institute of Health and Welfare: Australia's Health. 2008, Canberra: Australian Institute of Health and Welfare, 2006. Cat. no. AUS 99

    Google Scholar 

  11. Chau J, Smith B, Bauman A, Meron D, Eveson-Annan M, Chey T, Farrell L: Recent trends in physical activity in New South Wales. Is the tide of inactivity turning?. Aust NZ Public Health. 2008, 32 (1): 82-85. 10.1111/j.1753-6405.2008.00170.x.

    Article  Google Scholar 

  12. Katzmarzyk PT, Tremblay MS: Limitations of Canada's physical activity data: implications for monitoring trends. Can J Public Health. 2007, 98 (Suppl 2): S185-194.

    PubMed  Google Scholar 

  13. Craig CL, Russell SJ, Cameron C: Reliability and validity of Canada's Physical Activity Monitor for assessing trends. Med Sci, Sports Exerc. 2002, 34 (9): 1462-1467. 10.1097/00005768-200209000-00010.

    Article  Google Scholar 

  14. National Institute of Medicine: Dietary Reference Intakes for Energy, Carbohydrate, Fiber, Fat, Fatty Acids, Cholesterol, Protein, and Amino Acids (Macronutrients). 2002, Washington, DC, National Academies Press

    Google Scholar 

  15. Vink JM, Willemsen G, Stubbe JH, Middeldorp CM, Ligthart RS, Baas KD, Dirkzwager HJ, de Geus EJ, Boomsma DI: Estimating non-response bias in family studies: application to mental health and lifestyle. Eur J Epidemiol. 2004, 19 (7): 623-630. 10.1023/B:EJEP.0000036814.56108.66.

    Article  PubMed  Google Scholar 

  16. Van Loon AJ, Tijhuis M, Picavet HS, Surtees PG, Ormet J: Survey non-response in the Netherlands: effects on prevalence and associations. Ann Epidemiol. 2003, 13 (2): 105-110. 10.1016/S1047-2797(02)00257-0.

    Article  PubMed  Google Scholar 

  17. Macera CA, Jackson KL, Davis DR, Kronenfeld JJ, Blair SN: Patterns of non-response to a mail survey. J Clin Epidemiol. 1990, 43 (12): 1427-1430. 10.1016/0895-4356(90)90112-3.

    Article  CAS  PubMed  Google Scholar 

  18. Craig CL, Gauvin L, Cragg SE, Katzmarzyk P, Stephens TM, Russell SJ, Bentz L, Potvin L: Towards a Social Epidemiological Perspective of Physical Activity and Health: The Aims, Design and Methods of the Physical Activity Longitudinal Study (PALS). JPAH. 2005, 3: 272-284.

    Google Scholar 

Pre-publication history

Download references

Acknowledgements

This work was funded by the Public Health Agency of Canada and the Interprovincial Sport and Recreation Council. The views expressed herein do not necessarily represent the views of these agencies. We would also like to thank the Institute for Social Research, York University for their work in selecting the sample and conducting the interviews.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Cora L Craig.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

CLC conceived this study, conceived the original study on which it is based, and wrote the manuscript. CC acquired the data, and contributed to interpretation of results and to the writing of the manuscript. JG assisted in the analysis and critically reviewed earlier drafts of the manuscript. AB contributed to the conceptualization of this study and interpretation of findings, and contributed to the writing. CTL and REA assisted conceptualize this study and provided substantial revision of earlier drafts of the manuscript. All authors have read and approved the final manuscript.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Craig, C.L., Cameron, C., Griffiths, J. et al. Non-response bias in physical activity trend estimates. BMC Public Health 9, 425 (2009). https://doi.org/10.1186/1471-2458-9-425

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1471-2458-9-425

Keywords