Non-response bias in physical activity trend estimates

The Global Strategy for Diet and Physical Activity [1] emerged in response to the recognition of the growing global burden posed by non communicable disease (NCD). Many national health surveys monitor NCD risk factors using self-reported data to assess trends in these risk factors over time. [2–5].

The physical activity levels of Canadian have been tracked since 1981 [6]. These data have been essential in setting national physical activity goals, identifying which groups are less active than others and determining whether differences in physical activity levels between groups have been increasing, decreasing or remaining stable over time. In Canada, population levels of leisure-time physical activity (PA) based on self-reports have increased over a twenty-year period. [6] However, during this same period the prevalence of overweight and obesity has also increased at alarming rates. This co-occurrence of increases in reported leisure time PA and obesity has also been observed in Finland [7, 8], Spain [9], Australia [10, 11] and elsewhere. These apparently contradictory trends may be explained by changes in dietary patterns, the limitation of physical activity measures based on only leisure-time rather than total activity across all domains, or of other changing patterns. Other explanations include differences in response bias, such as changes in social desirability bias regarding active lifestyles, or decreasing response rates (for example due to the increase in telemarketing) leading to differential non-response bias in estimates over time [12]. Differentials in the magnitude of bias over time may lead to erroneous conclusions regarding the changes in the monitored prevalence estimates and relationships with other variables of interest.

The purpose of this short report is to examine potential non-response biases (e.g., in terms of number of attempted contacts) in self-reported data introduced during the recruitment to random digit dial surveys and how these may influence trend estimates, using the example of PA.

Study participants 18 years and older were recruited to the Physical Activity Monitor (PAM) a nationally representative Canadian survey of adults launched in 1995, repeated in 1997 and run continuously since 1998 employing a monthly sample. The sample is selected in two steps. Households are selected using random digit dialing and then the individual with the nearest birth date to the interview date is selected. During the recruitment process, the following information is recorded 1) number of attempts made to call the household, 2) number of times someone answered in trying to reach the selected individual, 3) the number of contacts with the selected individual and completion status (initial agreement, refusal converted to participation). Verbal informed consent was obtained according to the procedures of the Ethics Committee, York University.

An adapted version of the Minnesota Leisure Time Physical Activity questions was administered from 1995-2002 and then again in 2007. The PA score was calculated by summing the product of the number of times each type of PA was performed in the previous 12 months, the duration in minutes and the intensity of the activity expressed in METs [13]. 'Adequate' PA was defined as 5294 kilojoules (1260 MET-minutes) per week and equivalent to at least 60 minutes of moderate-intensity activity daily [14]. Differences in the prevalence of adequate PA by attempts, answers, contacts, and completion status were tested across and within survey periods using Student's t tests (SPSS Complex Samples, version15) and adjusting significance levels for multiple comparisons by Holm's procedure.

This study found no evidence of a differential bias in levels of adequate PA according to the various stages at which non-response could occur during recruitment. Therefore, there is no support for the hypothesis that increase in physical activity over time were related to changes in the underlying representivity of the samples. As a result, differences in non-response rates do not explain the apparent contradictory trends of increases in both sufficient leisure time physical activity and obesity over time. More investigation is required to understand changes in the multiple factors influencing the energy balance equation. In particular, a better understanding is required of changes in sedentary behaviours, total energy expenditure through physical activity, dietary patterns, and total energy intake, and how these changes relate to the rising rates of overweight and obesity.

One limitation of this study is that comparisons were made between individuals who were easier and harder to recruit, but was unable to make comparisons with those not recruited at all due to a lack of information external to the surveys. External data has been used to quantify non-response bias in other studies with equivocal results. Using proxy reports from more cooperative or readily accessible family members for PA and other key variables, Vink [15] determined that although the non responding family members generally had less favorable estimates than those responding, the impact on estimates was small. In the Netherlands, the prevalence of healthy lifestyle behaviours was over estimated, but there was no bias in associations [16]. In contrast, Macera et al [17] found limited impact on estimates of key variables comparing an initial face-to-face contact (as opposed to telephone contact herein) to subsequent follow-up. This was also the case in a different Canadian study using the same PA measure as in the current study [18]. However, trend estimates may also be influenced by response bias including changes in question order, inclusion of sensitive subject areas in surveys, and changes in social desirability (for example, higher over-reporting of PA as low levels of participation become more widely recognized as a health issue). Accounting for bias is important to those involved in health policy to ensure that any observed changes are indeed due to secular trends and not an artifact of temporal differences in bias.

Although it was not possible to quantify differential impacts of response bias over time, the data do not demonstrate a significant effect on trend estimates due to differential response rates over the twelve year span.