Sampling methods
Seven Community Health Services [CHSs] in the eastern part of the Netherlands collaborated with Maastricht University on the project named E-MOVO, a Dutch acronym for Electronic Monitor and Health Education [42]. E-MOVO is an electronic monitoring instrument, aimed at providing insight into health of adolescents of the 8th and 10th graders of secondary education. Whereas in most regions participation for adolescents at participating schools was mandatory, regions had the option to choose another sampling method. We used the results of two regions which used two different ways of sampling. In the mandatory sample (region Twente) sampling occurred mandatory and adolescents were recruited via secondary schools. Students in participating schools were instructed to complete the online questionnaire during a single class session (approximately 45 min) [43]. In the voluntary sample (region IJsseland) the adolescents were recruited voluntarily and were invited via a postal mailing to their home address, containing a hyperlink and personal code to the online questionnaire.
Non-response bias in the mandatory sample is considered minimal, as non-participation occurs in clusters (i.e. schools and classes) instead of the individual level. Each school in the region was invited to have all classes participate. There were several schools that did not participate at all, and some participating schools did not include all classes, due to practical reasons such as scheduling difficulties and lack of computer rooms. Therefore, we assume that there is minimal non-response bias in the data of the mandatory sample at the individual level. In contrast, due to higher non-response in the voluntary sample, it is likely that there is more non-response bias compared to the mandatory sample, as non-respondents here may differ in several characteristics from respondents.
An important requirement for the purpose of this study is that both populations from which the two samples were recruited are indeed comparable. Both regions are geographically adjacent, and similar with respect to socio-economic and urbanisation characteristics. With regard to risk behaviour prevalence, interregional comparability can be verified with two Dutch data resources on alcohol and tobacco consumption. In both resources data were collected across all regions with a standardized recruitment strategy and questionnaire, allowing direct interregional comparisons without a differential bias due to non-response. First, in the Health Monitor of 2012, with a representative sample of Dutch adults of 19 years and older, smoking prevalence was estimated at 23.9% in Twente and 22.0% in IJsselland [44, 45]. Weekly prevalence of heavy drinking (consuming 5 or more standard units on a single day at least once a week) was estimated at 9.2% in Twente and 8.7% in IJsselland [44, 45]. Second, the Dutch Health Survey with a representative sample of Dutch individuals of 12 years and older, identified the percentage smokers in 2008 at 32.3% in Twente and 29.8% in IJsselland. Hazardous drinking prevalence, defined in this study as either heavy drinking or exceeding moderate drinking levels (≥14 units a week for females and 21 units for males), was estimated in 2008 at 20.7% in Twente and 19.8% in IJsselland [46]. In general, available national data show that both regions included in this study show negligible differences in alcohol consumption, and a small difference in smoking prevalence. Although these data could not be specified for adolescents, in the case of the Dutch Health Survey adolescents of 12 year and older were included in the estimates. Nevertheless, it seems reasonable to assume that the magnitude of interregional differences found among adults may also apply to the adolescent populations of these regions.
Participants
In the mandatory sample, the CHS of Twente was involved in recruiting schools in the 2011 study and maintained contact with its 14 municipalities within the region. All 59 secondary schools were approached, from which 39 participated in the E-MOVO study of 2011. The research team of E-MOVO informed the municipalities via e-mail about the study. The CHS of Twente informed each municipality and recruited schools within the community by sending an information sheet. Within participating schools informed consent was obtained from parents via an opt-out procedure. In the voluntary sample, the CHS of IJsselland selected a random sample of youngsters between the ages of 12 and 23, stratified on all municipalities in the region. For comparison of the regions, only the ages from 13 through 16 were included. Informed consent was obtained by sending a postal mail to the parents with an information sheet and the invitation for their child to participate.
Measures
All matching items between the two surveys (Twente and IJsseland) were analysed. Measures were based on self-reports which have been shown to be reliable regarding tobacco, alcohol, and other drug use among adolescents [47, 48].
Demographics
Gender, age (in years), and education (11 options in Twente, 15 options in IJsselland) were assessed. For analytic purposes, education was dichotomised into low (“preparatory middle-level vocational education”) or high (“higher general continued education”/“preparatory scholarly education”).
Tobacco consumption
Participants were asked how often they smoked at present (0 = not at all; 1 = less than once a week; 2 = at least once a week, 3 = but not daily; 4 = every day). As previous studies reported whether or not youngsters smoke daily and due to violation of the linearity assumption, tobacco consumption was dichotomised into ‘daily smoker’ and ‘non-daily smoker’ [49, 50].
Alcohol consumption
Alcohol consumption was operationalised with three items. Participants were asked whether they had ever consumed alcohol (yes; no), how often they had had alcohol in their lives, and how often they had consumed alcohol in the past four weeks (0; 1; 2; 3; 4; 5; 6; 7; 8; 9; 10; 11–19; >20 times). As multiple reports mention whether youngsters had or had not consumed alcohol in the past four weeks [49, 50] and due to violation of the linearity assumption, alcohol in the past four weeks was dichotomised (yes/no).
Mental health
The Strengths and Difficulties Questionnaire [SDQ] is a behavioural screening questionnaire for children aged 4–16 years [51, 52]. The SDQ consists of 25 items and measures five scales of five items each (i.e. emotional symptoms, conduct problems, hyperactivity-inattention, peer problems, and prosocial behaviour). It has been extensively validated in many countries [53, 54]. The internal consistency (Cronbach’s alpha of .64), test-retest stability (except for the prosocial behaviour subscale (.59), all intraclass correlation coefficients were above.70), and parent-youth agreement of the various SDQ scales have been found acceptable [54]. To estimate the ‘probability for any behavioural problems from the SDQ scores, a modified version of Goodman’s algorithm [51] was used for the total score. Based on the algorithm, the probability of a psychiatric disorder was calculated as ‘1 = unlikely’ (0–15), ‘2 = possible’ (16–19), and ‘3 = probable’ (20–40) [51].
Subjective health status
One item was used to measure the subjective health status, consistent with other studies (e.g. DeSalvo, Bloser, Reynolds, He, & Muntner, 2006 [55]) Individuals were asked how they perceived their health in general (1 = very good; 2 = good; 3 = neutral; 4 = not good; 5 = poor).
School experiences
Participants were asked with one item how they experienced school (1 = great fun; 2 = fun; 3 = neutral; 4 = not fun; 5 = dreadful).
Sexual behaviour
In order to measure sexual behaviour one item was used [56]. Individuals were asked whether they had ever had sexual intercourse with someone (1 = never; 2 = once; 3 = couple of times; 5 = regularly).
Statistical analyses
First, for both samples we examined whether the observed distribution of demographics deviated from the expected distribution in the population. For gender, a one sample t-test was performed. For the distribution of age and education level we provided descriptive comparisons of the mean age and education level (high vs. low) of the samples to the population estimates available to the best of our knowledge. Statistical tests were not performed with these demographic variables as the reliability of these estimates was lower than for gender.
Second, tests were performed of differences between both samples. For demographic characteristics, an independent samples t-test was used for age, and Pearson χ2-test for gender and education level (high vs low). To examine whether the samples differed on health-related variables, we first conducted univariable logistic regression analyses for each health-related variable of interest as independent variable and sampling method as dependent variable (mandatory sample Twente =0, voluntary sample IJsselland =1). Although, theoretically, sampling method would be considered as the independent variable, this was reversed in these analyses to allow a uniform analysis technique to be used for all health-related variables, regardless of the different measurement levels of these variables.
For the logistic regression analyses we checked the linearity assumption for non-binary variables (i.e. sexual intercourse, subjective health, school experiences, tobacco consumption, alcohol in past four weeks, lifetime alcohol consumption, and SDQ). Except for SDQ, alcohol in past four weeks, and tobacco consumption, variables did not violate the linearity assumption. To solve this issue, these three outcome measures were recoded into binary (tobacco consumption: 0 = no daily smoker, 1 = daily smoker; alcohol past four weeks: 0 = no, 1 = yes) or three-level (SDQ: 1 = unlikely, 2 = possible, 3 = likely). Further, to examine whether the differences in health characteristics between the samples could be explained by differences in demographic characteristics, all multivariable logistic regression analyses were repeated, with demographics (i.e. age, gender, and education) added as covariates. Intercorrelations were checked to test for collinearity between the health-related variable and demographic variables entered into the model. No signs of collinearity issues were found among the independent variables with all tolerance levels above 0.1 [57] and VIF values below 10 [58].
To examine moderation effects of sampling bias on associations between health related variables within subjects, an interaction term was computed for sampling method with tobacco consumption. Then interaction analyses were performed using logistic regression analysis according to the procedure by Baron and Kenny [59], with tobacco consumption, sampling method, and the sampling*tobacco use interaction term entered as independent variables. As independent variables the following health variables were tested in consecutive models: mental health, subjective health status, and school experiences. The same procedure was followed for tobacco consumption, alcohol consumption, and alcohol in past four weeks as dichotomous dependent variables. Due to the large sample size in this study a significance level of <0.01 was used in all analyses. All analyses were carried out using SPSS 20.0.