Reliability and validity of the Adolescent Stress Questionnaire in a sample of European adolescents - the HELENA study
BMC Public Health volume 11, Article number: 717 (2011)
Since stress is hypothesized to play a role in the etiology of obesity during adolescence, research on associations between adolescent stress and obesity-related parameters and behaviours is essential. Due to lack of a well-established recent stress checklist for use in European adolescents, the study investigated the reliability and validity of the Adolescent Stress Questionnaire (ASQ) for assessing perceived stress in European adolescents.
The ASQ was translated into the languages of the participating cities (Ghent, Stockholm, Vienna, Zaragoza, Pecs and Athens) and was implemented within the HELENA cross-sectional study. A total of 1140 European adolescents provided a valid ASQ, comprising 10 component scales, used for internal reliability (Cronbach α) and construct validity (confirmatory factor analysis or CFA). Contributions of socio-demographic (gender, age, pubertal stage, socio-economic status) characteristics to the ASQ score variances were investigated. Two-hundred adolescents also provided valid saliva samples for cortisol analysis to compare with the ASQ scores (criterion validity). Test-retest reliability was investigated using two ASQ assessments from 37 adolescents.
Cronbach α-values of the ASQ scales (0.57 to 0.88) demonstrated a moderate internal reliability of the ASQ, and intraclass correlation coefficients (0.45 to 0.84) established an insufficient test-retest reliability of the ASQ. The adolescents' gender (girls had higher stress scores than boys) and pubertal stage (those in a post-pubertal development had higher stress scores than others) significantly contributed to the variance in ASQ scores, while their age and socio-economic status did not. CFA results showed that the original scale construct fitted moderately with the data in our European adolescent population. Only in boys, four out of 10 ASQ scale scores were a significant positive predictor for baseline wake-up salivary cortisol, suggesting a rather poor criterion validity of the ASQ, especially in girls.
In our European adolescent sample, the ASQ had an acceptable internal reliability and construct validity and the adolescents' gender and pubertal stage systematically contributed to the ASQ variance, but its test-retest reliability and criterion validity were rather poor. Overall, the utility of the ASQ for assessing perceived stress in adolescents across Europe is uncertain and some aspects require further examination.
Chronic stress is assumed to have role in the development of obesity by interacting with mechanisms underlying energy intake and expenditure, and stimulating visceral fat accumulation in favour of abdominal obesity . Recently, further investigation of the facilitating effect of chronic stress on obesity was highlighted, particularly in adolescents . Adolescence is characterized by remarkable plasticity, with fundamental physical, psychological and behavioural changes that require attention in terms of obesity research , and stress is considered to be inherent to this developmental stage . As far as we are aware, there is no standardized methodology for the assessment of adolescent psychosocial stress, although stressor questionnaires (checklists or interview-based) are often used. Stressor checklists are self-reported questionnaires concerning the experience of stressful events by the respondent during a certain time period. They can be implemented on a large scale and are cost-effective . Criteria for choosing a suitable checklist are a high validity, it must be relevant to its current time and relate to the scope of measurement (general or specifically focusing on certain stressors such as school stress), and be suitable for the target population. When using an existing checklist on a new population for which it was not originally developed, its validity should be considered.
Within the HELENA (Healthy Lifestyle in Europe by Nutrition in Adolescence) project , the association between stress and the onset of obesity in European adolescents was assessed. Owing to the lack of a recent stress checklist for use in European adolescents, the Adolescent Stress Questionnaire (ASQ), recently developed and validated by Byrne et al. for Australian adolescents, was utilized . The ASQ was developed to address the requirement for systematic research examining adolescent stress in the early 21st century. Byrne and colleagues state that 'the ASQ is not a measure of symptomatic distress though it does assess subjective stressor load'. The ASQ is a checklist containing 56 items covering a broad range of perceived adolescent stresses, with complete relevance to its current time. The items are statements concerning events or situations which adolescents could find stressful. These items require to be evaluated by the respondent in terms of the extent to which they were experienced as being stressful during the previous 12 months. The items were generated in adolescent focus groups, where adolescents brainstormed on the concerns and challenges associated with adolescence that had affected them or their peers. Using Principal Component Analysis, the items were attributed to 10 components or dimensions of adolescent stressor experiences (stress of home life, school performance, school attendance, romantic relationships, peer pressure, teacher interaction, future uncertainty, school/leisure conflict, financial pressure and emerging adult responsibility). These component scales were considered to be 'thematically meaningful within the existing body of theory and knowledge regarding the experience of adolescent stress' . The ASQ has a good construct validity (positive correlations with measures of anxiety and depression, and negative correlations with self-esteem), internal and test-retest reliability, and demonstrates consistent gender differences (girls report higher stress levels than boys), but poor correlations with age for certain stress dimensions . The present study investigated the internal and test-retest reliability of the ASQ, its construct and criterion validity, and the independent contributions of socio-demographic characteristics (gender, age, pubertal stage and socio-economic status) to the variance in ASQ, when implemented in a multinational European epidemiological survey. If the ASQ was demonstrated to be a valid and reliable tool in a sample of European adolescents, it could be applied to identify important sources of adolescent stress in Europe and to investigate associations between adolescent stress and overall health, chronic morbidities (the development of obesity for example), mental well-being and other health-related outcomes.
Study design and population
The present study was implemented within the framework of the HELENA project . The aim of the HELENA cross-sectional study (HELENA-CSS) was to obtain reliable and comparable data from a selected cohort of European adolescents concerning a broad variety of parameters related to nutrition, health, physical activity and fitness . Adolescents were selected by random cluster sampling (all pupils from a selection of classes from all schools in 10 European cities), and stratified by geographical location, age and socio-economic status (SES). The sample size for the HELENA-CSS was calculated with a confidence level of 95% and ± 0.3 error, based on the variance in body mass index, and yielded a sample size of 300 in each city. A detailed description of the sample size estimation, the sampling and recruitment procedure of HELENA-CSS, is given by Moreno et al. . In 10 European cities, 3865 adolescents (more than 300 per city) aged 12 to 17 years participated in the HELENA-CSS, and 3528 adolescents were eligible for inclusion (criteria included written informed consent, not participating in another survey, aged between 12.5 and 17.5 years, and data being available concerning an individual's gender, height and weight). Six of the 10 European cities included in the HELENA-CSS, participated in the stress module: Ghent, Stockholm, Vienna, Pécs, Zaragoza and Athens. Of the 2177 eligible adolescents from these six cities, a reduced sample size of 1240 adolescents participated in the stress sub-study. The sample was reduced because the stress module was optional and omitted when fieldworkers were constraint by time limits or logistically restricted.
The HELENA-CSS fieldwork was conducted between October 2006 and December 2007 throughout the whole school year except before, during or immediately after examinations, as this could have influenced the test results. The HELENA fieldwork consisted of a clinical and body composition examination, physical activity and fitness assessment, blood sampling and assessment of questionnaires . The stress module encompassed the ASQ and measurement of salivary cortisol. These measurements were integrated in the fieldwork of the HELENA-CSS. Salivary cortisol was measured in a randomly selected sub sample of 50 pupils per city, but with the aim of having an equal distribution with regard to age and gender across the pupils who provided saliva samples. The present study was conducted according to the guidelines in the Declaration of Helsinki and the project protocol was approved by the local or national Ethics Committees of all the participating cities (the Ethics Committee of the Ghent University Hospital (Ghent, Belgium), the Regional Ethics Committee in Stockholm (Stockholm, Sweden), the Ethics Committee of the Medical University Vienna and the Vienna General Hospital (Vienna, Austria), the Regional Research Ethics Committee of the Medical Center in Pécs (Pécs, Hungary), the Research Ethics Committee of the Government of Aragon (CEICA, Zaragoza, Spain), and the Ethics Committee of Harokopio University (Athens)). All participants and their parents provided written informed consent for participation. Detailed information concerning the ethical/regulatory aspects and Good Clinical Practice within HELENA-CSS is described by Beghin et al. .
In addition to the HELENA-CSS study, a test-retest study of the ASQ was performed. A convenience sample of 55 voluntary adolescents (44% girls), with a mean age of 14.6 (± 1.1) years, was recruited in the region of Ghent. Criteria for participation included being aged between 13 and 17 years, having access to the Internet, non-participation in HELENA-CSS and a willingness to complete several paper-and-pencil questionnaires. Informed consent was obtained from the participants and their parents. Thirty-seven of the 55 adolescents completed the ASQ twice, with a test-retest interval of two weeks.
In the HELENA-CSS, a questionnaire was administered concerning socio-economic characteristics such as the number of cars and computers at home, internet at home and whether or not the individual had their own bedroom. Using these characteristics, the Family Affluence Scale was determined on the basis of a model developed by Currie et al. . This model was adapted by replacing the item concerning 'holidays' (this was not assessed in HELENA) by 'internet at home'. The Family Affluence Scale indicated the socio-economic status (SES) on a scale of 0 (very low SES) to 8 (very high SES).
A clinical examination was performed during the fieldwork: birth date, gender and pubertal stage (according to the protocol developed by Styne ) were recorded and current medication use was reported.
Adolescent stress was assessed with the ASQ, developed and validated by Byrne et al. . This ASQ assesses subjective stressor load, covering the broad domains of adolescent stressor exposure. The 56 items on this checklist were grouped into 10 stress component scales: stress of home life, school performance, school attendance, romantic relationships, peer pressure, teacher interaction, future uncertainty, school/leisure conflict, financial pressure and emerging adult responsibility. A complete list of the items and their allocation to the scales is presented in the 'Results'-section (Table 1). The items were ordered randomly (not by component scale) and preceded by a short introduction on how to fill in the ASQ. Respondents were asked to indicate on a 5-point Likert scale (1 = "not at all stressful (or is irrelevant to me)", 2 = "a little stressful", 3 = "moderately stressful", 4 = "quite stressful", 5 = "very stressful") how stressful these items had been to them during the past year .
The original English version of the ASQ was translated twice into the local languages of the six participating cities (Dutch, Swedish, German, Hungarian, Greece and Spanish). These two independent translations were compared and a compilation of the best translations was carried out. Back translations were performed for quality control and local questionnaires were modified accordingly until agreement was obtained between the persons performing the translations. The ASQ was administered together with the other questionnaires of the HELENA study following a standardized procedure: questionnaires were completed in a classroom setting in total silence and fully supervised to avoid between-subject interaction. The answer categories 1 to 5 got the respective score (1-5). A score for each stress component scale was calculated by counting the scores of the items belonging to that scale. In addition, a stress summary score was obtained by adding the individual scores of all 56 items.
Baseline wake-up salivary free (BWSF) cortisol was measured in the adolescents as a biomarker for chronic stress to investigate associations with the ASQ. It is hypothesized that the ASQ scores of the component scales and the summary score are positively associated with BWSF cortisol. Cortisol is a main end product of the 'stress system'. Serum cortisol levels are the result of appraising all stress-inputs on the brain, coping and recovery from them, and are influenced by several neuro-endocrine and physiological pathways [1, 11]. Unbound or free cortisol is also present in saliva , and is positively associated with acute  and chronic [12, 14, 15] stress. To reduce variance in cortisol levels due to diurnal variations in salivary cortisol , baseline (without stimulation) salivary cortisol was measured immediately after awakening. To account for intra-individual variability in salivary cortisol , awakening samples from seven consecutive days were collected. Saliva was sampled during the same week as the other HELENA measurements. Participants were subject to an oral introduction on how to take saliva samples (including time and conditions of sampling, demonstration of sampling procedure, points of attention to be addressed such as no eating, drinking or brushing teeth, and cooled storage of samples) and a detailed instruction sheet was provided.
A thorough protocol was composed in English explaining the procedure of saliva sampling to ensure standardization across the six cities. Saliva was collected with Salivettes® (Sarstedt, Germany), providing stable samples at room temperature for a minimum of one week. The Salivettes® were centrifuged at 2000 g for 10 min, and the filtrates were stored at -20°C. Before analysis, the samples were thawed and mixed. Salivary cortisol was measured using a modification of an unextracted radioimmunoassay method (Diasorin) for serum cortisol. Briefly, 200 μl saliva was pipetted into the coated tube and incubated with 125I cortisol for 45 minutes at 37°C. The modified cortisol assay had a measuring range from 0.5-30 μg/L and within- and between-run coefficients of variation of < 5% and < 10%, respectively. The cortisol concentration units from the analysis (μg/L) were converted into SI units (nmol/L) by multiplying the values with a conversion factor of 2.759 . Only adolescents who had taken at least three saliva samples taken on awakening between 6 am and 8 am and with a cortisol concentration within the reference range of 3.0 to 54.9 nmol/L, as suggested by Groschl, Rauh & Dorr , were included in the validation analysis. For these adolescents, a mean value of their valid BWSF cortisol levels was calculated and used for further analyses.
A total of 1240 eligible HELENA participants completed the ASQ, of whom 100 adolescents were excluded for the internal reliability and construct validity analysis of the ASQ, as they had more than four missing ASQ scales (n = 16) or 50% answers of 'do not apply' (n = 84), resulting in a study sample of 1140 adolescents. Comparison of the adolescents participating in the stress sub-study (N = 1240) with the total eligible HELENA sample (N = 2177) of the six cities revealed no significant differences in terms of gender distribution (p = 0.063) or age (p = 0.495).
Cronbach α-values of the stress component scales were calculated to ensure internal reliability. A test-retest reliability analysis was carried out by calculating the Intraclass Correlation Coefficient (ICC) between the scales of two assessments, two weeks apart, on a separate sample of 37 Belgian adolescents. A threshold of 0.8 for Cronbach α-values and ICCs was considered to indicate good internal reliability  and test-retest-reliability , respectively. Contributions of socio-demographic characteristics (gender, age, pubertal stage, SES) on the variability in ASQ scale scores and the summary score were investigated using Hierarchical Linear Models (HLM) with Restricted Maximum Likelihood Estimation , whereby 'city' was specified as subject grouping variable, and gender, age, pubertal stage and SES as fixed parameters.
A Confirmatory Factor Analysis (CFA) was performed to examine whether the original scale construct of the ASQ was confirmed by the data from the European adolescent population. The first-order model, with the individual items as manifest variables and the 10 component scales as first-order factors, demonstrated high mutual correlations between the factors (two thirds of the correlations were ≥ 0.6, data not shown). Therefore, a second-order CFA with Maximum Likelihood Estimation was executed with the summary score as the second-order factor. The error variances of the individual items were set as free parameters, while the variances of the latent factors were fixed at 1.0 in the model. The standardized factor loadings obtained represent the correlation between the observed variables and the extracted factors. Standardized loadings above 0.4 indicated an acceptable correlation and loadings above 0.5 indicated a good correlation; loadings below 0.4 were indicative of a poor correlation. Model evaluations were carried out using three 'goodness of fit' indices: χ2 is reported as an absolute fit index, while Bentler's comparative fit index (CFI) and Root Mean Square Error of Approximation (RMSEA) are reported as comparative indices. For Bentler's CFI, a threshold of > 0.90 was considered to indicate a good fit  and for RMSEA, values < 0.05 or < 0.08 were representative of a good or acceptable fit , respectively.
For comparison of the ASQ with BWSF cortisol (criterion validity), 255 eligible participants provided saliva samples. Of these, 15 adolescents were excluded as they did not have a valid ASQ, 11 adolescents were excluded for taking oral contraceptives or steroids (these were the only reported medications documented to influence salivary cortisol levels ), and 29 adolescents were excluded for having no valid saliva samples (a minimum of three saliva samples taken on awakening between 6 am and 8 am and with a cortisol concentration within the reference range of 3.0 to 54.9 nmol/L were required). Data from 200 adolescents were used to assess the criterion validity of the ASQ. To investigate positive associations between the ASQ scores (scales and summary) and BWSF cortisol, linear regression analyses were performed with BWSF cortisol as dependent and with one of the ASQ scale scores or summary score as predictor (one model for each score), additionally controlling for pubertal stage, and this for boys and girls separately. Assumptions for performing linear regression analysis (independent observations, linearity, homoscedasticity, normality of residuals, non-collinearity between predictors) were verified and these were all met.
The CFA analysis was performed using SAS version 9.2; all other analyses were conducted using SPSS version 15.0.
Table 2 describes the study samples. Results of the internal and test-retest reliability of the ASQ are presented in Table 3. Only 50% of the stress component scales had Cronbach α-values of ≥ 0.8, indicating a moderate internal reliability of the ASQ scale construct. Test-retest reliability analysis of the ASQ resulted in ICCs lower than 0.8, with the exception of romantic relationships, demonstrating poor test-retest reliability between these measurements (Table 3).
The independent contributions of socio-demographic variables to the variance in ASQ scale scores and the summary score were investigated using HLM (Table 4). Intercept-only models demonstrated that 'city' as grouping variable had a proportional explained variance of 6.7% for the summary score, while for the separate scales these proportional explained variances varied from 1.3% to 10.9%. Results of the HLM showed that mainly gender and pubertal stage systematically contributed to the variance of the majority of ASQ scales and the summary score: boys and those in a pre-, beginning-, mid-, or advanced pubertal stage (stage I to IV) reported lower stress scores compared to girls or those in a post-pubertal stage (stage V), respectively. Age and SES in general contributed to a lesser extent to the variance in ASQ scale scores and the summary score: age was only a significant negative predictor for stress from teacher interaction, peer pressure, and the summary score, while SES was a negative predictor for stress from teacher interaction and financial pressure.
The results of the second-order CFA are presented in Table 1. One item (item 'getting up early in the morning to go to school' of school attendance) had a factor loading below 0.4, indicating a poor correlation with the respective factor (school attendance), while four items had acceptable factor loadings between 0.4 and 0.5. The majority of the items (51 out of 56) demonstrated high correlations with their component scale, with factor loadings > 0.5. The component scales correlated highly with the summary score, with factor loadings > 0.6. The models' absolute fit index χ2 was 5112.11 (df = 1474, p < 0.001). The comparative model fit indices Bentler's CFI and RMSEA were 0.81 and 0.054, indicating a poor and acceptable fit, respectively, of the theoretical model of the ASQ in the data.
Results of the linear regression analysis for assessing the criterion validity of the ASQ are presented in Table 5. In boys, scale scores for school performance, peer pressure, future uncertainty and emerging adult responsibility were a significant positive predictor for BWSF cortisol. Results for the scale scores of home life and financial pressure and the ASQ summary score were borderline significant. In girls, none of the scale scores or the summary score came out as a significant positive predictor of the BWSF cortisol levels.
The present study investigated the validity and reliability of the ASQ for assessing stress in European adolescents. The ASQ, originally developed for Australian adolescents, is documented to assess subjective stressor load and consists of 10 different stress dimensions covering a broad domain of adolescent stress experiences . The reliability analysis in the present study indicated a moderate internal reliability of the ASQ scales, as five of 10 Cronbach α-values were > 0.8. The Cronbach α-values were lower than those in the validation study of the original ASQ carried out by Byrne et al. . Test-retest ICCs of the ASQ scales in the present study were, with the exception of stress from romantic relationships, < 0.8, indicative of poor stability over time. They were lower than those reported by Byrne et al. , probably because of the greater within-measurement period (two weeks in the present study rather than one week in the study of Byrne et al. ) and the lack of power owing to a small sample size (37 in the present study compared with 105 in the study of Byrne et al. ).
The present study demonstrated that adolescent girls experienced higher stress levels than boys, confirming the gender differences in the ASQ as presented by Byrne et al. , and confirming previously reported gender differences in adults  and adolescents . In addition, the present study demonstrated that pubertal stage was also a predictor for the ASQ, with the adolescents that are already in a post-pubertal stage of development having higher scores than those still in a pre-pubertal stage or in full pubertal development. This is in line with the hypothesis of Dahl and Gunnar that pubertal development is the driving force behind increasing stress sensitivity during adolescence  and with findings from Sumter et al. that the biological stress sensitivity increased with pubertal status . The differential functioning of the ASQ according to the adolescents' gender and pubertal stage give strength to the validity of the ASQ. Age was no systematic predictor of the ASQ scores, suggesting that the ASQ functions similarly across the whole age range (12.5-17.5 years). This is not in agreement with observations of Byrne et al., where bivariate correlations with age were weak positive for five out of ten scales (significant correlations varied from 0.12 to 0.35) . This might be indicative of differential scales functioning across cultures (see below). Surprisingly, no systematic association was evident between SES and the ASQ scores, conflicting with previous evidence that resource- and prestige-based indices of family SES were associated with adolescents' risk for uncontrollable and controllable negative life events, respectively . These controversial results are probably because the ASQ assesses daily stressful situations or events, which have a lower impact on life and more frequent occurrence than the negative life events assessed in the study of Brady & Matthews .
The results of the second-order CFA suggest that the theoretical model of the ASQ fits moderately well with the observed data in the present population. The absolute goodness of fit index χ2 was not indicative of a good fit. However, this fit index is widely recognized to be problematic as it is highly sensitive to sample size. The comparative fit indices, Bentler's CFI and RMSEA, were indicative of a marginally good or acceptable fit. The factor loadings of the majority of the observed items (51/56) were indicative of high correlations with the extracted factors. The factor loadings of the component scales on the summary score were higher (> 0.6), suggesting very good correlations between the scales and the summary score. The moderate construct validity could be explained by the fact that the nature of adolescent stress differs across culture boundaries. European adolescents may experience stress in a different way from Australian adolescents owing to differences in cultural background, education, living conditions, norms and standards. It could be hypothesized that the greater these differences, the lower the applicability of the original scale construct will be. This finding is in agreement with the concern outlined by Byrne et al. that 'the capacity of scales of adolescent stress to cross boundaries of culture is not yet well understood' .
Linear regression analyses demonstrated that in boys, four out of 10 ASQ scale scores were found to be a significant positive predictor for their BWSF cortisol values, while in girls this positive association was not observed for any ASQ scale. These results indicate a rather poor criterion validity of the ASQ when compared with BWSF cortisol, especially in girls. A possible explanation for the weak or lack of distinct positive association between the ASQ and BWSF cortisol could be that the ASQ assesses only one aspect of the stress concept, namely the personal perception of stress from certain situations and events in adolescent life (so-called cognitive appraisal in the 'Stress and Coping Theory' developed by Lazarus and Folkman ), while the adolescents' coping mechanisms (defined as the second process in the 'Stress and Coping Theory' ) are not determined by the ASQ. Salivary cortisol is an indicator of Hypothalamic-Pituitary-Adrenal (HPA) axis activity and can be considered as a reflection of the adaptational outcome, which is the result of the appraisal of and coping with stress . No controls for coping mechanisms could be used, and this could explain why increasing ASQ scores are not associated with increased salivary cortisol levels. This limitation of the ASQ restricts its utility to the assessment of cognitive appraisal of stress and not chronic stress, as a result of which its associations with stress-related health outcomes could be weakened or non-existent. Therefore, the information obtained with the ASQ should be complemented with information concerning previous lifetime stressful events (e.g. death of family members, sexual, psychological or physical abuse) as these are likely to influence the test results. Moreover, this additional information would complement the picture of the adolescents' chronic exposure to stress and provide more opportunities for investigating the potential role of chronic stress in the etiology of certain disorders. The lack of association between self-reported subjective measures of stress and cortisol measurements in biological samples has previously been denominated the 'lack of psychoendocrine covariance' . Hellhammer et al. indicated that this phenomenon is not surprising given the complex interplay of neurobiological events that link perceived stress to HPA axis activity and the difficulties in assessing perceived stress .
In addition, in children with post-traumatic stress disorder (PTSD), long-term cortisol levels were no longer elevated as a reaction to trauma, but normalized . The adolescents in the present study were not screened for previous development of PTSD or other severe traumas in their lifetime, and this could have reduced the expected positive association between perceived stress and cortisol levels. Therefore, future research is recommended to include all lifetime traumatic/stressful events in addition to focusing on a certain time period.
Strengths and limitations
The multi-national character of HELENA-CSS and the stress sub-study, together with the strict standardization of the fieldwork across all European cities are great advantages of this project and make this project unique in its kind. The integration of the stress module in the HELENA-CSS had the advantage of investigating in-time associations between the experience of psychosocial stress and other health-related parameters (socio-demographic, clinical, food-related and physical activity parameters) conducted in HELENA. Unfortunately, the study also faces some limitations, which are specified below.
For feasibility reasons, the study population within the HELENA-CSS was determined on the basis of random cluster sampling. Inherent in cluster sampling, the standard errors for estimates are greater than for simple random samples, the so-called 'design effect' . In view of these possible higher standard errors, it could be the case that some differences in ASQ scores between groups were not revealed.
Test-retest reliability analysis and criterion validity analysis were performed on a convenient sample of adolescents and are subject to limitations inherent to convenience sampling (e.g. non-generalization, and a potentially large and unmeasured bias). These data were collected for validation purposes only and the consequences of this convenience sampling are assumed to be minimal. In addition, the samples for these analyses were small and the lack of power could have attenuated the test results and contributed to the rather poor test-retest reliability and criterion validity.
Other limitations included the time limits for the fieldwork of HELENA (only a well-defined period could be spent in the schools) and the high load of the measurements. Owing to these logistic limitations, BWSF cortisol was measured as this could be carried out at home rather than during school hours. Alternative procedures and biomarkers, recently suggested to be promising biomarkers in the context of stress assessment, are the cortisol awakening response (defined by Clow et al. as 'the period of cortisol secretory activity in the first 45-60 min immediately post-awakening') [12, 14, 36], hair cortisol  and salivary α-amylase . These alternative procedures and biomarkers for chronic stress may have led to better criterion validity. However, this large body of evidence was not available at the beginning of the present study-. In addition, several methodological challenges associated with the measurement of these alternative biomarkers have been documented [14, 39], which would have been difficult to address in the school setting of the HELENA project. Previously, several methodological difficulties related to salivary cortisol assessment have been addressed in the literature, which are hard to control in epidemiological surveys [25, 40]. In the present study, researchers experienced some difficulties in collecting and analysing salivary cortisol. For instance, it was impossible to standardize the time of arousal and therefore sampling time, a factor that is documented to influence cortisol values . This was partly controlled by using samples taken on awakening between 6 am and 8 am. The adolescents' compliance was also an issue of concern, as it might have been weakened by the high burden of measurements in HELENA. Sometimes individuals forgot to take a sample and there was no guarantee of correct timing (immediate after awakening) when samples were provided. The issue of protocol compliance for saliva sampling has previously been outlined for children and adolescents  and in adults . Samples were taken on different weekdays, another factor that influences cortisol values . To what extent these methodological shortcomings have influenced cortisol values and further analyses in this study is difficult to evaluate. Therefore, it is recommended that salivary sampling protocols are strictly standardized in future research (a particular challenge in epidemiological surveys).
The present study demonstrated an acceptable internal reliability and scale construct of the ASQ when it was implemented in a European adolescent sample. In addition, significant independent contributions of gender and pubertal stage to the variance in the ASQ were established. These strengths were counterbalanced by a poor test-retest reliability and criterion validity of the ASQ, probably because of methodological shortcomings, and failure to demonstrate effects of socio-economic status. Based on these shortcomings, the utility of the ASQ within an European adolescent sample is uncertain and further research concerning these aspects is required. In addition, supplementing the ASQ with questions concerning previous (severe) lifetime stressful events is recommended in order to have a more complete picture of adolescent chronic stress exposure.
Healthy Lifestyle in Europe by Nutrition in Adolescence
Adolescent Stress Questionnaire
Healthy Lifestyle in Europe by Nutrition in Adolescence Cross-Sectional Study
Baseline Wake-up Salivary Free
Hierarchical Linear Models
Confirmatory Factor Analysis
Comparative Fit Index
Root Mean Square Error of approximation
Post-Traumatic Stress Disorder
De Vriendt T, Moreno LA, De Henauw S: Chronic stress and obesity in adolescents: scientific evidence and methodological issues for epidemiological research. Nutr Metab Cardiovasc Dis. 2009, 19: 511-519. 10.1016/j.numecd.2009.02.009.
Steinbeck K, Baur L, Cowell C, Pietrobelli A: Clinical research in adolescents: challenges and opportunities using obesity as a model. Int J Obes. 2009, 33: 2-7. 10.1038/ijo.2008.263.
Colten ME, Gore S: Adolescent stress: causes and consequences. 1991, New Jersey: Aldine Transaction
Grant KE, Compas BE, Thurm AE, Mcmahon SD, Gipson PY: Stressors and child and adolescent psychopathology: Measurement issues and prospective effects. J Clin Child Adolesc Psychol. 2004, 33: 412-425. 10.1207/s15374424jccp3302_23.
Moreno LA, Gonzalez-Gross M, Kersting M, Molnár D, De Henauw S, Beghin L, et al: Assessing, understanding and modifying nutritional status, eating habits and physical activity in European adolescents: The HELENA (Healthy Lifestyle in Europe by Nutrition in Adolescence) Study. Public Health Nutr. 2008, 11: 288-299.
Byrne DG, Davenport SC, Mazanov J: Profiles of adolescent stress: The development of the adolescent stress questionnaire (ASQ). J Adolesc. 2007, 30: 393-416. 10.1016/j.adolescence.2006.04.004.
Moreno LA, De Henauw S, Gonzalez-Gross M, Kersting M, Molnar D, Gottrand F, et al: Design and implementation of the Healthy Lifestyle in Europe by Nutrition in Adolescence Cross-Sectional Study. Int J Obes. 2008, 32 (Suppl 5): S4-11.
Beghin L, Castera M, Manios Y, Gilbert CC, Kersting M, De Henauw S, et al: Quality assurance of ethical issues and regulatory aspects relating to good clinical practices in the HELENA Cross-Sectional Study. Int J Obes. 2008, 32 (Suppl 5): S12-S18.
Currie C, Molcho M, Boyce W, Holstein B, Torsheim T, Richter M: Researching health inequalities in adolescents: the development of the Health Behaviour in School-Aged Children (HBSC) family affluence scale. Soc Sci Med. 2008, 66: 1429-1436. 10.1016/j.socscimed.2007.11.024.
Styne DM: Clinical paediatric endocrinology. 1995, Oxford: Blackwell Science, 3
Charmandari E, Tsigos C, Chrousos G: Endocrinology of the stress response. Ann Rev Physiol. 2005, 67: 259-284. 10.1146/annurev.physiol.67.040403.120816.
Jessop D: Measurement and meaning of salivary cortisol: A focus on health and disease in children. Stress. 2008, 11: 1-14. 10.1080/10253890701365527.
Gaab J, Rohleder N, Nater UM, Ehlert U: Psychological determinants of the cortisol stress response: the role of anticipatory cognitive appraisal. Psychoneuroendocrinology. 2005, 30: 599-610. 10.1016/j.psyneuen.2005.02.001.
Chida Y, Steptoe A: Cortisol awakening response and psychosocial factors: A systematic review and meta-analysis. Biol Psychol. 2009, 80: 265-278. 10.1016/j.biopsycho.2008.10.004.
Steptoe A, Cropley M, Griffith J, Kirschbaum C: Job strain and anger expression predict early morning elevations in salivary cortisol. Psychosom Med. 2000, 62: 286-292.
Stone AA, Schwartz JE, Smyth J, Kirschbaum C, Cohen S, Hellhammer D, et al: Individual differences in the diurnal cycle of salivary free cortisol: a replication of flattened cycles for some individuals. Psychoneuroendocrinology. 2001, 26: 295-306. 10.1016/S0306-4530(00)00057-3.
Hruschka DJ, Kohrt BA, Worthman CM: Estimating between- and within-individual variation in cortisol levels using multilevel models. Psychoneuroendocrinology. 2005, 30: 698-714. 10.1016/j.psyneuen.2005.03.002.
Young DS, Huth EJ: SI Units for Clinical Measurement. 1998, Philadelphia: American College of Physicians
Groschl M, Rauh M, Dorr HG: Circadian rhythm of salivary cortisol, 17 alpha-hydroxyprogesterone, and progesterone in healthy children. Clin Chem. 2003, 49: 1688-1691. 10.1373/49.10.1688.
Nunnally JC, Bernstein IH: Psychomteric Theory. 1994, New York: McGrw-Hill, 3
Kline P: Handbook of psychological testing. 2000, London: Routledge Francis and Taylor, 2
Hox JJ: Multilevel Analysis. Techniques and Applications. 2002, Mahway, New Jersey: Lawrence Erlbaum Associates
Bentler PM, Bonett DG: Significance Tests and Goodness of Fit in the Analysis of Covariance-Structures. Psychol Bull. 1980, 88: 588-606.
Browne MW, Cudeck R: Alternative Ways of Assessing Model Fit. Sociol Methods & Res. 1992, 21: 230-258.
Hanrahan K, McCarthy AM, Kleiber C, Lutgendorf S, Tsalikian E: Strategies for salivary cortisol collection and analysis in research with children. Appl Nurs Res. 2006, 19: 95-101. 10.1016/j.apnr.2006.02.001.
Matud MP: Gender differences in stress and coping styles. Personality and Individual Differences. 2004, 37: 1401-1415. 10.1016/j.paid.2004.01.010.
Rudolph KD: Gender differences in emotional responses to interpersonal stress during adolescence. J Adolesc Health. 2002, 30: 3-13.
Dahl RE, Gunnar MR: Heightened stress responsiveness and emotional reactivity during pubertal maturation: Implications for psychopathology. Dev Psychopathol. 2009, 21: 1-6. 10.1017/S0954579409000017.
Sumter SR, Bokhorst CL, Miers AC, Van Pelt J, Westenberg PM: Age and puberty differences in stress responses during a public speaking task: Do adolescents grow more sensitive to social evaluation?. Psychoneuroendocrinology. 2010, 35: 1510-1516. 10.1016/j.psyneuen.2010.05.004.
Brady SS, Matthews KA: The influence of socioeconomic status and ethnicity on adolescents' exposure to stressful life events. J Pediatr Psychol. 2002, 27: 575-583. 10.1093/jpepsy/27.7.575.
Lazarus RS, Folkman S: Stress, appraisal and coping. 1984, New York: Springer
Dettenborn L, Tietze A, Bruckner F, Kirshbaum C: Higher cortisol content in hair among long-term unemployed individuals compared to controls. Psychoneuroendocrinology. 2010, 35: 1404-1409. 10.1016/j.psyneuen.2010.04.006.
Hellhammer DH, Wust S, Kudielka BM: Salivary cortisol as a biomarker in stress research. Psychoneuroendocrinology. 2009, 34: 163-171. 10.1016/j.psyneuen.2008.10.026.
Pervanidou P: Biology of post-traumatic stress disorder in childhood and adolescence. Journal of Neuroendocrinology. 2008, 20: 632-638. 10.1111/j.1365-2826.2008.01701.x.
Carlin JB, Hocking J: Design of cross-sectional surveys using cluster sampling: an overview with Australian case studies. Aust N Z J Public Health. 1999, 23: 546-551. 10.1111/j.1467-842X.1999.tb01317.x.
Clow A, Thorn L, Evans P, Hucklebridge F: The awakening cortisol response: methodological issues and significance. Stress. 2004, 7: 29-37. 10.1080/10253890410001667205.
Gow R, Thomson S, Rieder M, Van Uum S, Koren G: An assessment of cortisol analysis in hair and its clinical applications. Forensic Sci Int. 2010, 196: 32-37. 10.1016/j.forsciint.2009.12.040.
Nater UM, Rohleder N: Salivary alpha-amylase as a non-invasive biomarker for the sympathetic nervous system: Current state of research. Psychoneuroendocrinology. 2009, 34: 486-496. 10.1016/j.psyneuen.2009.01.014.
Rohleder N, Nater UM: Determinants of salivary alpha-amylase in humans and methodological considerations. Psychoneuroendocrinology. 2009, 34: 469-485. 10.1016/j.psyneuen.2008.12.004.
Kudielka BM, Broderick JE, Kirschbaum C: Compliance with saliva sampling protocols: Electronic monitoring reveals invalid cortisol daytime profiles in noncompliant subjects. Psychosom Med. 2003, 65: 313-319. 10.1097/01.PSY.0000058374.50240.BF.
Kelly SJ, Young R, Sweeting H, Fischer JE, West P: Levels and confounders of morning cortisol collected from adolescents in a naturalistic (school) setting. Psychoneuroendocrinology. 2008, 33: 1257-1268. 10.1016/j.psyneuen.2008.06.010.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/11/717/prepub
The writing group takes sole responsibility for the content of this article. The authors want to thank the participating adolescents and their parents for their voluntary participation to the HELENA study. The authors also want to acknowledge D. Byrne for his permission to use the Adolescent Stress Questionnaire and the service of BioMedES for copyediting the manuscript. The HELENA Study takes place with the financial support of the European Community Sixth RTD Framework Programme (Contract FOOD-CT-2005-007034). The content of this article reflects only the authors' views and the European Community is not liable for any use that may be made of the information contained therein. Tineke De Vriendt is financially supported by the Research Foundation - Flanders (Grant number: 1.1.746.09.N.01). The authors want to acknowledge the HELENA Study Group for succesfully finalizing this project. A complete list of HELENA Study Group members is provided here:
HELENA Study Group
Co-ordinator: Luis A. Moreno.
Core Group members: Luis A. Moreno, Fréderic Gottrand, Stefaan De Henauw, Marcela González-Gross, Chantal Gilbert.
Steering Committee: Anthony Kafatos (President), Luis A. Moreno, Christian Libersa, Stefaan De Henauw, Sara Castelló, Fréderic Gottrand, Mathilde Kersting, Michael Sjöstrom, Dénes Molnár, Marcela González-Gross, Jean Dallongeville, Chantal Gilbert, Gunnar Hall, Lea Maes, Luca Scalfi.
Project Manager: Pilar Meléndez.
1. Universidad de Zaragoza (Spain): Luis A. Moreno, Jesús Fleta, José A. Casajús, Gerardo Rodríguez, Concepción Tomás, María I. Mesana, Germán Vicente-Rodríguez, Adoración Villarroya, Carlos M. Gil, Ignacio Ara, Juan Revenga, Carmen Lachen, Juan Fernández Alvira, Gloria Bueno, Aurora Lázaro, Olga Bueno, Juan F. León, Jesús Mª Garagorri, Manuel Bueno, Juan Pablo Rey López, Iris Iglesia, Paula Velasco, Silvia Bel, Luis A. Gracia Marco, Theodora Mouratidou.
2. Consejo Superior de Investigaciones Científicas (Spain): Ascensión Marcos, Julia Wärnberg, Esther Nova, Sonia Gómez, Ligia Esperanza Díaz, Javier Romeo, Ana Veses, Belén Zapatera, Tamara Pozo, David Martínez.
3. Université de Lille 2 (France): Laurent Beghin, Christian Libersa, Frédéric Gottrand, Catalina Iliescu, Juliana Von Berlepsch.
4. Research Institute of Child Nutrition Dortmund, Rheinische Friedrich-Wilhelms-Universität Bonn (Germany): Mathilde Kersting, Wolfgang Sichert-Hellert, Ellen Koeppen.
5. Pécsi Tudományegyetem (University of Pécs) (Hungary): Dénes Molnar, Eva Erhardt, Katalin Csernus, Katalin Török, Szilvia Bokor, Mrs. Angster, Enikö Nagy, Orsolya Kovács, Judit Répasi.
6. University of Crete School of Medicine (Greece): Anthony Kafatos, Caroline Codrington, María Plada, Angeliki Papadaki, Katerina Sarri, Anna Viskadourou, Christos Hatzis, Michael Kiriakakis, George Tsibinos, Constantine Vardavas, Manolis Sbokos, Eva Protoyeraki, Maria Fasoulaki.
7. Institut für Ernährungs- und Lebensmittelwissenschaften - Ernährungphysiologie. Rheinische Friedrich Wilhelms Universität (Germany): Peter Stehle, Klaus Pietrzik, Marcela González-Gross, Christina Breidenassel, Andre Spinneker, Jasmin Al-Tahan, Miriam Segoviano, Anke Berchtold, Christine Bierschbach, Erika Blatzheim, Adelheid Schuch, Petra Pickert.
8. University of Granada (Spain): Manuel J. Castillo, Ángel Gutiérrez, Francisco B Ortega, Jonatan R Ruiz, Enrique G Artero, Vanesa España, David Jiménez-Pavón, Palma Chillón, Cristóbal Sánchez-Muñoz, Magdalena Cuenca
9. Istituto Nazionalen di Ricerca per gli Alimenti e la Nutrizione (Italy): Davide Arcella, Elena Azzini, Emma Barrison, Noemi Bevilacqua, Pasquale Buonocore, Giovina Catasta, Laura Censi, Donatella Ciarapica, Paola D'Acapito, Marika Ferrari, Myriam Galfo, Cinzia Le Donne, Catherine Leclercq, Giuseppe Maiani, Beatrice Mauro, Lorenza Mistura, Antonella Pasquali, Raffaela Piccinelli, Angela Polito, Raffaella Spada, Stefania Sette, Maria Zaccaria.
10. University of Napoli "Federico II" Dept of Food Science (Italy): Luca Scalfi, Paola Vitaglione, Concetta Montagnese.
11. Ghent University (Belgium): Ilse De Bourdeaudhuij, Stefaan De Henauw, Tineke De Vriendt, Lea Maes, Christophe Matthys, Carine Vereecken, Mieke de Maeyer, Charlene Ottevaere, Inge Huybrechts.
12. Medical University of Vienna (Austria): Kurt Widhalm, Katharina Phillipp, Sabine Dietrich, Birgit Kubelka Marion Boriss-Riedl.
13. Harokopio University (Greece): Yannis Manios, Eva Grammatikaki, Zoi Bouloubasi, Tina Louisa Cook, Sofia Eleutheriou, Orsalia Consta, George Moschonis, Ioanna Katsaroli, George Kraniou, Stalo Papoutsou, Despoina Keke, Ioanna Petraki, Elena Bellou, Sofia Tanagra, Kostalenia Kallianoti, Dionysia Argyropoulou, Katerina Kondaki, Stamatoula Tsikrika, Christos Karaiskos.
14. Institut Pasteur de Lille (France): Jean Dallongeville, Aline Meirhaeghe.
15. Karolinska Institutet (Sweden): Michael Sjöstrom, Jonatan R Ruiz, Francisco B. Ortega, María Hagströmer, Anita Hurtig Wennlöf, Lena Hallström, Emma Patterson, Lydia Kwak, Julia Wärnberg, Nico Rizzo.
16. Asociación de Investigación de la Industria Agroalimentaria (Spain): Jackie Sánchez-Molero, Sara Castelló, Elena Picó, Maite Navarro, Blanca Viadel, José Enrique Carreres, Gema Merino, Rosa Sanjuán, María Lorente, María José Sánchez.
17. Campden BRI (United Kingdom): Chantal Gilbert, Sarah Thomas, Elaine Allchurch, Peter Burgess.
18. SIK - Institutet foer Livsmedel och Bioteknik (Sweden): Gunnar Hall, Annika Astrom, Anna Sverkén, Agneta Broberg.
19. Meurice Recherche & Development asbl (Belgium): Annick Masson, Claire Lehoux, Pascal Brabant, Philippe Pate, Laurence Fontaine.
20. Campden & Chorleywood Food Development Institute (Hungary): Andras Sebok, Tunde Kuti, Adrienn Hegyi.
21. Productos Aditivos SA (Spain): Cristina Maldonado, Ana Llorente.
22. Cárnicas Serrano SL (Spain): Emilio García.
23. Cederroth International AB (Sweden): Holger von Fircks, Marianne Lilja Hallberg, Maria Messerer
24. Lantmännen Food R&D (Sweden): Mats Larsson, Helena Fredriksson, Viola Adamsson, Ingmar Börjesson.
25. European Food Information Council (Belgium): Laura Fernández, Laura Smillie, Josephine Wills.
26. Universidad Politécnica de Madrid (Spain): Marcela González-Gross, Jara Valtueña, David Jiménez-Pavón, Ulrike Albers, Raquel Pedrero, Agustín Meléndez, Pedro J. Benito, Juan José Gómez Lorente, David Cañada, Alejandro Urzanqui, Rosa María Torres, Paloma Navarro.
The authors declare that they have no competing interests.
TDV and SDH developed the design of the stress measurement. LM coordinated the total HELENA project on international level. LM, YM and SDH were involved in the design of the HELENA project and locally coordinated the HELENA project. TDV, PB, EN, SD, GVR organized the fieldwork and performed the data collection locally. EC was involved in manuscript drafting and coordinated the statistical analysis. All authors have reviewed the manuscript, gave input and made significant improvements.
About this article
Cite this article
De Vriendt, T., Clays, E., Moreno, L.A. et al. Reliability and validity of the Adolescent Stress Questionnaire in a sample of European adolescents - the HELENA study. BMC Public Health 11, 717 (2011). https://doi.org/10.1186/1471-2458-11-717
- Summary Score
- Salivary Cortisol
- Stress Module
- Pubertal Stage