Using a gamified monitoring app to change adolescents’ snack intake: the development of the REWARD app and evaluation design
© The Author(s). 2016
Received: 3 June 2016
Accepted: 9 July 2016
Published: 5 August 2016
As the snacking pattern of European adolescents is of great concern, effective interventions are necessary. Till now health promotion efforts in children and adolescents have had only limited success in changing adolescents’ eating patterns and anthropometrics. Therefore, the present study proposes an innovative approach to influence dietary behaviors in youth based on new insights on effective behavior change strategies and attractive intervention channels to engage adolescents. This article describes the rationale, the development, and evaluation design of the ‘Snack Track School’ app. The aim of the app is to improve the snacking patterns of Flemish 14- to 16-year olds.
The development of the app was informed by the systematic, stepwise, iterative, and collaborative principles of the Intervention Mapping protocol. A four week mHealth intervention was developed based on the dual-system model with behavioral change strategies targeting both the reflective (i.e., active learning, advance organizers, mere exposure, goal-setting, monitoring, and feedback) and automatic processes (i.e., rewards and positive reinforcement). This intervention will be evaluated via a controlled pre-post design in Flemish schools among 1400 adolescents.
When this intervention including strategies focused on both the reflective and automatic pathway proves to be effective, it will offer a new scientifically-based vision, guidelines and practical tools for public health and health promotion (i.e., incorporation of learning theories in intervention programs).
NCT02622165 registrated November 15, 2015 on clinicaltrials.gov.
KeywordsAdolescents Nutrition Snacks Diet Obesity Overweight Learning theories Conditioning Dual-system model App mHealth Intervention
The dietary pattern of European adolescents with a high intake of energy-dense low-nutritious foods and a low intake of essential food groups is of great concern [1, 2]. In Flanders, 27 % of adolescents consume sweet snacks on a daily basis, and snacks between meals account for 20–24 % of their total energy intake [3, 4]. A recent study among Flemish 14- to 16-year-olds indicated a higher intake of unhealthy –fat- and/or sugar-rich– snacks (214.4 ± 147.3 g) than healthy snacks (122.0 ± 133.4 g) . Given the association between unhealthy snacking and the development of obesity, dental carries and other chronic diseases during adolescence and later in life [6–8], attention to healthy snacking in adolescents is key.
Based on the bio-psycho-social model , the development of an effective health promotion intervention requires a multidisciplinary approach addressing multiple contexts both at the individual and environmental level. Most health promotion efforts to improve dietary behaviors in youth have focused mostly on schoolchildren and their families in enhancing knowledge and changing well-known environmental factors in order to improve dietary behaviors [10, 11]. However, till now these school- and family-based multicomponent interventions have had only limited success in changing adolescents’ eating patterns and anthropometrics [12–14]. Therefore, new approaches to influence dietary behaviors in youth should be considered based on new insights on effective behavior change strategies and attractive intervention channels to engage adolescents.
Michie and colleagues  conducted a meta-regression to identify the most effective techniques derived from different behavioral change theories on healthy eating and physical activity interventions and found that interventions combining self-monitoring with at least one other technique derived from the Control Theory (i.e., goal setting, feedback, review goals) of Carver and Scheier  were more effective than other interventions. In addition, an innovative in-depth analysis of theoretical explanations for behavior change following interventions (i.e., a comparison between the theoretical basis of an intervention, the delivery in practice = dose delivered, and the intervention receipt = dose received/exposure in participants) by the same authors  indicated a clear theoretical difference between intervention delivery and receipt. Participants reported a lower exposure than the dose delivered for all used theoretical methods apart from those based on operant theory. This latter result highlights the important role of operant techniques (i.e., rewarding/positive reinforcement) in behavior change.
Next, also personality theories provide important insight as they focus on differences between individuals. An important gap in intervention research is the effectiveness of interventions that take into account individual differences in personality to promote healthy diets . This is unfortunate as unhealthy eating behaviors, such as the intake of energy dense snacks, are often driven by hedonic motives like eating for the palatability or the reward value of food in the absence of hunger rather than by homeostatic hunger or eating out of biological needs [18, 19]. Reward Sensitivity (RS) is a bio-psychological concept which reflects one’s ability to experience pleasure or rewarding feelings when exposed to positive (appetitive/palatable) stimuli  and has been found to play a critical role in eating unhealthy food products, overeating and becoming overweight or obese [5, 21, 22]. During adolescence, RS has an increased influence on behavior as the reward processing peaks while regulative control matures at a slower pace . Taking into account RS in preventive health promotion among adolescents by using reward-based strategies might increase intervention effects in this population. However, little is known about the specific role of RS in food cue reactivity and learning processes to adopt healthy food preferences and healthy nutrition behavior. Therefore, the REWARD-project aimed to develop a new approach to motivate adolescents to opt for healthy snack choices by using reward-based learning and taking into account individual differences in RS.
Although recent insight on the role of learning theories in behavior change has increased considerably, we have to take care not to rely too much on a simplistic perspective. Kremers and colleagues  indicated that nutrition behaviors are the result of a joint function between conscious and unconscious processes. Dual-system models or dual-process models explain health behaviors as two interconnected mental systems, each operating according to different principles [24, 25]: a top-down reflective system including elaboration and cognitive efforts to build beliefs and decisions and a bottom-up impulsive/automatic system (i.e., habits) in which certain stimuli or cues are linked to certain behaviors based on earlier learned associations [11, 26]. Based on previous research and theoretical evidence [11, 24–26], it was decided to incorporate behavioral change methods and strategies in the intervention to influence the reflective pathway in addition to the reward-strategies that target the more automatic pathway. This broad framework is attractive as it allows to include other models that focus on unique determinants like personality theory and operant learning models that fit well with the bottom-up perspective as well as the Control theory that focuses more on the reflective pathway.
Finally, besides using new theoretical approaches, it is also warranted to revise the intervention channel to reach adolescents. Traditional channels, via school, family, and/or community, had only limited success in previous studies [12–14]. We therefore assume that intervention approaches imbedded in the actual world of adolescents are needed. Smartphones have a great potential to reach large numbers of adolescents as they have become an integral part of adolescents’ daily life and usage rates are rising significantly in European adolescents . Recent systematic reviews have shown that mHealth interventions are promising in changing health behaviors in youth [28–30]. Additional advantages are the cost-effectiveness of the dissemination, lowered participant burden, flexible program tailoring, data for self-monitoring, and more visually appealing and engaging multimedia modalities . Therefore, it was decided to deliver the intervention as an mHealth intervention.
This paper describes the rationale, design and methods of a four-week controlled trial designed to measure the effectiveness of an mHealth intervention named the ‘Snack Track School’ to improve adolescents’ snacking behavior. The first hypothesis is that the intervention will increase the healthy snack intake and reduce the unhealthy snack consumption (i.e., measured by a healthy snack index) of adolescents compared to the control group. Secondary hypotheses are that the intervention will 1) decrease the age- and gender-adjusted Body Mass Index, 2) positively influence socio-cognitive factors and habits related to healthy snacking, and 3) affect high reward-sensitive adolescents’ snacking behavior more than low reward-sensitive adolescents.
Study design and setting
The design consists of a controlled pre‐post design. The intervention will be organized in three secondary schools in one city in Flanders (Belgium), while three schools from a similar (matched) city (comparable socio‐economical characteristics, population density, size) will be selected as a control setting. The adolescents in the intervention schools will receive a four-week mobile intervention while the control schools will continue their usual practices. The full study period will consist of a baseline test, the four-week ‘The Snack Track School’ intervention (or 4 week control), and a post-test immediately after the intervention.
Study population and recruitment
The study population will consist of 14- to 16- year-old Flemish adolescents (i.e., 3rd- 4th grade of Flemish secondary schools). The necessary sample size for this study was calculated based on the healthy snacking index (i.e., the primary outcome of the intervention study). Assuming an intraclass correlation (ICC) of 0.02 at school level and an ICC of 0.03 at class level with a mean and standard deviation of the healthy snacking index of 37.8 ± 20.2, at least 12 classes with 15 adolescents per school (three intervention schools and three control schools) are needed to detect a difference of 20 % between intervention and control at the 5 % significance level with a power of 80 % . The ICC’s, mean and standard deviation regarding the healthy eating index were based on the earlier cross-sectional REWARD study and the test-retest of the REWARD Food Frequency Questionnaire (FFQ) to measure snack intake (, De Cock unpublished Data]. To account for possible loss to between pre and post an oversampling of 33 % will be applied. A total sample of 1437 adolescents (control and intervention) in 24 classes per school will be recruited. No exclusion criteria will be applied.
Following the acceptance of the school to participate in the study, parents will receive a letter explaining the purpose of the study and asking for passive consent for participation of their child in the study. In addition, consent of the adolescent will be asked at the beginning of the study.
The study adheres to the Helsinki declaration and the conventions of the council of Europe on human rights and biomedicine. Approval for the intervention study was provided by the Medical Ethics Committee of the University Hospital Ghent and the University of Leuven.
The ‘Snack Track School’ intervention
As earlier mentioned, the ‘Snack Track School’ intervention includes behavioral change strategies to influence both the automatic pathway as well as the reflective pathway. The systematic, stepwise, iterative, and collaborative principles of the Intervention Mapping protocol  were used to put this idea in practice. The intervention was preceded by an extensive problem analysis (i.e., needs assessment) using existing and newly generated evidence, including a cross-sectional survey, experimental and focus group research (, De Cock Unpublished data).
Following the Intervention Mapping protocol, different stakeholders were involved in the intervention development: adolescents, teachers, principals, stakeholders from food industry, professional organizations active in health promotion, community members, umbrella school organizations. Adolescents were involved in both the conceptual (i.e., a large scale focus group research among 101 adolescents) as well as the pretesting phase of the intervention (i.e., via test labs in smaller groups of four to eight students to regularly gather feedback regarding feasibility, usability, and attractiveness of the app). Monthly teacher contacts (two per participating school) were also included during the process to ensure that the app format was fully compatible with existing pedagogic guidelines, expectations, and school programs. Other stakeholders (i.e., food industry, professional organizations active in health promotion, community members, umbrella school organizations) were frequently consulted through stakeholder meetings to explore if and affirm that the current intervention can be disseminated to the wider society. A strong participation of these actors was used in the different phases of intervention development to ensure a culturally-, age- and community-relevant intervention which might increase the likelihood of program success.
The dual-system model , a model explaining health behaviors as a joint function between conscious and unconscious processes, was chosen as the broad framework of our intervention, as it allows to include other models like personality models, operant learning models and the Control theory. First, the dual model suggests that behavior change has to take into account the bottom-up impulsive/automatic system (i.e., habits) in which certain stimuli or cues are linked to certain behaviors based on earlier learned associations. Moreover, this fits with personality theory that describes a high peak of reward sensitivity in adolescence and its important and understudied role in nutrition behavior. However, this also fits with operant models of behavior change that argue that reward-based strategies (i.e., provision of rewards and positive reinforcement) are needed as the main behavioral change strategies.
The ‘Snack Track School’ app is a four week intervention and constitutes of a virtual high school environment with typical school locations (i.e., classrooms, gym hall, bike area, bathrooms, etcetera) in which all participants have their own locker, their own Snack Track Tool, and their own ‘anonymous’ avatar, which they can customize as preferred based on several options (i.e., many options for head of character, clothes, shoes, gadgets). During the four-week intervention, every week of snack monitoring has its own story line and challenges.
The central idea of the ‘Snack Track School’ intervention is that participants earn credits/points based on the nutritional value of the snacks they consume (i.e., positive reinforcement/providing rewards to influence automatic behavior). For every snack the adolescents enter in the app via the ‘Snack Track’ tool, they will be awarded by points. The healthier the snacks they consume, the more points they will receive. These credits will be responsible for the progress in the weekly challenges of the four-week app. No negative points will be provided for unhealthy snacks as punishment can have contra-productive effects . A large snack database was constructed based on the Internubel Trade Name database . Points will be awarded according to the UK Ofcom Nutrient Profile model .
In order to stimulate a well-balanced snacking pattern and not merely the tracking of as many snacks as possible, some gratuities and limitations are built in the app related to the credit system. Participants are able to track as many snacks as they want, however, only the credits of the first ten snacks are included. Three gratuities are included: 1) a bonus for a snack intake ≤ six snacks per day, 2) a bonus for a snack intake of ≥2/3 healthy snacks per day, and 3) a bonus for non snackers that are involved in the app (logging in ≥3 times in the app per day). The cut-offs of six snacks and more than 2/3 healthy snacks were based on Flemish guidelines of necessary food and nutrient intake . It is advised to consume maximum 10 % energy from snacks (defined here as unhealthy snacks). Fourteen to 16- year-old girls and boys have a general advised total energy intake per day of 2100 and 2600 kcal, respectively . This corresponds with a maximum intake of 210–250 kcal from unhealthy snacks (i.e., one or two unhealthy snacks per day depending on the size and nutritional content). Additionally, it is advised for adolescents to eat three pieces of fruit and four dairy products per day (i.e., not only via snacks but also via meals) . The cut-off of maximum six snacks and more than 2/3 healthy snacks per day was deduced from these guidelines, an intake of three or four healthy snacks together with one or two unhealthy snacks is acceptable for adolescents as they need a lot of energy to grow.
The following theoretical methods were included in the app to affect the reflective pathway including knowledge, attitude, and self-efficacy. Active learning and advance organizers based on the Elaboration Likelihood Model were included to enhance the knowledge of the nutritional quality of a snack item. The Elaboration Likelihood Model  suggests that people only process information centrally (i.e., careful consideration which can increase knowledge) when they are motivated to do so, when information is personally relevant, and when the information is presented repeatedly. Through the provision of credits in the app and the progress in the app linked to these credits, participants will unconsciously learn about the healthiness of a snack. The credit system used in the app can be seen as an external motivator to change their snacking behavior. Insights about the (un)healthiness of snack items based on this credit system can stimulate central processing of this information as more knowledge about the specific credits per snack item and the application of this knowledge can enhance their progress in the app. These points are also personally relevant (i.e., linked to the own snacks consumed and related to progress in the app) and are repeatedly presented. In addition, this healthy-unhealthy continuum ranging from 0 to 50 and the placement of the different snack items on this continuum based on the credits, can act as a mental reminder (i.e., a mental axis) regarding the nutritional value of snack items using the method of advance organizers. Furthermore, the earlier mentioned gratuities and limitations also learn adolescents about a well-balanced snacking pattern.
The method of mere exposure is used to activate positive attitudes regarding healthy snacks. Zajonc  indicated that participants develop a more positive attitude about stimuli following repeated exposure. In our intervention more exposure to healthy snacks is expected as participants receive more credits for the healthy choice, i.e. the expected behavior, which subsequently can lead to increased exposure and thus more liking or positive attitude even if they are not consciously aware of the process . Moreover, the association between positive reinforcement and healthy snacks can also result in more positive attitudes .
Self-efficacy will be targeted via goal setting, monitoring, and feedback. Goal setting has a high likelihood to improve health behaviors because persons who set goals exert themselves to a greater extent, persevere in their aims, concentrate more, and develop strategies to be able to perform the necessary behavior [16, 45]. Adolescents will choose a goal per week regarding improving their snacking habits. The chosen goal will be evaluated per day and extra bonus credits/rewards will be provided in case of success. At the end of every week, feedback will be provided via a week-report that portrays all consumed snacks per weekday with their matching credits. This week-report will also provide feedback about the fail or success of reaching their goal. In case of success (i.e., linked to the goals), it will increase self-efficacy/capability of eating more healthy snacks. In case of failure, the feedback and the pending reward can stimulate elaboration and another attempt, and a possible increase in self-efficacy after success in the future . Goal setting will be applied from week 2 till week 4. At the beginning of these three weeks, participants will be provided four goals. They need to choose one specific week goal, which they need to reach every day during that week. In case of success, a bonus of 150 points will be won at the end of the day. Earlier research has indicated that self-monitoring is the most successful behavioral change technique in energy balance-related intervention research . Participants need to enter every consumed snack in the app and can monitor their snack intake in a weekly report. Based on the self-monitoring, they will have more insight into and awareness of their snacking behavior. This week-report can create elaboration about the snacking behavior.
To increase participants’ motivation to play our app, the most significant game motivations and dynamics based on the cross-sectional REWARD study, focus group research and previous studies (REWARD unpublished data, [46–48]) were included. Challenge (i.e., to push yourself to a higher level of skill or personal accomplishment), competition (i.e., proving who has the best skills and can react and think the fastest), and social interaction (i.e., playing together with others both online as offline) were found to be important motivations for adolescents to play games. Furthermore, in the cross-sectional REWARD study, adolescents indicated the following characteristics as most crucial: (1) competitive aspects/playing against other people/leader board rankings, (2) cooperation/working together to reach goals, (3) different story outcomes based on your player actions, (4) ‘levelling up’ a game character, and (5) earning points or other rewards (unpublished data). Earlier studies also highlighted the importance of cooperation, and rewards as game dynamics [46–48]. Baranowski and colleagues  and Peng  indicated the importance of story lines to increase immersion and intrinsic motivation to play games. Moreover, the inclusion of three psychological needs based on the Self-Determination Theory [46, 49, 50], namely autonomy, competence, and relatedness, have been linked to increased intrinsic motivation to play games. From this viewpoint, it is necessary that participants can make their own choices in a game that lead to different endings (i.e., autonomy), can customize their own avatar according to their preferences (i.e., autonomy and relatedness), have indicators for their achievements as this better satisfies players’ needs of competence.
Implementation and evaluation of the intervention
The four-week ‘Snack Track School’ app will be evaluated via a matched controlled pre‐post- design including a baseline test and post-test immediately after the end of the intervention. The app will be launched via the schools but will not be imbedded in the school curriculum, the app is a stand-alone intervention that can be used by the adolescents without involvement of the teachers. The intervention will be conducted in the spring of 2016.
Weekly process evaluation moments will provide insight into participation of adolescents in the app and will also be used as a moment to encourage participation. During the study period, teachers and students will be able to contact the researchers by phone or email. If necessary, visits of the researchers will be possible in addition to the weekly evaluation moments. In addition, a help function is built in the app so participants can always contact the researchers and game developers in case of technical or other problems. Smartphones (Nokia Lumia 435) will be provided to the adolescents without (functional) smartphone so all adolescents in the intervention schools can participate.
Data collection and outcome measurements
Primary outcome measure
The primary outcome consists of the Healthy Snacking Index. A Food Frequency Questionnaire (FFQ) was developed to measure average snack consumption in adolescents via 28 snack items. This new FFQ is based on an earlier FFQ instrument for children developed by Huybrechts and colleagues , but includes portion sizes that match adolescents’ intake patterns. Adolescents will indicate their answer in a list of six frequency categories, namely never or seldom; 1–3 days/month; 1 day/week; 2–4 days/week; 5–6 days/week; every day . The FFQ contains four to six daily portion size categories per snack item and a list of common standard measures as examples. Snacks are in this study defined as all food items consumed outside (>30 min) of breakfast, lunch and dinner, in accordance to Rodriguez and Moreno’s definition of snacking . The 28 snack items assessed in the FFQ are: dried fruit, fruit, raw vegetables, nuts and seeds, chocolate and pralines, candy bars, candy, dry cookies, other cookies such as chocolate cookies, breakfast rolls, pastries, breakfast cereals, unsweetened yoghurt, sweetened yoghurt, pudding, mousses, ice-cream, popsicles, sandwiches with sweet or savoury spread, cheese or meat cubes, chips and similar products, other savoury snacks such as bread sticks, sausage/cheese rolls and pizza, other fried snacks such as spring rolls and cheese croquettes, fries, kebab, hamburgers and pasta cups.
The classification of snacks and drinks into healthy and unhealthy was based on the nutrient profiling model as developed by the UK NP Ofcom model . This model calculates for each food product a score that represents its healthiness In order to apply this model, an average nutrient composition per FFQ category was needed to apply this nutrient profile model. The average nutrient composition per FFQ category was calculated by averaging the nutritional composition (obtained from the Belgian food composition table  expressed per 100 g) of the most frequently consumed food items by adolescents within that category, as reported in the HELENA study . The average energy, sugar, and fat and sodium intakes per FFQ category were then calculated by multiplying the amounts (g) of the food consumed and the average nutritional values expressed per g (the average values per 100 g divided by 100) . According to the average amount of sodium, sugar, fat and kJoules, fibre, proteins and the portion of fruit and vegetables or nuts present in the product the NP model gives the food product a score, if a food product scores more than 4 points it was considered unhealthy. The FFQ items crisps, other salty snacks, sausage/cheese rolls and pizza, other fried snacks, fries, hamburgers, cheese or meat cubes, icecream, popsicles, breakfast cereals, pudding, sandwiches with sweet or savory spread, mousses, chocolate, candy bars, candy, dry cookies, other cookies, breakfast rolls and pastries were considered to be unhealthy.
The daily intake of each FFQ category will be obtained by multiplying the frequency of consumption with the quantity of consumption per week (g) divided by 7. These daily intakes, will then be summed to obtain the daily intake of healthy snacks (g), and unhealthy snacks (g). Finally a health index for snacks will be calculated: (gram healthy snacks/‘gram healthy snacks + gram unhealthy snacks))*100. The reliability and validity of this FFQ to assess the healthy snack ratio for interventions purposes were tested and reported elsewhere [De Cock unpublished]. The healthy snack index’ reliability was good and the validity was acceptable.
Secondary outcome measure
Two trained research assistants will measure body height and weight according to a standardized protocol. Adolescents will be measured without shoes and will be allowed to wear light clothing, such as a t-shirt and shorts/short pants. Body height will be measured with a Seca Leicester Portable stadiometer with an accuracy of 0.1 cm. Weight will be measured with a calibrated electronic scale SECA 861 with an accuracy of 0.1 kg. Two readings of each measurement (weight and height) will be obtained to assure accuracy. If the two readings differ more than 1 %, a third measurement will be taken. These three measurements will be recorded and the outlier will be excluded during the data cleaning process.
Awareness about the healthiness of the adolescents’ snacking and intention to change the snacking behavior within the next six months will be assessed using one question with a five-point answer format.
Attitude will be measured with five items in which adolescents’ opinion will be asked on statements linking healthy snacks to taste and health.
Self-efficacy will be assessed via three items asking adolescents how hard it is to eat healthy snacks in general and in two more specific situations (at home, and at school).
Habit will be assessed by a four-item automaticity subscale (the ‘Self-Report Behavioral Automaticity Index’ ) based on the twelve-item Self-Report Habit Index (SRHI) . This subscale was found to be reliable and sensitive to detect the habit-behavior association and moderation of the intention-behavior relationship in energy balance-related behavior domains .
Knowledge about the healthiness of specific snacks will be measured by means of a scoring test. Adolescents will be asked to rate the healthiness of each FFQ item (28 in total) by giving it a score ranging from 0 to 100. Zero represents “very unhealthy” and 100 “very healthy”. Afterwards the results will be compared with the actual score of the 28 snack items calculated by means of the UK NP Ofcom model .
Perceived peers’ snacking behavior, peers’ social support, social pressure, and subjective norm regarding healthy snacks will be measured by valid and reliable items based on the HELENA and ENERGY study [55, 58].
Other measurements for explorative research including mediators and moderators
Adolescent characteristics, including gender, date of birth, ethnicity, and family status, socio-economic status of the family, via the occupational social class of father and mother and the Family Affluence Scale (FAS)  will be assessed.
Overview of other measurements for explorative research
Environmental variables/determinants related to dietary behaviors
Snack availability at home
The home availability of the 28 snack items used in the FFQ.
The constructs of these questions are based on valid questions from the European HELENA study .
Perceived peers’ snacking behavior, peers’ social support, social pressure, and subjective norm regarding healthy snacks.
Parents’ modeling, rules at home, and monitoring in relation to snacks.
Personality traits/biological factors
Reward sensitivity and punishment sensitivity
The Dutch child version of the Carver and White’s Behavioral Inhibition System (BIS)/Behavioral Approach System (BAS) - scale as developed by Franken and colleagues .
The five-items subscale restraint from the Child Eating Disorder Examination Questionnaire (ChEDE-Q ).
Research demonstrated the reliability and validity of the ChEDE-Q for examining eating pathology in youngsters from the general population  and in clinical samples of treatment seeking obese youngsters .
The Pubertal Development Scale  which is a five-item self-report questionnaire to ascertain pubertal status in adolescents.
Other measurements on nutrition behaviors
Total energy intake per day
Adolescents’ meal patterns:
The frequency of eating breakfast, lunch and dinner, the frequency of snacking, and the source from which they usually obtain the snacks.
All questions originate from the validated questionnaires of the European HELENA study .
Game behavior-related variables
Duration and frequency of game play
Duration will be measured via a timeline separately for each day of the week. Frequency will be assessed by using a 7-point Likert scale from ‘almost never’ to ‘almost every day’. Respondents are asked to indicate how often they play games on any device.
Both methods have been used in previous studies to measure game use .
Preferences for game genres
This will be measured separately for computer/console games and smartphone/tablet games. For both categories a list of 13 game genres has been developed based on previous studies and current popular game genres [74, 75]. Given that new games and even new game genres arise every year, the list was updated to the current situation.
The Uses and Gratifications Questionnaire for game developed by Sherry et al. .
The Game Engagement Questionnaire of Brockmeyer and colleagues .
This validated instrument consists of 19 items from which a total Game Engagement Score can be derived in order to examine one’s involvement in games.
A 7-item Game Addiction Scale  of which each item corresponds with one of the seven criteria for pathological gambling according to the DSM: salience, tolerance, mood modification, withdrawal, relapse, conflict and problems.
Smartphone and tablet use
Respondents will be asked to indicate whether they use a smartphone or tablet (yes or no). Given that our serious game can be played on either of these devices it is important to take into account previous experience with the devices.
Adolescents will be asked to indicate how important several structural characteristics of video games are for their game experience. King, Delfabbro & Griffiths  recently developed a list of game dynamics based on psychological structures. The researchers have analyzed games and made an overview of game dynamics that are engaging to game players, which they tested on a broad sample of various age groups.
School management questionnaire
At the pretest, the principal of every school will be asked to complete a questionnaire assessing the food policy at school. This school management questionnaire is based on the school management questionnaire of the ENERGY project . Since the REWARD project only focuses on healthy food choices only the parts of ENERGY questionnaire that were related to food facilities are used. The questionnaire comprises of the following sections: general characteristics of the school, the physical environment (opportunities to eat and drink at school), the political environment concerning regulations and practices pertaining to food and drinks and a last section specifically on vending machines (typical sale of snacks and soft drinks).
Overview of the measurements of the different process evaluation concepts
What/how to measure
1a) Reach (participation rate) Description of who participated and who did not
Schools - describe by type and composition (gender, SES, ethnicity)
Adolescents - gender/age/school type/SES
Drop-outs - Description of those participating and those not
2a) Dose delivered (completeness) Implementation of all components
Before the start of the intervention, a presentation will be given to the involved teachers and principals to inform them about the intervention and the weekly process evaluation moments.
A weekly process evaluation moment will be conducted in the classroom by the teachers or researchers depending on schools’ and teachers’ preference.
During this weekly moment, a classical discussion will be held to evaluate the previous game week, to ask about possible problems, to introduce next week’s week story/challenge and to highlight the importance of continuing with the intervention/game. Weekly logs will be held to capture this process.
2b) Dose received (exposure) Participants engagement, interaction, initial and/or continued use
Dose received (exposure) will be measured via two ways: 1) the core game module of the game experience questionnaire that will be included in the post-test questionnaire, and 2) the in-game logs.
2c) Dose received (satisfaction) Satisfaction with intervention
Dose received (satisfaction) will also be measured via the game experience questionnaire.
3) Fidelity Implemented as planned.
Researcher weekly logs - summary of comments/open ended questions on weekly logs
3) Context Physical/social/political contextual factors that might influence implementation/reach
School management questionnaire to describe differences in school systems/organizations and priority of nutrition
Teacher post intervention questionnaire, Principle post-test, and question in adolescent posttest questionnaire on school related factors likely to influence implementation/contamination
During the intervention, weekly process evaluation moments (15 min) will be included in the classroom, these will be implemented by the researchers and will be audiotaped. A process evaluation questionnaire will be added to the adolescent questionnaire at the post-test to assess participants’ engagement and use of the app, and the interaction between peers resulting from playing the app. In addition, adolescents’ dose received and satisfaction will also be assessed through logs of the app. Contextual factors will be measured through a post-test questionnaire for adolescents, the principals of the schools (both control and intervention) and involved teachers.
Data management and analysis
After every data collection wave (baseline, post), adolescent questionnaires will be scanned using the scanning software package TELEForm (version 6.1, Cardiff Software Inc., San Marcos, California, USA). Afterwards data will be translated into an SPSS file.
Descriptive analyses will be used at the individual level to compare the intervention and control group at baseline. Linear mixed models taking into account the clusters (child-class-school) will be used to evaluate differences in the primary and secondary outcome variables. In addition, subgroup analyses, and mediating and moderating analyses will be conducted to provide more insight into intervention effects. The process data (including objectively measured data and self-reported data) will be translated in a variable indicating the degree of intervention implementation and will be linked to the effect evaluation.
The newly developed gamified ‘Snack Track School’ app tries to tackle some of the problems identified in earlier prevention programs as these previous interventions were not successful in changing nutrition behaviors and anthropometrics [12–14]. Previous nutrition interventions were often based on providing general information: all participants received comparable information on nutrition and eating behavior, irrespective of their individual differences. Since interventions taking into account individual differences are expected to have a higher likelihood of being effective in establishing sustainable behavior change [14, 61], the ’Snack Track School’ app aims to motivate adolescents to opt for healthy snack choices by using reward-based learning and taking into account individual differences in RS.
During the development process of the ’Snack Track School’ all the possible intervention success factors based on own and earlier research were gathered, i.e., an intervention focusing on both the automatic and reflective pathway using significant behavior change methods and techniques and an attractive intervention channel for adolescents. By describing RS as a determinant of nutrition behavior and food choices in adolescents and combining this with reward-based learning paradigms in the intervention, we believe an innovative framework is introduced in public health interventions. If this paradigm shift would turn out to be successful, it will offer a new scientifically-based vision, guidelines and practical tools that stakeholders can use to move to better dietary habits in youth.
FFQ, food frequency questionnaire; RS, Reward sensitivity
This study was supported by the agency for Innovation by Science and Technology (IWT) of Flanders (Belgium). The sponsors were not involved in the study design, collection, analysis or interpretation of the data.
Availability of data and materials
LG, KB, CL, LM, JVC, BD, WVL, LV, CB have made substantial contributions to conception and design of this study. WVL, JV, NDC have been involved in drafting the manuscript. All authors have been involved in critically reviewing the paper. All authors read and approved the manuscript.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
The study adheres to the Helsinki declaration and the conventions of the council of Europe on human rights and biomedicine. Approval for the intervention study was provided by the Medical Ethics Committee of the University Hospital Ghent and the University of Leuven. Both parental written passive consent and a verbal active consent of the adolescents will be obtained.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Diethelm K, Janckovic N, Moreno LA, Huybrechts I, De Henauw S, De Vriendt T, Gonzales-Gross M, Leclercq C, Gotrand F, Gilbert CC, the HELENA group. Food intake of European adolescents in the light of different food-based dietary guidelines: results of the HELENA (Healthy Lifestyle in Europe by Nutrition in Adolescence) Study. Public Health Nutr. 2012;15:386–98.View ArticlePubMedGoogle Scholar
- Duffey KJ, Huybrechts I, Mouratidou T, Libuda L, Kersting M, De Vriendt T, Gottrand F, Widhalm K, Dallongeville J, Hallstrom L, the HELENA group. Beverage consumption among European adolescents in the HELENA study. Eur J Clin Nutr. 2012;66:244–52.View ArticlePubMedGoogle Scholar
- Vereecken CA, De Henauw S, Maes L. Adolescents’ food habits: results of the health behavior in school-aged children survey. Br J Nutr. 2005;94(3):423–31.View ArticlePubMedGoogle Scholar
- Matthys C, De Henauw S, Devos C, De Backer G. Estimated energy intake, macronutrient intake and meal pattern of Flemish adolescents. Eur J Clin Nutr. 2003;57:366–75.View ArticlePubMedGoogle Scholar
- De Cock N, Van Lippevelde W, Vervoort L, Vangeel J, Maes L, Eggermont S, Braet C, Lachat C, Huybregts L, Goossens L, Beullens K, Kolsteren P, Van Camp J. Sensitivity to reward is associated with snack and sugar-sweetened beverage consumption in adolescents. Eur J Nutr. 2015.Google Scholar
- Nicklaus S, Remy E. Early origins of overeating: tracking between early food habits and later eating patterns. Curr Obes Rep. 2013;2:179–84.View ArticleGoogle Scholar
- World Health Organization. Report of the World Health Organization Study Group. Diet, Nutrition and the prevention of chronic diseases; 2008. Accessed May 12, 2014. Available from: http://www.who.int/dietphysicalactivity/publications/trs916/en/. Accessed 2 Aug 2016.
- Gopinath B, Louie JCY, Flood VM, Burlutsky G, Hardy LL, Baur LA, Mitchell P. Influence of obesogenic behaviors on health-related quality of life in adolescents. Asia Pac J Clin Nutr. 2014;23:121–7. Ref socio-ecolog model.PubMedGoogle Scholar
- Kolbe L, Story M. Preventing childhood obesity. Washington, DC: National Academies Press; 2005.Google Scholar
- Crosby RA, Kegler MC, DiClemente RJ. Understanding and applying theory in health promotion practice and research. In: DiClemente RJ, Crosby RA, Kegler MC, editors. Emerging theories in health promotion practice and research: strategies for improving the public health. San Francisco: Josey-Bass; 2002.Google Scholar
- Kremers SPJ, de Bruyn G-J, Visscher TLS, van Mechelen W, de Vries NK, Brug J. Environmental influences on energy balance-related behaviors: a dual-process view. Int J Behav Nutr Phys Act. 2006;3:9.View ArticlePubMedPubMed CentralGoogle Scholar
- De Bourdeaudhuij I, Van Cauwenberghe E, Spittaels H, Oppert J-M, Rostami C, Brug J, Van Lenthe F, Lobstein T, Maes L. School-based interventions promoting both physical activity and healthy eating in Europe: a systematic review within the HOPE project. Obes Rev. 2010;12:205–16.View ArticleGoogle Scholar
- Van Cauwenberghe E, Maes L, Spittaels H, van Lenthe FJ, Brug J, Oppert JM, De Bourdeaudhuij I. Effectiveness of school-based interventions in Europe to promote healthy nutrition in children and adolescents: systematic review of published and ‘grey’ literature. Br J Nutr. 2010;103(6):781–97.View ArticlePubMedGoogle Scholar
- Stice E, Shaw H, Marti CN. A meta-analytic review of obesity prevention programs for children and adolescents: the skinny on interventions that work. Psychol Bull. 2006;132(5):667–91.View ArticlePubMedPubMed CentralGoogle Scholar
- Michie S, Abraham C, Whittington C, McAteer J, Gupta S. Effective techniques in healthy eating and physical activity interventions: a meta-regression. Health Psychol. 2009;28(6):690–701. doi:https://doi.org/10.1037/a0016136. Review.View ArticlePubMedGoogle Scholar
- Carver CS, Scheier MF. Control theory: a useful conceptual framework for personality -social, clinical, and health psychology. Psychol Bull. 1982;92:111–35.View ArticlePubMedGoogle Scholar
- Michie S, Hardeman W, Fanshawe T, Prevost T, Taylor L, Kinmonth AL. Investigating theoretical explanations for behavior change: the case study of ProActive. Psychol Health. 2008;23(1):25–39.View ArticlePubMedGoogle Scholar
- Berridge KC. Food reward: brain substrates of wanting and liking. Neurosci Biobehav Rev. 1996;20:1–25.View ArticlePubMedGoogle Scholar
- Sclafani A. Oral and postoral determinants of food reward. Physiol Behav. 2004;81(5):773–9.View ArticlePubMedGoogle Scholar
- Davis C, Strachan S, Berkson M. Sensitivity to reward: implications for overeating and overweight. Appetite. 2004;42:131–8.View ArticlePubMedGoogle Scholar
- Verbeken S, Braet C, Lammertyn J, Goossens L, Moens E. How is reward sensitivity related to bodyweight in children? Appetite. 2012;58:478–83.View ArticlePubMedGoogle Scholar
- De Decker A, Sioen I, Verbeken S, Braet C, Michels N, De Henauw S. Associations of reward sensitivity with food consumption, activity pattern, and BMI in children. Appetite. 2016;100:189–96. doi:https://doi.org/10.1016/j.appet.2016.02.028. Epub 2016 Feb 17.View ArticlePubMedGoogle Scholar
- Hardin M, Ernst M. Functional brain imaging of development‐related risk and vulnerability for substance use in adolescents. J Addict Med. 2009;3:47–54.View ArticlePubMedPubMed CentralGoogle Scholar
- Deutsch R, Strack F. Duality in models in social psychology: from dual processes to interacting systems. Psychol Inq. 2006;17:166–72.View ArticleGoogle Scholar
- Moskowitz GB, Skurnik I, Galinski AD. The history of dual-process notions, and the future of preconscious control. In: Dual process theories in social psychology. New York: Guilford Press; 1999. p. 12–36.Google Scholar
- Bargh JA, Chartrand TL. The unbearable automaticity of being. Am Psychol. 1999;54:462–79.View ArticleGoogle Scholar
- Haddon L, Livingstone S. EU Kids Online: National perspectives. Online report accessed on June 2016 at http://www.lse.ac.uk/media@lse/research/EUKidsOnline/EU%20Kids%20III/Reports/PerspectivesReport.pdf.
- DiFilippo KN, Huang WH, Andrade JE, Chapman-Novakofski KM. The use of mobile apps to improve nutrition outcomes: a systematic literature review. J Telemed Telecare. 2015;21(5):243–53. doi:https://doi.org/10.1177/1357633X15572203. Epub 2015 Feb 12.View ArticlePubMedGoogle Scholar
- Hieftje K, Edelman EJ, Camenga DR, Fiellin LE. Electronic media-based health interventions promoting behavior change in youth: a systematic review. JAMA Pediatr. 2013;167(6):574–80. doi:https://doi.org/10.1001/jamapediatrics.2013.1095.View ArticlePubMedPubMed CentralGoogle Scholar
- Lewis ZH, Lyons EJ, Jarvis JM, Baillargeon J. Using an electronic activity monitor system as an intervention modality: a systematic review. BMC Public Health. 2015;15:585. doi:https://doi.org/10.1186/s12889-015-1947-3.View ArticlePubMedPubMed CentralGoogle Scholar
- Tate EB, Spruijt-Metz D, O’Reilly G, Jordan-Marsh M, Gotsis M, Pentz MA, Dunton GF. mHealth approaches to child obesity prevention: successes, unique challenges, and next directions. Transl Behav Med. 2013;3(4):406–15. doi:https://doi.org/10.1007/s13142-013-0222-3.View ArticlePubMedPubMed CentralGoogle Scholar
- Konstantopoulos S. The power of the test for treatment effects in three-level cluster randomized designs. J Res Educ Eff. 2008;1:66–88.Google Scholar
- Bartholomew LK, Parcel GS, Kok G, Gottlieb NH. Planning health promotion programs: an Intervention Mapping Approach. San Francisco: Jossey-Bass; 2006. p. 8–14.Google Scholar
- McClain AD, Chappuis C, Nguyen-Rodriguez ST, Yaroch AL, Spruijt-Metz D. Psychosocial correlates of eating behavior in children and adolescents: a review. Int J Behav Nutr Phys Act. 2009;6(54). doi:https://doi.org/10.1186/1479-5868-6-54.
- Martens MK, Van Assema P, Brug J. Why do adolescents eat what they eat? Personal and social environmental predictors of fruit, snack and breakfast consumption among 12–14-year-old Dutch students. Public Health Nutr. 2005;8(8):1258–65.View ArticlePubMedGoogle Scholar
- Pearson N, Ball K, Crawford D. Predictors of changes in adolescents’ consumption of fruits, vegetables and energy-dense snacks. Br J Nutr. 2011;105:795–803.View ArticlePubMedGoogle Scholar
- Story M, Neumark- Sztainer D, French S. Individual and environmental influences on adolescent eating behaviors. J Am Diet Assoc. 2002;102(3):40–51.View ArticleGoogle Scholar
- Kadzin AE. Acceptability of aversive procedures and medication as treatment alternatives for deviant child behavior. J Abnorm Child Psychol. 1984;12(2):289–301.View ArticleGoogle Scholar
- NUBEL. The Belgian Food composition table. 5th ed. 2009.Google Scholar
- Lobstein T, Davies S. Defining and labelling ‘healthy’ and ‘unhealthy’ food. Public Health Nutr. 2009;12(3):331–40. doi:https://doi.org/10.1017/S1368980008002541.Epub2008May.PubMedGoogle Scholar
- Flemish Institute for Health Promotion and Disease prevention. Nutrition guidelines for adolescents. Accessed on 1 June 2016 at http://www.vigez.be/themas/voeding-en-beweging/aanbevelingen/aanbevelingen-voor-voeding/jongeren.
- Petty RE, Cacioppo JT. The elaboration likelihood model of persuasion. In: Berkowits L, editor. Advances in experimental social psychology. New York: Academic Press; 1986. p. 123–205.Google Scholar
- Zajonc RB. Mere exposure: a getaway to subliminal. Curr Dir Psychol Sci. 2001;10:224–8.View ArticleGoogle Scholar
- Fazio RH. On the automatic activation of associated evaluations: an overview. Cognit Emot. 2001;15:115–41.View ArticleGoogle Scholar
- Latham GP, Locke EA. New developments in and directions for goal-setting research. Eur Psychol. 2007;12:290–300.View ArticleGoogle Scholar
- Baranowski T, Buday R, Thompson DI, Baranowski J. Playing for real: video games and stories for health-related behavior change. Am J Prev Med. 2008;34(1):74–82.View ArticlePubMedPubMed CentralGoogle Scholar
- Dickey M. Engaging by design: how engagement strategies in popular computer and video games can inform instructional design. Educ Technol Res Dev. 2005;53(2):67–83.View ArticleGoogle Scholar
- Annetta LA. The “I’s” have it: a framework for serious educational game design. Rev Gen Psychol. 2010;14(2):105–12.View ArticleGoogle Scholar
- Baranowski T, Baranowski J, Thompson D, Buday R. Behavioral science in video games for children’s diet and physical activity change: key research needs. J Diabet Sci Technol. 2011;5(2):229–33.View ArticleGoogle Scholar
- Peng W, Lin JH, Pfeiffer KA, Winn B. Need satisfaction supportive game features as motivational determinants: an experimental study of a self-determination theory guided exergame. Media Psychol. 2012;15(2):175–96.View ArticleGoogle Scholar
- Huybrechts I, Vereecken C, De Bacquer D, Vandevijvere S, Van Oyen H, Maes L, Vanhauwaert E, Temme L, De Backer G, De Henauw S. Reproducibility and validity of a diet quality index for children assessed using a FFQ. Brit J Nutr. 2010;104:135–44.View ArticlePubMedGoogle Scholar
- Rodriguez G, Moreno LA. Is dietary intake able to explain differences in body fatness in children and adolescents? Nutr Metab Cardiovasc. 2006;16(4):294–301.View ArticleGoogle Scholar
- HELENA study, 2006–2007 accessed at http://www.helenastudy.com. August 2nd 2016.
- Bandura A. Social foundations of thought and action: a social cognitive theory. Englewood Cliffs: Prentice‐Hall; 1986.Google Scholar
- Vereecken C, De Henauw S, Maes L, Moreno L, Manios Y, Phillipp K, Grp HS. Reliability and validity of a healthy diet determinants questionnaire for adolescents. Public Health Nutr. 2009;12(10):1830–8.View ArticlePubMedGoogle Scholar
- Gardner B. Habit as automaticity, not frequency. Euro Health Psychologist. 2012;14:32–6.Google Scholar
- Verplanken B, Orbell S. Reflections on past behavior: a self-report index of habit strength. J Appl Soc Psychol. 2003;33:1313–30. doi:https://doi.org/10.1111/j.1559-1816.2003.tb01951.x.View ArticleGoogle Scholar
- Singh AS, Vik FN, Chinapaw MJM, Uijtdewilligen L, Verloigne M, Fernandez-Alvira JM, et al. Test-retest reliability and validity of the ENERGY child questionnaire on energy balance-related behaviors and their potential behavioral determinants: the ENERGY project. Int J Behav Nutr Phys Act. 2011;8:136.View ArticlePubMedPubMed CentralGoogle Scholar
- Hartley JEK, Torsheim R, Currie C, et al. FASIII Development Study Report 2013.Google Scholar
- Van Stralen MM, Te Velde SJ, Singh AS, De Bourdeaudhuij I, Martens MM, Van der Sluis M, et al. EuropeaN Energy balance Research to prevent excessive weight Gain among Youth (ENERGY) project: design and methodology of the ENERGY cross-sectional survey. BMC Public Health. 2011;11:65.View ArticlePubMedPubMed CentralGoogle Scholar
- Hamel LM, Robbins LB. Computer- and web-based interventions to promote healthy eating among children and adolescents: a systematic review. J Adv Nurs. 2013;69:16–30.View ArticlePubMedGoogle Scholar
- Franken IHA, Muris P, Rassin E. Psychometric properties of the Dutch BIS/BAS Scales. J Psychopathol Behav Assess. 2005;27:25–30.View ArticleGoogle Scholar
- Cooper A, Gomez R, Aucote H. The Behavioral Inhibition System and Behavioral Approach System (BIS/BAS) Scales: Measurement and structural invariance across adults and adolescents. Personal Individ Differ. 2007;43:295–305.View ArticleGoogle Scholar
- Yu R, Branje SJT, Keijsers L, Meeus WHJ. Psychometric characteristics of Carver and White’s BIS/BAS scales in Dutch adolescents and their mothers. J Pers Assess. 2011;93:500–7.View ArticlePubMedGoogle Scholar
- Muris P, Meesters C, de Kanter E, Timmerman PE. Behavioral inhibition and behavioral activation system scales for children: relationships with Eysenck’s personality traits and psychopathological symptoms. Personal Individ Differ. 2005;38:831–41.View ArticleGoogle Scholar
- Decaluwé V. Child Eating Disorder Examination-Questionnaire. Dutch translation and adaptation of the Eating Disorder Examination-Questionnaire. In: Fairburn CG, Beglin SJ. 1999.Google Scholar
- Goossens L, Braet C. Screening for eating pathology in the pediatric field. Int J Pediatr Obes. 2010;5(6):483–90.View ArticlePubMedGoogle Scholar
- Decaluwé V, Braet C. Prevalence of binge-eating disorder in obese children and adolescents seeking weight-loss treatment. Int J Obes. 2003;27(3):404–9.View ArticleGoogle Scholar
- Petersen AC, Crockett LJ, Richards MH, Boxer AM. Measuring pubertal status: reliability and validity of a self-report measure. J Youth Adolesc. 1988;7:117–33.View ArticleGoogle Scholar
- Bolca S, Huybrechts I, Verschraegen M, De Henauw S, Van de Wiele T. Validity and reproducibility of a self-administered semi-quantitative food-frequency questionnaire for estimating usual daily fat, fibre, alcohol, caffeine and theobromine intakes among Belgian post-menopausal women. Int J Environ Res Public Health. 2009;6:121–50.View ArticlePubMedPubMed CentralGoogle Scholar
- Huybrechts I, De Bacquer D, Matthys C, De Backer G, De Henauw S. Validity and reproducibility of a semi-quantitative food-frequency questionnaire for estimating calcium intake in Belgian preschool children. Br J Nutr. 2006;95:802–16.View ArticlePubMedGoogle Scholar
- Moreno LA, Rodriguez G. Dietary risk factors for development of childhood obesity. Curr Opin Clin Nur Metab Care. 2007;10:336–41.View ArticleGoogle Scholar
- Van den Bulck J, Beullens K. Television and music video exposure and adolescent alcohol use while going out. Alcohol Alcohol (Oxford, Oxfordshire). 2005;40(3):249–53.View ArticleGoogle Scholar
- Tanis M, Jansz J. Gaming for different reasons: What motivates people to play a specific video game. In: Annual conference of the ICA, Montreal, Canada. 2008.Google Scholar
- Vorderer P, Bryant J (Eds). Playing video games: motives, responses, and consequences. New York: Routledge; 2006.Google Scholar
- Sherry JL, Lucas K, Greenberg BS, Lachlan K. Video game uses and gratifications as predictors of use and game preference. In: Vorderer P, Bryant J, editors. Playing video games motives responses and consequences. 2006. p. 213–24. Retrieved from http://citation.allacademic.com/meta/p_mla_apa_research_citation/1/1/1/4/7/p111471_index.html. Accessed 2 Aug 2016.
- Brockmeyer JH, Fox CM, Curtiss KA, McBroom E, Burkhart KM, Pidruzny JN. The development of the game engagement questionnaire: a measure of engagement in video game-playing. J Exp Soc Psychol. 2009;45(4):624–34.View ArticleGoogle Scholar
- Lemmens J, Valkenburg P, Peter J. Development and validation of a game addiction scale for adolescents. Media Psychol. 2009;12(1):77–95.View ArticleGoogle Scholar
- King DL, Delfabbro PH, Griffiths MD. The role of structural characteristics in problematic video game play: an empirical study. Int J Ment Heal Addict. 2011;9(3):320–33.View ArticleGoogle Scholar