- Research article
- Open access
- Published:
Test implementation of a school-oriented drug prevention program “Study without Drugs”: pre- and post-testing for effectiveness
BMC Public Health volume 14, Article number: 590 (2014)
Abstract
Background
In this article, the test implementation of a school-oriented drug prevention program “Study without Drugs” is discussed. The aims of this study were to determine the results of the process evaluation and to determine whether the proposed school-oriented drug prevention program during a pilot project was effective for the participating pupils.
Methods
Sixty second-grade pupils at a junior high school in Paramaribo, Suriname participated in the test implementation. They were divided into two classes. For the process evaluation the students completed a structured questionnaire focusing on content and teaching method after every lesson. Lessons were qualified with a score from 0–10. The process was also evaluated by the teachers through structured interviews. Attention was paid to reach, dose delivered, dose received, fidelity, connection, achieved effects/observed behaviors, areas for improvement, and lesson strengths. The effect evaluation was conducted by using the General Liniair Model (repeated measure). The research (-design) was a pre-experimental design with pre-and post-test.
Results
No class or sex differences were detected among the pupils with regard to the assessment of content, methodology, and qualification of the lessons. Post-testing showed that participating pupils obtained an increased knowledge of drugs, their drug-resisting skills were enhanced, and behavior determinants (attitude, subjective norm, self-efficacy, and intention) became more negative towards drugs.
Conclusions
From the results of the test implementation can be cautiously concluded that the program “Study without Drugs” may yield positive results when applied in schools). Thus, this pilot program can be considered a step towards the development and implementation of an evidence-based school-oriented program for pupils in Suriname.
Background
The test implementation of a school-oriented drug prevention program “Study without Drugs” was conducted in Suriname. Suriname is located in the northeast of South America and has an area of l163,820 km2. The population consists of ± 600,000 people and is very heterogeneous in composition. After a ± 300-year period of colonization, first by England and then the Kingdom of the Netherlands, Suriname became an independent republic on November 25, 1975. On November 25, 1980 a coup d’ etat took place, led by Desi Bouterse. During the period 1980-1987 Suriname was more often associated with drug trafficking. Democracy did his re-appearance in 1987. Studies suggest that there are no previously conducted evidence-based drug prevention programs developed and implemented in schools. “Study without Drugs” is the first school-based drug prevention program which during its development possible criteria of other effective school-based drug prevention programs have been integrated as much as possible. This program can serve as a first step towards the further development of evidence-based school programs in Suriname.
School plays an important role with regard to the presentation of interventions. Adolescents spend a great deal of time in school, and general education and the teaching of life skills are central in educational environments. Drug prevention programs in schools focus on teaching and/or improving skills and knowledge about drugs and the behavior determinants of drug use. Teachers can help improve and maintain the wellbeing of pupils by detecting and preventing drug use [1]. At school, information on drugs can/must be offered because defensible pupils perform better [2–5]. Thus, it is important to identify effective factors when determining the impact of such programs. This is necessary to properly rate the value of the executed programs and to make choices for theoretical methods and practical strategies.
In addition to identifying effective factors, school-oriented prevention programs must be evidence based. A method or approach is evidence based when its function has been proved through scientific research [6, 7].
The evidence should have been found largely through ‘hard’ research methods like experiments with randomly selected experimental and control groups. Van der Wolf and Van Beukering [8] indicated that evidence based means that the methods, programs, and approaches used have to be scientifically proven to be effective. An evidence-based approach promotes reflection on the effectiveness of the methods, approaches, and programs used. Influencing factors are brought into the picture and analyzed. One speaks of evidence-based drug prevention when the strategies, procedures, and methods used are carefully validated and standardized under research conditions and are found effective with regard to behavior changes in adolescents concerning drug use and drug attitudes [9].
A test implementation can be considered small-scale evaluation research. Evaluation research is defined as “a process for applying academic procedures when gathering reliable and valid evidence regarding the manner and the scale of which specific activities produce certain results and outcomes” [10].
Evaluation research measures the effects and outcomes of programs and suggests ways for refining these programs to obtain more favorable outcomes [11].
Process evaluation and effect evaluation
Process evaluation can focus on various levels (content/organization) and target groups (intermediary and end target groups), and can be conducted by applying several research methods. Process evaluation is also used to determine whether the intervention on each target group was implemented uniformly. Process evaluation provides information about the reasons behind a program’s effectiveness or ineffectiveness. Whether or not the intervention has an influence is also measured, and the reasons behind its success or failure. Many interventions consist of several activities. Process evaluations can then be used to determine how each activity influences the outcome results separately. It is important to know what activities the most or the least contributed to this success. This type of evaluation will help explaining the relationship between specific intervention activities. Process evaluation can help to prove why the intervention failed. Examples of failure of interventions are: inappropriate intervention design, incomplete implementation and insufficient accessibility of the target audience, an intervention can be adjusted to make it yet successful, through a process analysis.
Thus, research is conducted on the conditions necessary for the effect. Furthermore, insight is given into possible improvements or revisions, for example, to improve the intervention design and methods for the future [12–14]. The following components are also discussed with regard to process evaluation: context, reach, dose delivered, dose received, fidelity, implementation, and recruitment [15]. Not all components need to be included in a process evaluation. Priority should be given to questions on the size, dose delivered, dose received, and consistency.
With effect evaluation, the effects on behavior are measured. Effect evaluation can focus, for instance, on changes in behavior determinants. It is not always possible to measure hard, precise, and objective outcomes with health-promotion interventions. For this reason, secondary effect measures are established and measured. The secondary measures are behavior determinants that may have been influenced during and because of the intervention. The idea is that change in determinants will lead to a change in behavior [16]. Thus, two questions can be asked: 1) has the program attained its short-term expectations and 2) has the program achieved the desired long-term results?
Drug use among Surinamese adolescents
The Drug Demand Reduction Program (DDR) conducted a nationwide Rapid Situation Assessment in 2005 [17]. The assessment studied 480 participants aged 13 years and older. The results showed that the majority of smokers started smoking before they were 20 years old.
More than 50% of the youngsters (13-16 year) indicated that their fathers smoked. With relation to age, 60% of the male respondents had their first alcoholic drink before turning 20, as did 38% of the female respondents.
School surveys conducted in 2004 [18] and 2006 [19] by the DDR (by order of CICAD [Inter-American Drug Abuse Control Commission]) found alcohol, cigarettes, and marijuana to be the most commonly used drugs. Cigarettes and marijuana were predominantly used by male students aged between 13 and 15 years. Female students most commonly used alcohol for the first time when aged between 13 and 14 years. Most students who smoked were in the second year of high school and those who used alcohol were in their first year. On average, 30% of the respondents had access to illicit drugs. For detailed data on drug use among Surinamese adolescents, please see Tables 1 and 2.
The DDR (2005) [20] also executed a small-scaled study among 125 adolescents aged between 12 and 16. The results of this research are as follows: 20% of respondents stated that they knew someone (or more than one person) that used drugs. The drugs users were, for instance, direct family members, friends, schoolmates, or strangers (e.g., people from the neighborhood and junkies). The most commonly used drugs were tobacco (29.6%), and marijuana and alcohol (25%).
Based on the available data, alcohol, tobacco, and marijuana are popular among youth in Suriname. Although not many young people use cocaine, they could easily point out people in their social environment who did. This leads to the assumption that cocaine is readily available among the Surinamese population.
Thus, while the level of drug use among youth in Suriname is not alarmingly high (yet), prevention measures must be implemented to ensure it does not escalate.
Intervention mapping protocol
To interpret what has been recorded concerning evidence-based school-oriented drug prevention programs, and keeping in mind the assumption that the presented data from the executed school drug surveys and other (small-scale) studies represent just the tip of drug use, a school-oriented drug prevention program “Study without Drugs” was developed using intervention mapping (IM) protocol. “Study without Drugs” is a drugs demand-reduction program that has been developed for Surinamese adolescents at junior high level. The development of the drug prevention program was carried out using IM protocol, which consists of six steps. The added value of this protocol is that it lends itself to prepare and execute health-promotion activities. IM is based on theory, empirical evidence, and additional qualitative and quantitative research, and the involvement of stakeholders and actors is central. The six phases of the protocol enable health-promotion activities to be prepared and executed, taking into account situational and personal circumstances. In phase 6, an evaluation plan is made that focuses on effect and process evaluations of the program, and an evaluation model is developed [13]. The school-oriented drug prevention program developed in March–April 2010 consists of 10 activities, namely: seven lessons (e.g., Dutch, biology, physical education, drawing, religion, and social studies), one informative activity, and two evaluation procedures. The intervention was carried out from June 2010 until –July 2010 (see Table 3). In designing “Study without Drugs” attempts have been made to integrate many criteria which have been proven to be effective in school programs, such as: use of interactive methods; “Teaching of Skills” is central; training of persons who are going to offer the program is necessary; the research design includes a pre-and post-test; an integrated approach and position of the prevention program within the curriculum is desirable; peers as implementers ensure increased effectiveness; activities that fall outside the curriculum should be included; and there is an approach where the family (parents), the society or community or parts thereof are involved or have an input in the program [21–24].
Methods
Study participants
The two groups of respondents (all pupils from one junior high school) consisted of 60 pupils: 24 boys and 36 girls aged between 12 and 15 years, with an average age of 14 years. Class 2A had 31 pupils (12 boys and 19 girls) and 2B 29 pupils (12 boys and 17 girls). It was a select sample survey based on voluntary participation. School management and teachers committed themselves to carrying out the test implementation at their school. The group of teachers consisted of three women and two men (four teachers had been in service for less than 10 years). Two teachers were qualified to teach at senior high level, two at junior high level, and one had a different teaching qualification. Carrying out the surveys and interviews for both the process evaluation and the pre-and post-test were done by trained interviewers who were not known to the students nor the teaching staff.
Process evaluation
The process evaluation by the pupils was done via a questionnaire (developed by the researcher) after each lesson. The objective of the process evaluation by the pupils was to record their opinion on the content of the lessons taught and the teaching method. The pupils also got the opportunity to express whether or not they enjoyed the lessons by qualifying the lesson with a number between 1 and 10. The pupils had a maximum of 30 minutes to fill in the questionnaire.
A structured interview was also conducted with the teachers after each lesson. The interview lasted 45–60 minutes and consisted of questions on the following aspects: reach, dose delivered, dose received, fidelity, subject connection, general remarks, lesson strengths, and areas for improvement. These interviews were recorded with a voice recorder and later transcribed into a computer.
The transcribed conversations were presented to the respondents for possible modification (comments). Then the data were coded and the transcripts analyzed.
Effect evaluation
For the effect evaluation, a pre-test was conducted before the execution of the program, after two-and-a-half month followed by a post-test after the program. Each of the tests lasted 35 minutes. Measurements were taken on the basis of a one-group pre- and post-test design. These tests aimed to measure the pupils’ knowledge, skills, and the impact of behavior determinants before and after attending the prevention program.
Measuring instruments
The questionnaire for the effect measurement by pupils comprised of 17 propositions aimed to determine the respondents’ opinions on the content (nine propositions) and teaching method (eight propositions).
Measurement was done using a 5-point Likert scale that ranged from completely disagree (1) to completely agree (5). For the content, aspects such as attractiveness, level of difficulty, and use of material were measured. The measurement of the teaching method included the way in which the teacher dealt with the teaching methods, how (s)he used the material, and linking to the initial situation.
Cronbach’s alphas for the process evaluation questionnaires were calculated. The coefficients of the different lessons ranged from 0.789 to 0.859. The internal consistency of coefficients were also checked per subject per dimension (content and teaching method). The coefficients of the dimension content ranged from 0.659 to 0.808, and those for the teaching method from 0.693 to 0.813.
Table 4 shows the design of the pre- and post-tests. The pre- and post-tests have identical content. The Cronbach’s alphas of both tests were calculated: pre-test, 0.602–0.729; post-test, 0.633–0.851.
Ethical consideration
Ethical clearance was obtained from the Ministry of Education (in particular, the Department of Inspection for Continuing Education at Junior level). A letter from the Ministry was sent to the participating school in which had been indicated that consent for the test deployment was granted and cooperation of the management and staff were asked for the (test-) implementation.
With regard to obtaining permission/cooperation of the staff, each participating staff member was personally approached. Parents of participating students were asked by the school for permission to take part. None of the parents has objected against their children participating in the (test-) implementation.
Data analysis
Data were analyzed using Statistical Package for Social Sciences (SPSS) software version 19. For the process evaluation, the means, standard deviation, ANOVA, and Pearson correlation were calculated. For the effect evaluation, the pre- and post-test were administered. With questions 1, 2, 3, 4, 5, and 7 (see Table 4), two scores were possible, namely1 (correct/good) and 0 (incorrect/wrong). The total score is the number of correct answers.
The influence of the behavior determinants was measured in question 6. This question is based on the Theory of Planned Behavior (TPB). In question 6 pupils chose their answer by indicating a score (totally disagree–totally disagree) on a 5-point Likert scale. The General Repeated Measure (repeated measurement) was applied. (By using the General Linear Model (GLM) ‘repeated measures’ a variance analysis was carried out on the inter-dependent measurements. This occurs when in the same or similar (with the same level of measurement) measurements were carried out with the same subjects. In case Nnd measurements are performed on different groups, there is - apart from a ‘within subjects design’ also- a ‘between subjects design’).
The advantage of a within- subject design versus a between- subject design is that a within- subject design is more efficient in the sense that the sample need not be less large, while still a significant result is attained. This is because the scores of a person are associated with each other. This provides additional information, making the conclusions more reliable. In a between- subject design the averages of the condition will differ from each other because in one condition other subjects are present rather than in the other condition. In a within- subject design, this may cause no difference between the averages of the condition, because the same people are used for all conditions. The differences between the averages of condition parts of a within- subject design are therefore less subject to change. Due to the additional information of the link, the scores can be compared within a person. This makes it possible to assess whether the effect occurs consistently in all subjects (there is a small MS interaction); that consistency is additional information making the conclusions more reliable.
We next deal with the results of this measurement, and then a chi-square test is used to compare pre- and post-measurements. The statistical significance level was set at p < 0.05.
Results
Qualification of lessons by pupils
The pupils scored the lessons from 0 to 10. Except for the first Dutch lesson, there were no significant differences between classes (2A and 2B) and sexes. The average score for each lesson was 8.6 or higher, ranging from 8.6 to 9.4. A comparison of classes shows that class 2A had a range from 8.5–9.7 and class 2B from 8.0–9.3. There is no appreciable difference based on class and sex. With the boys, the social studies lesson scored lowest with 8.0 (range for boys: 4–10), while the girls’ lowest scoring lessons were social studies and biology with 8.7 (range for girls: 6–10). A comparison of the two classes shows that the first Dutch lesson significantly differed from the other lessons: F(2.120) = 0.5.41, p < 0.05.
Pupils’ assessment of lesson content and teaching method
A comparison of the average scores and p-values with the assessment of lesson content and teaching method did not show any significant differences between sexes and classes. The average grade given by pupils showing their (positive) assessment was between 38 and 41. Based on the SD one can also conclude that male pupils (both in the full sample and at a class level) showed a greater spread in relation to the average score per lesson; however, this difference is not significant. There was significant difference between classes 2B and 2A with regard to the content of the social studies lesson.
There was no difference in class and sex with the teaching method of the lessons. The average score given by the pupils for (positive) appreciation was between 25 and 32 for the full sample, and in class 2B there was a greater dispersion in relation to the average score of the lesson. Again, this difference was not significant.
Relationship between lesson content and teaching method
The results of the Pearson’s correlation show that (p ≥ 0.05) there is a strong positive relation between lesson content and teaching method (Table 5).
Process evaluation by teachers
The text below describes the responses of teachers to questions concerning key components: reach, dose delivered, dose received, and fidelity. Comments on areas for improvement and lesson strengths are also included.
Reach
The teachers indicated that they checked attendance at every lesson (7 lessons for each class). All 31 pupils from class 2A attended 6 lessons. Only in one lesson three students were absent.
All 29 pupils from class 2B attended 6 lessons. Where in one session two pupils were absent.
Thus, over 80% of pupils per class participated in the drugs prevention program. According to registration details, a total of 60 pupils participated in both the pre- and post-test.
Dose delivered
The biology teacher indicated that the one of the lesson objectives, to teach the “consequences of tobacco, alcohol and marijuana use”, was not achieved due to a lack of time. The rest of the teachers stated that the objectives were reached.
The social studies teacher found that the teaching method of ‘asking questions’ after showing a DVD did not work due to a lack of time. Furthermore, he remarked that not all of the lesson phases were dealt with in enough depth. For organizational reasons, he reduced the introduction because time had been lost during preparation.
Regarding the drawing lesson (making an anti-drugs poster), the teacher found that the pupils got quite stressed trying to finish their poster within the allowed time. The teacher also had to repeat the instructions and criteria a number of times to ensure the students knew what they were doing.
When showing the Spanish-language film with English subtitles, the Dutch teacher had to explain some sections as a small number of pupils asked for clarification. The teacher commented that it is advisable to work with Dutch subtitles, or to provide a short explanation or summary of the film before, during, or immediately it is shown.
Dose received
The teachers reported being satisfied with the availability and use of the material. They estimated that most of the material was suitable for the level and age of the pupils, and that it was ready for use. The teachers also noted that the majority of the lessons based on a didactical model (i.e., didactical analysis). The person-to-person instruction round preceding the program was deemed sufficiently supportive. The teachers indicated that the trainer/instructor gave the trainees enough room to put forward their own views on lesson content and teaching methods. Biology and Dutch teachers thought parts of the text of the drugs information book was difficult for the pupils to understand. The physical education teacher said that the pictures helped to clearly show the effects of steroid use.
The physical education teacher also stated that both the information book and folders were used well by the pupils when executing their assignment (making an anti-drugs poster). With the Dutch 3 lesson, the film was judged to be a realistic portrayal of the consequences of drugs smuggling.
There was very little deviation from the instructions on how to use the material. All teachers indicated that they would use the material again.
Fidelity
All lessons were executed in accordance with the lesson plans provided. The lesson phases were dealt with in the indicated order. Prescribed material, instructions, and teaching methods in the lesson plans were largely implemented in accordance with the instructions.
Areas for improvement and lesson strengths
The teachers also noted areas for improvement and lesson strengths. These comments are shown in (see Table 6).
Effect evaluation results
Listing known drugs
In the pre- and post-test pupils were asked to list all the drugs they knew of. The variable time was found to have a significant main effect pre-test: F(1.56) = 5.68, p = 0.000; post-test F(1.56) = 8.81, p = 0.000. The average shows that the number of known drugs was higher after the intervention than before it.
Although there is no significant main effect, class 2A showed a greater knowledge of drugs than class 2B. Girls showed more progress than boys (this difference is not significant, see Table 7).
The chi-square test indicates significant differences between the full sample with the pre-test and the full sample with the post-test, because a sizable increase was noticeable with all drugs when they were mentioned. The 10 drugs mentioned were cocaine, tobacco, alcohol, steroid, hashish, xtc, marijuana, crystal meth, and opium. With all 10 drugs, an increase can be noted with the number of drugs mentioned in the post-test (see Table 8).
Coupling of drugs with effects of use
Fourteen possible drug effects from drug use were indicated. Pupils were asked to write down one drug associated with each of the 14 effects. The variables time F(1, 56) = 42.94, p = 0.000 and class F(1, 56) = 5.91, p < 0.05 were found be significant main effects in the pre- and post-test. Furthermore, it should be noted that there is a significant interaction effect between time and sex and time and class. The average score was higher in the post-test (see Table 7).
The results from the effect and drug coupling exercise showed a significant difference for nine effects when comparing the pre- and post-test results for the full sample. Five of these effects showed a positive shift in the post-test. Observed differences concern the following items: baldness and hair loss (Chi 2 = 26.12; df = 1; p < 0.05), red eyes (Chi 2 = 6.42; df = 1; p < 0.05), male hair growth (Chi 2 = 14.90; df = 1; p < 0.05), damage to mucous membranes (Chi 2 = 8.29; df = 1; p < 0.05), development of anxiety (Chi 2 = 15.09; df = 1; p < 0.05 0), coma (Chi 2 = 8.16; df = 1; p < 0.05), headache (Chi 2 = 8.64; df = 1; p < 0.05), cirrhosis of the liver (Chi 2 = 33.41; df = 1; p < 0.05), and switching to hard drugs (Chi 2 = 7.35; df = 1; p < 0.05). Four effects showed a decrease in the post-test. Three showed an increase in the post-verification, but were not significant. Two items/effects showed a significant decrease in the post-test. None of the pupils successfully coupled the effect dizziness in the pre-test.
Recognizing drugs in photographs
The tests also included 10 colored photos clearly showing various drugs. The pupils were asked to identify the drugs and write the drug name beneath the photos.
Both time F(1, 56) = 105.69, p = 0.000 and class F(1, 56) = 16.93, p < 0.05 were found to be main effects. A noticeable difference (increase) can be seen in the averages between the measurements taken before and after the test. The significant interaction effect between time and class shows that class 2A was slightly better at recognizing drugs in the post-test than 2B (see Table 7). Taking into account the recognition of each drug separately, a comparison of the full sample pre-test and post-test shows that there is a significant difference in time because of an increase in the recognition of nine of ten drugs depicted (see Table 9).
Propositions on drugs
Eighteen propositions were presented with the intention to measure knowledge about drugs and drug-related items. There were significant main effects with the variables time F(1, 56) = 53.33, p = 0.000, sex F(1, 56) = 53.31, p < 0.05, and class F(1, 56) = 68.61, p = 0.000. The post-test averages were higher than the pre-test. There was also a significant interaction effect between sex and class F(1, 56) = 13.22, p = 0.000 and between time, sex and class F(1, 56) = 11.76, p = 0.000 (see Table 7).
Regarding the impact of time, classes 2A and 2B showed a positive shift in the post-test. Male and female pupils in class 2A scored better in the measurement beforehand than those in class 2B (see Tables 7 and 10).
When comparing the full sample in the pre- and post-tests, the groups differ significantly with regard to 7 of the 18 propositions, namely: proposition 5, marijuana is also seen as a gateway drug (Chi 2 = 3.75; df = 1; p < 0.05); proposition 9, the lungs are harmed by tar in tobacco smoke (Chi 2 = 6.00; df = 1; p < 0.05); proposition 10, drugs can change, stimulate and numb one’s consciousness (Chi 2 = 4.68; df = 1; p < 0.05); proposition 11, crystal meth affects the entire body (Chi 2 = 8.78; df = 1; p < 0.05); proposition 15, marijuana comes from the plant Cannabis sativa (Chi 2 = 5.91; df = 1; p < 0.05); proposition 17, doping is used to achieve top performances in a dishonest manner (Chi 2 = 4.48; df = 1; p < 0.05); and proposition 18, pure cocaine looks like fine crystalline powder (Chi 2 = 7.06; df = 1; p < 0.05).
Identifying aid organizations
In both the pre- and post-tests, pupils were presented with an overview of 18 institutes; students had to identify 7 as aid agencies. For time, the main effect is significant with both measurements taken before and after the test (F(1, 56) = 92.68, p = 0.000).
Pupils were better able to indicate aid organizations in the post-test. There appears to be a significant combined interaction effect between time, sex, and class (F(1, 56) = 4.35, p = 0.000). Regarding the impact of time, the boys in class 2B experienced a greater positive shift in the post-test. Furthermore, the girls in class 2A always scored better in the pre- and post-tests than those in class 2B (see Tables 7 and 10).
An analysis of the results for pupils’ recognition of aid organizations shows that the pupils differ significantly with all 10 instances and in the post-test (see Table 11).
Behavior determinants
Time is a significant main effect for attitude F(1, 56) = 6.24, p = 0.000. The post-test showed that pupils now have a more negative attitude towards drugs. Furthermore, sex is also a main effect for attitude F(1, 56) = 4.11, p = 0.000. The attitude of both girls and boys showed a more positive shift in the post-test.
Girls have a much more negative attitude towards drugs in the post-program test. With regard to the subjective norm, time F(1, 56) = 20.63, p = 0.000 and class F(1, 56) = 5.30, p = 0.000 are found to be mains effects. The averages of both classes (2A and 2B) were higher in the post-test. This indicates that pupils became more sensitive towards their own appraisal of the social pressure to use drugs or not.
Looking now at the behavior determinant, observed behavior control, the variables time F(1, 56) = 39.5, p = 0.000 and sex F(1, 56) = 0.487, p = 0.000 are shown to be main effects. The averages were higher in the post-test. The variables sex and class have a significant interaction effect F(1, 56) = 5.05, p = 0.000. Although the boys showed a positive shift in the post-test, the girls showed a greater improvement. Thus, the girls were better able to control their behavior regarding drugs (use). With regard to the effect of time on class, 2A had a better initial score, but 2B showed greater improvement.
With intention there was a main effect for time F(1, 56) = 66.31, p = 0.000 and class F(1, 56) = 8.45, p = 0.000. There was an increase in the averages, which indicates that the pupils experienced a positive shift: there was an increase in the intention to show/execute behavior against drugs. Class 2B had a higher average than class 2A in both the pre- and post-tests (see Table 7).
Skills for defensibleness
Twelve propositions were presented representing a skill that indicates defensibleness against drugs. The variable class has a main effect. Based on class F(1, 56) = 118.08, p < 0.05), a difference (increase) can be observed between the averages of the measurements before and after. Class 2A showed a greater positive shift in the post-test.
Two skills showed a significant decrease in the post-test, one remained stable and nine skills showed a positive shift (increase) (see Table 7).
Discussion
Process evaluation
The interviews with the teachers showed that they considered the process evaluation of the drug prevention program to accurately represent the success of the program. The teachers’ positive reactions regarding the implementation and/or adoption of the interventions are based on the following characteristics: appropriate level of complexity, can be communicated, can be included with existing curricula, observability of activities to be executed and results, possibility to adapt content through reversibility and modification, reformulating instructions and getting feedback [25–27].
Personal characteristics like attitude, subjective norm, observed behavior control, and the intention of the recipient or person delivering the intervention contribute to the experiences/opinions [28]. The factors that impact on the willingness of teachers to pass on the content of an intervention are as follows: the subjective norm, own effectiveness, interpretation of one’s job, the extent in which one feels responsible for passing on the content, and instrumentality of materials [29, 30]. That is, the extent in which the material has clear instructions for its preparation, presentation, and evaluation and the extent in which the material leads pupils to be active and enthusiastic [31–33].
In the process evaluation, it was mentioned that in some instances, time constraints led to improvisation. With other health-promotion programs, times limitations have been described as bottlenecks [34–37].
With regard to dose delivered, it can be concluded that the teachers worked fairly well with the (sub) goals of the lessons and that those goals were achieved. With regard to dose received, the teachers reported being satisfied with the intervention. The material was received positively; it was largely suitable for the age and level of the pupils. With regard to its reach, the majority of pupils participated in the program.
The assessment/qualification of lessons by pupils depends on the way in which they (participants in an innovation/intervention) experience the session. Factors like a pupil’s initial level, attractiveness of the lesson content and materials, complexity of the lesson, instructional skills and expertise of the teacher, and application/use of multimedia appear to affect the performance of pupils [26, 31, 33, 38, 39]. Group dynamics processes can also affect a class’s assessment/qualification, as well as assessment by sex. Group dynamics can affect an individual’s performance, making it either better or worse [40, 41].
Effect evaluation
The differences that became apparent through the chi-square test can be explained with the taxonomy of Bloom [42] and Anderson et al. [43]. Educational psychology indicates that pupils can score considerably better with less complex cognitive activities/goals. These cognitive activities can include listing and/or recognizing drugs (reproduction of knowledge).
A comparison of the results of the pre- and post-tests showed an increase in the percentage of correct responses with the listing of drugs, recognition of drugs, and aid organizations. There is, however, no great increase in knowledge when coupling effects with the correct drugs, propositions on drugs, and mastery of skills [44, 45]. It is difficult to give a precise reason for the occasional decreases in the post-test. It is possible that an overestimation by the pupils occurred in the pre-test, resulting in a more flattering picture of the situation. The post-test is possibly a more realistic representation because the pupils were confronted with accurate information at the intervention. With replication research, the execution of tasks can also be measured by means of observation.
With the measurement of the effect of time on sex and class, class influence is a feature of the group dynamics processes [40, 46]. Items that are of influence in the class are, among others, the number of pupils, composition of the group, group atmosphere and the resulting social interaction, and the group objectives [33].
With regard to the impact of the pupils’ sex, such characteristics can also affect the overall school performance. There are clear differences between the performances of female and male pupils.
Females profit more form individual learning than from collaborative learning. Males profit from both to the same extent. Divergent patterns in the collaboration between females and males show a cognitive chasm between female and male pupils. There are also likely to be differences in the way they elaborate, mode of communication, and their leadership styles [47–49].
According to Veendrick, Tavecchio, and Doornenbal [50] the learning performance of boys can also be affected by the phenomenon whereby education is becoming feminized. Thus, boys go without male role models, with all its consequences. The difference in learning and performance motivation between individuals in groups/classes may play an important part. Thorpe [51] stated that boys are expected not to like school, as reinforced by comments from their parents and teachers. This has negative consequences for learning motivation and performance. Girls experience just the opposite and they do like school, thus performing better. A third explanation in the literature is a difference in learning styles. A learning style is the manner in which somebody best acquires knowledge. One person learns better, for instance, when the material is presented visually, while another learns best when the content is presented aurally [52]. Boys and girls presumably differ with regard to learning styles, and presumably those used in schools are not appropriate for boys. Van Langen and Driessen [53] have stated that boys perform well in mathematics and physics. They prefer learning rules and abstract facts by heart, and respond to episodic, factual, and detailed comments. Girls prefer open-ended tasks that are coupled with real situations. They respond more extensively to questions, putting lessons learned in a broad context.
Conclusion
The dual purpose of “Study without Drugs” test-implementation was to determine the results of the process evaluation and whether the Proposed school-oriented drug prevention program was effective for the participating pupils when offered as a pilot.
From the results of the evaluation process can be stated that:
all classes were assessed as to be sufficient by the students. Furthermore, the students involved were positive about the content of the courses offered and the teaching methods used. Teachers had for the large part a quite positive opinion on the following aspects: reach, dose delivered, dose received, fidelity and. subject connection,
With regard to the measurement of the effect, it can be noted with caution that: participating pupils showed a (slight) increase in knowledge of drugs (that is, in the field of listing known drugs, recognizing drugs in photographs, identifying aid organizations) and their drug-resisting skills. There was no significant increase in knowledge regarding coupling of drugs with effects of use, Propositions on drugs). Furthermore, after attending the program, a number of behavior determinants—attitude, subjective norm, self-efficacy and intention—were more negative towards drugs.
Some limitations of this test implementation were: The test implementation has been done on a small scale, therefore it is desirable to note that the results cannot be generalized to other groups of students outside the group of respondents (no external generalizability).
The results (both process and impact) may become affected by aspects such as: impact test, instrumentation and maturation, previous experience concerning drug prevention programs offered outside the intervention.
The design of the study subject is pre-experimental, with no control group. Respondents were selected without randomization. Necessary adaptations must be made namely: application of randomization since this reduces the risk of selection bias. The research design should be in line with the experiment and control group in which respondents who are randomly selected are also to be included in both. Matching of the research ¬ populations should take place at the start of the study design. Furthermore, it is possible when analyzing data from a study conducted to perform “stratify or statistical control” (through multivariate regression analysis ¬) afterwards. Previously enumerated improvements may lead to increase of internal and external validity and generalizability of results.
Finally, it can be said that the developed program “Study without Drugs” - after adjustments on the basis of the recorded points for improvement/limitations observed - can be used as a starter for further development of evidence-based school drug prevention programs. A structural implementation of the program into the school curriculum could lead to (improvement/increase) enhance the quality of the lives of adolescents as well as reducing the risk that they will use drugs.
Abbreviations
- DDR:
-
Drug demand reduction program
- IM:
-
Intervention mapping.
References
Humanos: Children in the Street. Models Drug for Prevention based on Human Rights. Boletín del Instituto Interamericano del Niño “INFANCIA”. 2001, Tomo 69N° 236: Montevideo, Febrero de
Davis JM, Cooke SM: Educating for a healthy, sustainable world: an argument for integrating health promoting schools and sustainable schools. Health Promot Int. 2007, 22 (4): 346-353. 10.1093/heapro/dam030.
Suhrcke M, de Paz NC: The Impact of Health and Health Behaviours on Educational Outcomes in High-Income Countries: A Review of the Evidence. 2011, Copenhagen, WHO Regional Office for Europe: World Health Organization
Wester P, De Jong A: Uitgaan en Drugs; Producten Binnen het Domein School [in Dutch]. 1999, Utrecht, The Netherlands: Trimbos-instituut
Cuijpers P: Effective ingredients of school-based drug prevention programs. A systematic review. Addict Behav. 2002, 27 (6): 1009-1023. 10.1016/S0306-4603(02)00295-2.
Alton-Lee A: Improving Educational Policy and Practice through an Iterative Best Evidence synthesis Program. 2004, Wellington: Ministry of Education
Onderwijsraad: Naar meer evidence based onderwijs. [Towards more evidence based Education]. 2006, The Netherlands: Onderwijsraad
Davies P: What is Evidence-Based Education?. 1999, Oxford: Department for Continuing Education, University of Oxford
Van der Wolf K, Van Beukering T: Gedragsproblemen in Scholen. Het Denken en Handelen van Leraren [Behavioral Problems in Schools, the Thinking and Action of Teachers]. 2009, Cecco: The Netherlands
Pentz MA: Evidence-based prevention: characteristics, impact and future direction. J Psychoactive Drugs. 2003, 35 (Suppl 1): 143-152.
Rutman LS: Evaluation Research Methods: A Basic Guide. 1977, London: Beverly Hills
Windsor RA, Baranowski T, Clark N, Cutter G: Evaluation of Health Promotion, Health Education, and Disease Prevention Programs. 1994, Mayfield: Mountain View
Bartholomew LK, Parcel GS, Kok G, Gottlieb NH, Fernandez ME: Planning Health Promotion Programs: An Intervention Mapping Approach. 2011, San Francisco: Jossy-Bar
Linnan L, Steckler A: Process evaluation for public health interventions and research: an overview. Process Evaluation for Public Health Interventions and Research. Edited by: Steckler A, Linnan L. 2002, San Francisco: Jossey-Bass, 1-23.
Saunders RP, Evans MH, Josh P: Developing a process-evaluation plan for assessing health promotion program implementation: a how to guide. Health Promot Pract. 2005, 6: 134-147. 10.1177/1524839904273387.
Brug J, van Assema P, Lechner L: Gezondheidsvoorlichting en Gedragsverandering: Een Planmatige Aanpak [Health Education and Behavior Change: A Systematic Approach]. 2007, Assen, The Netherlands: Van Gorcum & Comp
Drug Demand Reduction Program: Rapid Situation Assessment. 2005, Verslag: DDR
Drugs Demand Reduction Program Suriname: Verslag van de School-Drugs Survey. [Report of the school drugs survey]. 2004, Suriname: Paramaribo
Drugs Demand Reduction Program Suriname: Verslag van de School-Drugs Survey. [Report of the school drugs survey]. 2006, Suriname: Paramaribo
Drugs Demand Reduction Program Suriname: Kennis en Attitude van Jeugdigen met Betrekking tot Drugs [Knowledge and Attitude of Young People with regard to Drugs]. 2005, Paramaribo: Suriname
Green LW, Kreuter MW: Health Program Planning: An Educational & Ecological Approach. 2005, New York: McGraw-Hill
Dusenbury L, Falco M, Lake A: A review of the evaluation of 47 drug abuse prevention curricula available nationally. J Sch Health. 1997, 67 (4): 127-132. 10.1111/j.1746-1561.1997.tb03431.x.
Gottfredson DC, Wilson DB: Characteristics of effective school-based substance abuse prevention. Prev Sci. 2003, 4: 27-38. 10.1023/A:1021782710278.
Tobler NS, Roona MR, Ochshorn P, Marshall DG, Streke AV, Stackpole KM: School-based adolescent drug prevention programs: 1998 Meta-Analysis. J Prim Prev. 2000, 20 (4): 275-336. 10.1023/A:1021314704811.
Wiefferink CH, Poelman J, Linthorst M, Vanwesenbeeck I, van Wijngaarden JC, Paulussen TG: Outcomes of a systematically designed strategy for the implementation of sex education in Dutch secondary schools. Health Educ Res. 2005, 20: 323-333. 10.1093/her/cyg120.
Rogers EM: Diffusion of Innovations. 2003, New York: The Free Press, 5
Zaltman G, Duncan R: Strategies for Planned Change. 1977, New York: Wiley
Ajzen I: The theory of planned behavior. Organ Behav Hum. 1991, 50: 179-211. 10.1016/0749-5978(91)90020-T.
Paulussen TGW: Adoption and Implementation of AIDS Education in Dutch Secondary Schools. 1994, Dutch Centre for Health Promotion and Health Education: Utrecht
Paulussen T, Kok G, Schaalma HP, Parcel GS: Diffusion of AIDS curricula among Dutch secondary school teachers. Health Educ Q. 1995, 22: 227-243. 10.1177/109019819502200210.
Boekaerts M: Towards a model that integrates motivation, affect, and learning. Development and Motivation: Joint Perspectives. Edited by: Smith L, Rogers C, Tomlinson P. 2003, England: The British Psychological Society
Boekaerts M, Simons RJ, Simons PR: Leren en instructie [Learning and Instruction]. 1995, Assen, the Netherlands: Van Gorcum
Tjipke van der Veen: Lesgeven en Zelfstandig Leren [Teaching and Independent Learning]. 2009, Assen, The Netherlands: Van Gorcum
Buston K, Wight D, Hart G, Scott S: Implementation of a teacher-delivered sex education program: obstacles and facilitating factors. Health Educ Res. 2002, 17 (1): 59-72. 10.1093/her/17.1.59.
Reubsaet A, Reinaerts EBM, Brug J, Van Hooff JP, van den Borne HW: Process evaluation of a school-based education program about organ donation and registration, and the intention for continuance. Health Educ Res. 2004, 196: 720-729.
Reynolds KJ, Oakes PJ, Haslam SA, Nolan MA, Dolnik L: Responses to powerlessness: Stereotyping as an instrument of social conflict. Group Dyn Theor Res. 2000, 4 (4): 275-290.
Singh AS, Chin A, Paw MJ, Brug J, van Mechelen W: Dutch obesity intervention in teenagers: Effectiveness of a school-based program on body composition and behavior. Arch Pediatr Adolesc Med. 2009, 163 (4): 309-317. 10.1001/archpediatrics.2009.2.
Pankratz M, Hallfors D, Cho H: Measuring perceptions of innovation adoption: the diffusion of a federal drug prevention policy. Health Educ Res. 2002, 17 (3): 315-326. 10.1093/her/17.3.315.
Rohrbach LA, Sussman S, Dent CW, Sun P: Tobacco, alcohol, and other drug use among high-risk young people: a five-year longitudinal study from adolescence to emerging adulthood. J Drug Issues. 2005, 35 (2): 333-356. 10.1177/002204260503500206.
Johnson DW, Johnson FP: Joining together: Group Theory and Group Skills. 2000, Boston: Allyn and Bacon, 7
Van Oudenhoven JP: Groepsdynamica [Group Dynamics]. 1998, Groningen, The Netherlands: Wolters-Noordhoff
Bloom B: Handbook on Formative and Summative Evaluation of Student Learning. 1971, New York: McGraw-Hill
Anderson LW, Krathwohl DR: Developing and Writing Behavioural Objectives. 1970, Tucson, AZ: Educational Innovators Press, New cognitive domain: http://oregonstate.edu/instruct/coursedev/models/id/taxonomy/#table Based upon Dave RH, as reported in Armstrong RJ et al
Massaro G, Veerhoeven W: Beter ten Halve Gekeerd; Pretesten van VoorlichtingsmaterialenvoorMigranten [Better to Turn Back Halfway; Pretesting Information Materials for Migrants]. 1992, Utrecht, The Netherlands: Nederlands Centrum Buitenlanders, 7-11.
Romiszowski A: The development of physical skills: instruction in the psychomotor domain. Instructional Design Theories and Models: A New Paradigm of Instructional Theory, Volume. Edited by: Reigeluth CM. 1999, New Jersey: Lawrence Erlbaum Associates, 457-481.
Remmerswaal JLM: HandboekGroepsdynamica: EenNieuweInleiding op Theorie en Praktijk [Group Dynamics: A New Introduction to Theory and Practice]. 2003, Soest, The Netherlands: Nelissen
Breedlove W, Burkett T, Winfield I: Collaborative testing, gender, learning styles, and test performance. Int J Scholarsh Teach Learn. 2007, 4 (2): 33-42.
Ding N: Computer-Supported Collaborative Learning and Gender. 2009, The Netherlands: University of Groningen
Fuchs D, Fuchs LS, Mathes PG, Simmons DC: Peer-assisted learning strategies: making classrooms more responsive to diversity. Am Educ Res J. 1997, 34 (1): 174-206. 10.3102/00028312034001174.
Veendrick L, Tavecchio LWC, Doornenbal J: Jongensalsprobleem: inleidingbij het themadeel [Guys like problems: introduction to the theme section]. The Netherlands. Pedagogiek. 2004, 24 (1): 12-22.
Thorpe G: Multilevel analysis of PISA 2000 reading results for the United Kingdom using pupil scale variable. Sch Eff Sch Improv. 2006, 17 (1): 33-62. 10.1080/09243450500264473.
Smith PL, Ragan TJ: Instructional Design. 1999, New York: Macmillan Publishing Company
van Langen A, Driessen G: Sekseverschillen in Schoolloopbanen. Een Internationaal Comparatieve Trendstudie [Gender differences in school careers. An international comparative trend study]. 2006, Nijmegen, The Netherlands: ITS
Pre-publication history
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/14/590/prepub
Acknowledgements
The authors would like to thank the Surinamese Ministry of Education, in particular the department responsible for inspecting junior high schools. We are also grateful to the heads, teachers, students, and parents who participated the test implementation in this study. Officials from the Ministry of Justice (especially the Public Relations and anti-narcotics department) and the Ministry of Health (especially the Bureau of Alcohol and Drugs) were also of great assistance as we planned and implemented this new school-based drug-prevention program. We also want to thank Brian Fung Loy and Gillian Atmopawirofor there part in helping developing the material
Author information
Authors and Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
FI developed the program and the material, she led the analysis and interpretation, and drafted the manuscript. DV and VdW and participated in supervision of the design of the program. All authors read, revised and approved the final manuscript. All authors contributed to the design of this study.
Rights and permissions
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Ishaak, F., de Vries, N.K. & van der Wolf, K. Test implementation of a school-oriented drug prevention program “Study without Drugs”: pre- and post-testing for effectiveness. BMC Public Health 14, 590 (2014). https://doi.org/10.1186/1471-2458-14-590
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/1471-2458-14-590