Design and validation of a self-administered test to assess bullying (bull-M) in high school Mexicans: a pilot study
© Ramos-Jimenez et al.; licensee BioMed Central Ltd. 2013
Received: 27 September 2012
Accepted: 2 April 2013
Published: 11 April 2013
Bullying (Bull) is a public health problem worldwide, and Mexico is not exempt. However, its epidemiology and early detection in our country is limited, in part, by the lack of validated tests to ensure the respondents’ anonymity. The aim of this study was to validate a self-administered test (Bull-M) for assessing Bull among high-school Mexicans.
Experts and school teachers from highly violent areas of Ciudad Juarez (Chihuahua, México), reported common Bull behaviors. Then, a 10-item test was developed based on twelve of these behaviors; the students’ and peers’ participation in Bull acts and in some somatic consequences in Bull victims with a 5-point Likert frequency scale. Validation criteria were: content (CV, judges); reliability [Cronbach’s alpha (CA), test-retest (spearman correlation, rs)]; construct [principal component (PCA), confirmatory factor (CFA), goodness-of-fit (GF) analysis]; and convergent (Bull-M vs. Bull-S test) validity.
Bull-M showed good reliability (CA = 0.75, rs = 0.91; p < 0.001). Two factors were identified (PCA) and confirmed (CFA): “bullying me (victim)” and “bullying others (aggressor)”. GF indices were: Root mean square error of approximation (0.031), GF index (0.97), and normalized fit index (0.92). Bull-M was as good as Bull-S for measuring Bull prevalence.
Bull-M has a good reliability and convergent validity and a bi-modal factor structure for detecting Bull victims and aggressors; however, its external validity and sensitivity should be analyzed on a wider and different population.
KeywordsSchool violence Bullying Mexico Validation
Bullying at schools is defined as systematic physical, verbal and/or psychological abuse from one or more students toward another . Involved parties are identified as aggressor(s) (“bully”) and victim (“bullied”), respectively. The prevalence of this psychopathology is very high around the globe ranging from 9% to 45% and 5% to 36% among boys and girls, respectively [2, 3]. In Mexico, it goes from 10% to 83%, with those living in highly violent, poor, and insecure US-border cities being particularly vulnerable [4–7]. However, despite these figures and being considered a psychiatric disorder [8, 9], its detection and management is often restricted to school and social services.
Bullying implies the presence of certain psychopathological behaviors in aggressors and victims. Manifestations in the former include threatening, harassment, mocking, menacing, discrediting, or insulting , which requires timely psychiatric attention . The victims generate many health disturbances, anxiety and depression [10–12], and problems with interpersonal relationships [13, 14]. In turn, these behaviors may result in domestic violence, criminality, substance abuse , and even suicidal thoughts . There are significant differences in such bullying manifestations according to gender and socioeconomic status ; although other social factors also contribute such as having a family that promotes violence, teachers that ignore or dismiss bullying, schools that have a negative social climate, or students who socialize with bullies (aggressors) .
Several screening methods for detecting bullying have been applied within and outside of the school environment. Qualitative observation methods have been used successfully to evaluate students’ behavior at school, which constitute the most common assessment strategy used by school professionals . Direct interviews and self-reports have also been used to establish the incidence of bullying, its impact on student development, and the effectiveness of anti-bullying interventions [20, 21]. However, besides being time-consuming, these methods do not measure the true prevalence of bullying due to the fact that they rely on anonymity , so victims are overestimated while aggressors are underestimated . Recently, detection of adolescent victimization through mobile phone and internet tests have been proposed to guarantee anonymity . Although reliable for detecting self-perceived stress, loneliness, traditional victimization, and satisfaction with life, these tests have been applied only to asses cyber bullying and may prove to be very sophisticated for poor populations that do not have access to these technologies.
Validated, self-administered tests are practical and economic methods for evaluating bullying behaviors. They can be grouped into two categories: a) Those that allow analysis of the incidence anonymously and b) those that seek to evaluate a particular aspect of the problem through custom surveys or projective situations [25, 26]. In Mexico, the type and incidence of students’ aggressive behaviors have been studied by using the Bull-S test [9, 26] and the Concept of Intimidation among Equals (CIMEI) test . The former is a self-administered 15-item questionnaire for collective application and was developed mainly to detect aggressiveness among peers within the school context from a double perspective: students and teachers. It attempts, in a 25 to 30 minute administration, to identify aggressors, victims, victim-aggressors, and neutral parties. The problem with Bull-S, as occurs with other tests , is that it asks for personal information from the respondent, which does not guarantee anonymity and reduces the answering veracity. CIMEI detects harassment or mistreatment at school and includes three sections, each of them aimed at a different audience: Students, teachers, and parents. The student section includes 12 questions with multiple-choice, descriptive answers; it can be self-applied; and it has good internal consistency. Although the structure of this test allows the examiner to classify bullying into three different roles (victim, aggressor, and victim-aggressor), the operational definition of these roles within the test is somewhat difficult and confusing [9, 26].
The aim of this study was to validate a 10-item self-administered test (Bull-M) for assessing bullying among high school Mexicans. The development of a test with fewer questions (as compared to Bull-S) and whose only intention is to rapidly characterize the behavior of victims and aggressors may help to build epidemiological indicators without sacrificing the anonymity of the respondent.
Design and structure of bull-M
The test was originally designed by four experts in the fields of social sciences and community health (ARJ, AWM, OEDV and RPHT). They were supported by seven teachers with over five years of experience in bullying (Bull) management at schools with a high incidence of violence and drug abuse in Ciudad Juarez (Chihuahua, México). Focus groups and in-depth interviews were conducted to identify the most common Bull behaviors observed in or out of school. Ten questions (items) were included in the final test (Additional files 1 and 2): Five (items 1 to 5) about twelve Bull behaviors (representations) occurring in or out of school, four (items 6 to 9) on the student’s and/or peer’s participation in bullying acts, and one (item 10) on several somatic consequences in bullying victims (“In the last four weeks, how often have you had a stomachache, headache, loss of appetite, or problems sleeping?”) [12, 26, 27]. Also, in order to evaluate the frequency of each experienced situation or the student involvement in them, a 5-point Likert scale was added: never, rarely, sometimes, often, and every day. Only for validation purposes, this frequency was coded on a scale of 0–5. Lastly, although designed and applied in Spanish (Additional file 1), an English version of Bull-M is also provided (Additional file 2).
A preliminary version (9 items) of Bull-M, in which the order of all items was not that of the final instrument, was administered to 20 students (13–15 years old), to make sure that it was understandable and to evaluate response time. Although all participants found it clear, a certain degree of intimidation was observed due to the order of items. It was then decided to begin with another question (How often do your classmates allow or invite you to participate in their games, school activities, or extracurricular activities?) as the initial item (Additional files 1 and 2). Before answering the test, the general instructions were explained individually as well as the anonymous nature of Bull-M in order to reinforce the students’ confidence. The test was efficiently applied in 10–15 minutes by two collaborators (ARJ, OEDV) collectively within the classroom while the teacher was not present.
Bull-M was applied from February to May of 2011 among 400 students (60% male, 13.4 ± 1.1 y) from three out of 25 high schools located on the outskirts of Ciudad Juarez, zones with a long history of violence and poverty (bullying elicitors). The sample, although sampled completely randomly, was not representative of the population at all high schools in Ciudad Juarez. Two hundred participants were further selected for a second re-examination (test-retest validity) 14 days after the first application. An informed consent for participation was obtained from each parent, from school authorities, and from each participant. The protocol was approved by the Ethics Committee of the Autonomous University of Chihuahua (UACH).
Four criteria were chosen to validate Bull-M: a) content (CV, judges), b) reliability [Cronbach’s alpha (CA) and test-retest], c) construct [principal component (PCA) and confirmatory factor (CFA) analysis and goodness of fit (GF)], and d) convergent validity (Bull-M vs Bull-S). Their characteristics, rationale, procedure, and statistics used are described below:
Content validity (CV)
In the social sciences, CV (also known as logical validity) is important for demonstrating the degree to which a written test fulfills its purpose. Generally, this is the first validation criterion because it examines the overall comprehension of the test. It is usually performed by experts on the studied phenomenon (judge validation) who evaluate the design characteristics of the test as well as each individual item. In this study, CV was performed by twelve judges (RPHT, AWM, and ten more) by using the Expert Judgment Validity Test (EJVT; Additional file 3). All items (n = 19) included in EJVT were designed to intentionally evaluate the following characteristics of Bull-M: content, size, order, accuracy, and answering format, by using a 3-point Likert scale, which were further scored as follows: Poorly (0), fairly (1), and sufficiently (2). The average score (AS) for each item was then calculated (mean ± SD). An AS <1.5 to any item was taken into account for redesign when needed.
This type of validation is required for “adjusting” a test and contributes to its “most convenient” format. There are many statistical indicators used for this type of validation but the two most common are Cronbach’s alpha, which evaluates the consistency of results across items within a test, and test-retest, which evaluates the degree to which the test scores are consistent from one test application to the next. Here, a Cronbach’s alpha from 0.70-0.80, 0.81-0.90 and ≥0.90 was considered as acceptable, good, and excellent, respectively . Test-retest reliability was evaluated in 200 students within a 14 day frame. The difference between the items score of the first (test) and second application (retest) was evaluated by Spearman correlation (rs).
This was performed by principal component analysis (PCA) and confirmatory factor analysis (CFA), the latter through a structural equation model. PCA has two objectives: a) to reduce the number of items in a written test while retaining the variability of the data and b) to identify hidden patterns (components or factors) to classify them according to their contribution to the final test score. PCA is generally followed by CFA whose main objective is to test whether all items fit a hypothesized measurement model.
PCA and CFA were performed with samples of 200 and 198 participants, respectively. The factor structure of PCA was ascertained by Varimax rotation following the Kaiser-Guttman criterion in which eigenvalues are taken ≥1.0 as a decision rule . Also, in order to ensure an adequate representation of the variables, only those items whose communality (proportion of their variance explained by the factor) was ≥0.45 were included. Finally, in order to evaluate the fitting of the sampling and the possible sphericity of the data collected, the Kaiser-Meyer-Olkin (KMO)  and Bartlett  tests were applied. Lastly, the following goodness of fit indicators were calculated: Root mean square error of approximation (RMSEA), goodness of fit index (GFI), adjusted goodness of fit index (AGFI), comparative fit index (CGI), and normalized fit index (NFI). The model of structural equations was analyzed with Amos 16.0 (Amos development corporation, USA) while other analyses were done with PASW Statistic 18.0.
This refers to the degree to which two measures of constructs (e.g. two tests) that theoretically should be related are in fact related. Here, 100 participants from the second application (retest) of Bull-M were also invited to answer Bull-S. Bull-S is mainly used to detect Bull roles (victim, aggressor, and victim-aggressor) [9, 26]. However, one of its items (item #13) is related to the frequency in which Bull acts occur (how often the aggressions occur?) on a 4-point Likert scale (every day, twice a week, rarely, and never) . Pooled frequencies (%) from the “bullying other” subscale (items 6–9) of Bull-M and from item #13 of Bull-S were then compared, making a proper adjustment of both frequency scales for a proper comparison [Sometimes + often (Bull-M) = Twice a week (Bull-S)].
Content validity (CV)
There were no problems with the administration or comprehension of Bull-M. All 12 judges agreed that Bull-M was a valid test for assessing Bull behaviors (EJVT total score = 0.93).
Reliability of Bull-M
1.9 ± 0.09
1.9 ± 0.09
0.7 ± 0.06
0.8 ± 0.06
0.5 ± 0.05
0.9 ± 0.05*
0.8 ± 0.06
0.9 ± 0.06
0.7 ± 0.06
0.9 ± 0.06*
0.9 ± 0.06
1.0 ± 0.06
0.4 ± 0.05
0.6 ± 0.05*
0.8 ± 0.06
0.9 ± 0.06
0.7 ± 0.06
0.9 ± 0.06*
1.2 ± 0.08
1.1 ± 0.06
Construct validity of Bull-M
Factor loading 2
Convergent validity of Bull-M
Twice a week
Bullying (Bull) at schools in Mexico is reaching epidemic proportions. According to the First National Survey of Exclusion, Intolerance and Violence in Public Schools , surveyed by the Ministry of Public Education (SEP) in 2008, 44.6% and 26.2% of men and women between 15 and 19 years old recognized having abused their peers. On the other hand, the National Commission on Human Rights (CNDH) estimates that 40% of all Mexicans in elementary schools, both public and private, are victims of bullying . Also, previous studies have reported specific cultural conflicts and discrimination between ethnic groups due to differences in acculturation (e.g. language barriers), an additional factor seen in Mexican immigrants  and those settled in Mexico-US border cities (like the one studied here) . However, there have been very few systematic studies performed in Mexico on Bull epidemiology, none of them reviewing Bull impact on public health costs or its implications for community health.
Studies in social psychology indicate that Bull aggressors and victims are characterized by displaying recognizable behaviors . However, from an epidemiological standpoint, the psychosocial distress in both parties is not easy to detect because it depends largely on the complicity and anonymity provided by peers. As a consequence, Bull epidemiology and its early detection, at least in Mexico, is limited by the lack of validated tests to ensure the respondents’ anonymity. Here, the preliminary study and experts’ opinions showed that interviewing about school bullying can be intimidating. This can cause serious problems for the correct identification of both aggressors and victims of bullying. In this study it was observed that the addition of a single less intimidating question at the beginning, although having a low correlation with the overall scale (rs = 0.10; Table 1), did not affect the reliability of Bull-M. To our knowledge, there is no previous report that analyzes the intimidating effect of individual questions on the truthfulness of total response of any test that assesses bullying.
On the other hand, validation statistics used in this study (reliability, construct, and convergence) demonstrated that Bull-M is a useful tool for population studies. This assertion is supported by the following:
First, the internal consistency (Cronbach’s α; CA) and reproducibility (test-retest) showed that Bull-M is reliable. The general consensus on the interpretation of CA stating that α <0.70 means that a test is not uniform while an α > 0.90 suggests the existence of redundant items . Cronbach’s α for Bull-M was 0.75, signifying acceptable consistency. Baldry  and Cerezo  found similar CA results with Bull-S and a self-report anonymous questionnaire, respectively. Also, a high correlation was found between a first and second application [test-retest (rs = 0.91; p < 0.001)]. Although a higher score (p < 0.05) in retest was observed for items 3, 5, 7, and 9, this is non-meaningful because the transformation of the Likert scale (never, rarely, sometimes, often, and every day) into numbers (0, 1, 2, 3, 4) really shows that at both times, these items were identified as “rarely.” However, it should be kept in mind that the test-retest may be influenced by the complexity and ambiguity of the item ; although both the expert panel and the pilot study agreed on the fact that Bull-M is simple and clear, so complexity and ambiguity are not an issue.
Second, CPA and CFA correctly identified the bi-factorial character of Bull-M as was proposed from the beginning. CPA revealed that both factors (“Bullying me” and “Bullying others”) explained 44.6% of total variance. Although there are tests which explain up to 75% of total variance , it is more common to find lower percentages . A plausible explanation to this is the fact that no single test can contain all possible bullying representations as is the case of Bull-M which only reflect twelve situations (Additional files 1 and 2) which were gathered from in-depth interviews and focus groups with experts and teachers. Also, PCA is not able to demonstrate each factor structure or the dimensionality of the model , but CFA does; it confirmed the bi-factorial structure suggested by PCA, providing not only more evidence on the robust structure of Bull-M but also on the strong association between both factors.
Third, Bull-M was designed to allow assessment of the prevalence of bullying in schools which it actually allows, at least when compared to Bull-S (convergent validity). To our knowledge, comparisons on the psychometric properties of two tests designed to explore Bull at schools, have not yet been reported.
The authors recognize that there are many validation factors that need to be addressed in order to sustain even more the reliability and criterion validity of Bull-M. First, internal consistency and reproducibility, although commonly used in social sciences, are just two of many reliability criteria. External validation conducted on different populations with different social conditions could argue the reliability of Bull-M even more. Second, given the inherent subjectivity of any test that attempts to evaluate Bull behaviors, the calibration or criterion validity is difficult to evaluate since there is no “golden” criterion with which to contrast ; nevertheless, it has to be analyzed somehow. Third, although Bull-M was originally designed to be applied on subjects between the ages of 8 and 15, it also has to be validated with elementary school students because Bull at this age is somewhat different .
In this study, we have reported a suitable tool (Bull-M) for assessing bullying among high school Mexicans that may help to build epidemiological indicators without sacrificing the anonymity of the respondent. Bull-M showed good internal consistency, reproducibility and a robust structure capable of detecting two Bull roles (victim and aggressor). The number of items (n = 10) and factors (two) of Bull-M make it an ideal instrument for population surveys since it is easy to apply, and its results are economic. Moreover, it has the potential ability to evaluate not only the frequency (and thus the prevalence) of Bull situations and roles (aggressor/victim) but also has the ability to prioritize the types of Bull behaviors and their somatic effects on victims (data not reported here).
This paper is merely the first to demonstrate the potential of Bull-M for assessing Bull at schools. This highlights the need for further external validity not only in multiple populations but also with the inclusion of other criteria validity factors (e.g. perception of parents and teachers).
Implications for school health
Bull-M can improve the gathering of information on the presence of Bull in high school, allowing the design of better intervention programs to reduce this public health concern.
- Olweus D: Bullying at school. 1993, Reading, Massachusetts: Blackwell PublishersGoogle Scholar
- Craig W, Harel-Fisch Y, Fogel-Grinvald H, Dostaler S, Hetland J, Simons-Morton B, Molcho M, de Mato MG, Overpeck M, Due P, Pickett W, HBSC Violence & Injuries Prevention Focus Group; HBSC Bullying Writing Group: A cross-national profile of bullying and victimization among adolescents in 40 countries. Int J Public Health. 2009, 54 (Suppl 2): 216-224.View ArticlePubMedPubMed CentralGoogle Scholar
- Del Barrio C, Martín E, Montero I, Gutiérrez H, Barrios A, de Dios MJ: Bullying and social exclusion in Spanish secondary schools: National trends from 1999 to 2006. International J Clin Health Psych 2008. 2008, 8 (3): 657-667.Google Scholar
- Olaiz F, Rivera D, Shamah L, Rojas R, Villalpando H: Encuesta Nacional de Salud y Nutrición. 2006, Cuernavaca, México: Instituto Nacional de Salud PúblicaGoogle Scholar
- Abramovay M: Victimización en las escuelas. Ambiente escolar, robos y agresiones físicas. Revista Mex Invest Educ. 2005, 10: 833-864.Google Scholar
- Aviles-Dorantes DS, ZOnana-Nacach A, Anzaldo-Campos MC: Prevalencia de acoso escolar (bullying) en estudiantes de una secundaria pública. Salud pública Méx. 2012, 54 (4): 362-363.View ArticlePubMedGoogle Scholar
- Leiner M, Puertas H, Caratachea R, Avila C, Atluru A, Briones D, de Vargas C: Children’s mental health and collective violence: a binational study on the United States–Mexico border. Rev Panam Salud Publica. 2012, 31 (5): 411-416. 10.1590/S1020-49892012000500009.View ArticlePubMedGoogle Scholar
- Joffre-Velazquez VM, Garcia-Maldonado G, Saldivar-Gonzalez AH, Martinez-Perales G, Lin-Ochoa D, Quintanar-Martinez S, Villasana-Guerra A: Bullying in junior high school students: general characteristics and associated risk factors. Bol Med Hosp Infant Mex. 2011, 68 (3): 177-185.Google Scholar
- Albores-Gallo L, Sauceda-García JM, Ruiz-Velasco S, Roque-Santiago E: Bullying and its association with psychiatric disorders in a Mexican students sample. Salud Publica Mex. 2011, 53: 220-227.PubMedGoogle Scholar
- Due P, Damsgaard MT, Lund R, Holstein BE: Is bullying equally harmful for rich and poor children?: a study of bullying and depression from age 15 to 27. Eur J Public Health. 2009, 19 (Suppl 5): 464-469.View ArticlePubMedGoogle Scholar
- Fleming LC, Jacobsen KH: Bullying and symptoms of depression in chilean middle school students. J Sch Health. 2009, 79 (Suppl 3): 130-137.View ArticlePubMedGoogle Scholar
- Salmon G, James A, Smith DM: Bullying in schools: self-reported anxiety, depression, and self-esteem in secondary school children. BMJ. 1998, 317: 924-925. 10.1136/bmj.317.7163.924.View ArticlePubMedPubMed CentralGoogle Scholar
- Hanley AJ, Gibb BE: Verbal victimization and changes in hopelessness among elementary school children. J Clin Child Adolesc Psychol. 2011, 40 (Suppl 5): 772-776.View ArticlePubMedPubMed CentralGoogle Scholar
- Gilmartin BG: Peer group antecedents of severe love-shyness in males. J Persy. 1987, 55: 467-489. 10.1111/j.1467-6494.1987.tb00447.x.View ArticleGoogle Scholar
- Kim MJ, Catalano RF, Haggerty KP, Abbott RD: Bullying at elementary school and problem behavior in young adulthood: a study of bullying, violence and substance use from age 11 to age 21. Crim Behav Ment Health. 2011, 21 (2): 136-144. 10.1002/cbm.804.View ArticlePubMedPubMed CentralGoogle Scholar
- Kim YS, Leventhal B: Bullying and suicide. A review. Int J Adolesc Med Health. 2008, 20 (2): 19-33.View ArticleGoogle Scholar
- Domínguez LF, Manzo CMC: Las manifestaciones del bullying en adolescentes. Uaricha Rev Psicol (Nueva época). 2011, 8 (17): 19-33.Google Scholar
- Espelage DL, De la Rue L: School bullying: its nature and ecology. Int J Adolesc Med Health. 2012, 24 (1): 3-10.View ArticleGoogle Scholar
- Shapiro ES, Heick P: School psychologist assessment practices in the evaluation of students referred for social/behavioral/emotional problems. Psychol Sch. 2004, 41: 551-561. 10.1002/pits.10176.View ArticleGoogle Scholar
- Casey-Cannon S, Hayward C, Gowen K: Middle-school girls’ reports of peer victimization: Concerns, consequences, and implications. Sch Couns. 2001, 5 (Suppl 2): 138-147.Google Scholar
- Glover D, Gough G, Johnson M, Cartwright N: Bullying in 25 secondary schools: incidence, impact, and intervention. Educ Res. 2000, 42 (Suppl 2): 141-156.View ArticleGoogle Scholar
- Power JL, Brotheridge CM, Blenkinsopp J, Bowes-Sperry L, Bozionelos N, Buzády Z, Chuang A, Drnevich D, Garzon-Vico A, Leighton C, Madero SM, Mak W, Mathew R, Monserrat SI, Mujtaba BG, Olivas-Lujan MR, Polycroniou P, Sprigg CA, Axtell C, Holman D, Ruiz-Gutiérrez JA, Nnedumm AUO: Acceptability of workplace bullying: a comparative study on six continents. J Bus Res. 07342-http://dx.doi.org/10.1016/j.jbusres.2011.08.018. In press
- Pellegrini AD, Bartini M: An empirical comparison of methods of sampling aggression and victimization in school settings. J Educ Psych. 2000, 92: 360-366.View ArticleGoogle Scholar
- Buelga S, Cava MJ, Musitu G: Validation of the adolescent victimization trough mobile phone and internet scale. Rev Panam Salud Publica. 2012, 32 (1): 36-42. 10.1590/S1020-49892012000700006.View ArticlePubMedGoogle Scholar
- Crothers LM, Levinson EM: Assessment of bullying: a review of methods and instruments. J Couns Devel. 2004, 82 (Suppl 4): 496-503.View ArticleGoogle Scholar
- Cerezo F: Violencia y victimización entre escolares. El Bullying: estrategias de identificación y elementos para la intervención a través del test Bull-S. Rev Elect Inv Psicoeduc. 2006, 9 (4): 333-352.Google Scholar
- Kumpulainen K, Räsänen E, Henttonen I, Almqvist F, Kresanov K, Linna SL, Moilanen I, Piha J, Puura K, Tamminen T: Bullying and psychiatric symptoms among elementary school-age children. Child Abuse Negl. 1998, 22: 705-717. 10.1016/S0145-2134(98)00049-0.View ArticlePubMedGoogle Scholar
- Cortina JM: What is coefficient alpha? An examination of theory and applications. J Appl Psychol. 1993, 78: 98-104.View ArticleGoogle Scholar
- Cuadras CM: Nuevos Métodos de análisis multivariante. 2010, Barcelona: CMC EditionsGoogle Scholar
- Kaiser HF: A second generation little jiffy. Psychometrika. 1970, 35 (Suppl 4): 401-415.View ArticleGoogle Scholar
- Zwick WR, Velicer WF: Comparison of five rules for determining the number of components to retain. Psychol Bull. 1986, 99 (Suppl 3): 432-442.View ArticleGoogle Scholar
- Méndez I, Cerezo F: Test Bull-s: programa informático de evaluación de la agresividad entre escolares. 25 Años de Integración Escolar en España: Tecnología e Inclusión en el ámbito educativo, laboral y comunitario. Edited by: En A, Hurtado MD P, Soto FJ. 2010, Murcia: Consejería de Educación, Formación y EmpleoGoogle Scholar
- Szekely M: 1ª Encuesta Nacional Exclusión, Intolerancia y Violencia en Escuelas Públicas de Educación Media Superior. 2008, México: SEPGoogle Scholar
- Notimex: Cuatro de cada 10 alumnos sufren bullying. 2012, México: El UniversalGoogle Scholar
- Mendez JJ, Bauma S, Guillory RM: Bullying of Mexican inmigrant Studies: An examination of intracultural bullying. Hisp J Behav Sci. 2012, 34 (2): 279-304. 10.1177/0739986311435970.View ArticleGoogle Scholar
- Calderón D: Contradicción en mayúsculas. México Social. 2013, 30: 44-47.Google Scholar
- Baldry AC: Bullying in schools and exposure to domestic violence. Child Abuse Negl. 2003, 27: 713-732. 10.1016/S0145-2134(03)00114-5.View ArticlePubMedGoogle Scholar
- Otter ME, Mellenbergh GJ, Glopper KD: The relation between information processing variables and test-retest stability for questionnaire items. J Educ Meas. 1995, 32 (Suppl 2): 199-216.View ArticleGoogle Scholar
- Batista-Foguet JM, Coenders G, Alonso J: Análisis factorial confirmatorio. Su utilidad en la validación de cuestionarios relacionados con la salud. Med Clin (Barc). 2004, 122: 21-27.View ArticleGoogle Scholar
- Smith PK, Cowie H, Olafsson RF, Liefooghe AP, Almeida A, Araki H, del Barrio C, Costabile A: Definitions of bullying: a comparison of terms used, and age and gender differences, in a fourteen-country international comparison. Child Dev. 2002, 73 (4): 1119-1133. 10.1111/1467-8624.00461.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/13/334/prepub