Design and validation of a self-administered test to assess bullying (bull-M) in high school Mexicans: a pilot study

Bullying at schools is defined as systematic physical, verbal and/or psychological abuse from one or more students toward another [1]. Involved parties are identified as aggressor(s) (“bully”) and victim (“bullied”), respectively. The prevalence of this psychopathology is very high around the globe ranging from 9% to 45% and 5% to 36% among boys and girls, respectively [2, 3]. In Mexico, it goes from 10% to 83%, with those living in highly violent, poor, and insecure US-border cities being particularly vulnerable [4–7]. However, despite these figures and being considered a psychiatric disorder [8, 9], its detection and management is often restricted to school and social services.

Bullying implies the presence of certain psychopathological behaviors in aggressors and victims. Manifestations in the former include threatening, harassment, mocking, menacing, discrediting, or insulting [8], which requires timely psychiatric attention [9]. The victims generate many health disturbances, anxiety and depression [10–12], and problems with interpersonal relationships [13, 14]. In turn, these behaviors may result in domestic violence, criminality, substance abuse [15], and even suicidal thoughts [16]. There are significant differences in such bullying manifestations according to gender and socioeconomic status [17]; although other social factors also contribute such as having a family that promotes violence, teachers that ignore or dismiss bullying, schools that have a negative social climate, or students who socialize with bullies (aggressors) [18].

Several screening methods for detecting bullying have been applied within and outside of the school environment. Qualitative observation methods have been used successfully to evaluate students’ behavior at school, which constitute the most common assessment strategy used by school professionals [19]. Direct interviews and self-reports have also been used to establish the incidence of bullying, its impact on student development, and the effectiveness of anti-bullying interventions [20, 21]. However, besides being time-consuming, these methods do not measure the true prevalence of bullying due to the fact that they rely on anonymity [22], so victims are overestimated while aggressors are underestimated [23]. Recently, detection of adolescent victimization through mobile phone and internet tests have been proposed to guarantee anonymity [24]. Although reliable for detecting self-perceived stress, loneliness, traditional victimization, and satisfaction with life, these tests have been applied only to asses cyber bullying and may prove to be very sophisticated for poor populations that do not have access to these technologies.

Validated, self-administered tests are practical and economic methods for evaluating bullying behaviors. They can be grouped into two categories: a) Those that allow analysis of the incidence anonymously and b) those that seek to evaluate a particular aspect of the problem through custom surveys or projective situations [25, 26]. In Mexico, the type and incidence of students’ aggressive behaviors have been studied by using the Bull-S test [9, 26] and the Concept of Intimidation among Equals (CIMEI) test [8]. The former is a self-administered 15-item questionnaire for collective application and was developed mainly to detect aggressiveness among peers within the school context from a double perspective: students and teachers. It attempts, in a 25 to 30 minute administration, to identify aggressors, victims, victim-aggressors, and neutral parties. The problem with Bull-S, as occurs with other tests [26], is that it asks for personal information from the respondent, which does not guarantee anonymity and reduces the answering veracity. CIMEI detects harassment or mistreatment at school and includes three sections, each of them aimed at a different audience: Students, teachers, and parents. The student section includes 12 questions with multiple-choice, descriptive answers; it can be self-applied; and it has good internal consistency. Although the structure of this test allows the examiner to classify bullying into three different roles (victim, aggressor, and victim-aggressor), the operational definition of these roles within the test is somewhat difficult and confusing [9, 26].

The aim of this study was to validate a 10-item self-administered test (Bull-M) for assessing bullying among high school Mexicans. The development of a test with fewer questions (as compared to Bull-S) and whose only intention is to rapidly characterize the behavior of victims and aggressors may help to build epidemiological indicators without sacrificing the anonymity of the respondent.

Bullying (Bull) at schools in Mexico is reaching epidemic proportions. According to the First National Survey of Exclusion, Intolerance and Violence in Public Schools [33], surveyed by the Ministry of Public Education (SEP) in 2008, 44.6% and 26.2% of men and women between 15 and 19 years old recognized having abused their peers. On the other hand, the National Commission on Human Rights (CNDH) estimates that 40% of all Mexicans in elementary schools, both public and private, are victims of bullying [34]. Also, previous studies have reported specific cultural conflicts and discrimination between ethnic groups due to differences in acculturation (e.g. language barriers), an additional factor seen in Mexican immigrants [35] and those settled in Mexico-US border cities (like the one studied here) [7]. However, there have been very few systematic studies performed in Mexico on Bull epidemiology, none of them reviewing Bull impact on public health costs or its implications for community health.

Studies in social psychology indicate that Bull aggressors and victims are characterized by displaying recognizable behaviors [36]. However, from an epidemiological standpoint, the psychosocial distress in both parties is not easy to detect because it depends largely on the complicity and anonymity provided by peers. As a consequence, Bull epidemiology and its early detection, at least in Mexico, is limited by the lack of validated tests to ensure the respondents’ anonymity. Here, the preliminary study and experts’ opinions showed that interviewing about school bullying can be intimidating. This can cause serious problems for the correct identification of both aggressors and victims of bullying. In this study it was observed that the addition of a single less intimidating question at the beginning, although having a low correlation with the overall scale (r_s = 0.10; Table 1), did not affect the reliability of Bull-M. To our knowledge, there is no previous report that analyzes the intimidating effect of individual questions on the truthfulness of total response of any test that assesses bullying.

On the other hand, validation statistics used in this study (reliability, construct, and convergence) demonstrated that Bull-M is a useful tool for population studies. This assertion is supported by the following:

First, the internal consistency (Cronbach’s α; CA) and reproducibility (test-retest) showed that Bull-M is reliable. The general consensus on the interpretation of CA stating that α <0.70 means that a test is not uniform while an α > 0.90 suggests the existence of redundant items [34]. Cronbach’s α for Bull-M was 0.75, signifying acceptable consistency. Baldry [37] and Cerezo [26] found similar CA results with Bull-S and a self-report anonymous questionnaire, respectively. Also, a high correlation was found between a first and second application [test-retest (r_s = 0.91; p < 0.001)]. Although a higher score (p < 0.05) in retest was observed for items 3, 5, 7, and 9, this is non-meaningful because the transformation of the Likert scale (never, rarely, sometimes, often, and every day) into numbers (0, 1, 2, 3, 4) really shows that at both times, these items were identified as “rarely.” However, it should be kept in mind that the test-retest may be influenced by the complexity and ambiguity of the item [38]; although both the expert panel and the pilot study agreed on the fact that Bull-M is simple and clear, so complexity and ambiguity are not an issue.

Second, CPA and CFA correctly identified the bi-factorial character of Bull-M as was proposed from the beginning. CPA revealed that both factors (“Bullying me” and “Bullying others”) explained 44.6% of total variance. Although there are tests which explain up to 75% of total variance [6], it is more common to find lower percentages [37]. A plausible explanation to this is the fact that no single test can contain all possible bullying representations as is the case of Bull-M which only reflect twelve situations (Additional files 1 and 2) which were gathered from in-depth interviews and focus groups with experts and teachers. Also, PCA is not able to demonstrate each factor structure or the dimensionality of the model [39], but CFA does; it confirmed the bi-factorial structure suggested by PCA, providing not only more evidence on the robust structure of Bull-M but also on the strong association between both factors.

Third, Bull-M was designed to allow assessment of the prevalence of bullying in schools which it actually allows, at least when compared to Bull-S (convergent validity). To our knowledge, comparisons on the psychometric properties of two tests designed to explore Bull at schools, have not yet been reported.

In this study, we have reported a suitable tool (Bull-M) for assessing bullying among high school Mexicans that may help to build epidemiological indicators without sacrificing the anonymity of the respondent. Bull-M showed good internal consistency, reproducibility and a robust structure capable of detecting two Bull roles (victim and aggressor). The number of items (n = 10) and factors (two) of Bull-M make it an ideal instrument for population surveys since it is easy to apply, and its results are economic. Moreover, it has the potential ability to evaluate not only the frequency (and thus the prevalence) of Bull situations and roles (aggressor/victim) but also has the ability to prioritize the types of Bull behaviors and their somatic effects on victims (data not reported here).