Skip to main content
Download PDF

Psychometric validation of the Reported and Intended Behaviour Scale (RIBS) in Hungary with a particular focus on ‘Don’t know’ responses and further scoring recommendations

BMC Public Health volume 23, Article number: 1773 (2023) Cite this article

Abstract

Aims

Reported and Intended Behaviour Scale (RIBS) was designed to measure mental health stigma-related behaviors in the general public. We aimed to examine its psychometric properties and validate the scale in a Hungarian non-clinical community sample. The secondary aim of this study was to assess the appropriateness of the current scoring recommendations of ‘Don’t know’ responses being coded as neutral, which had never been investigated before. In addition, we provide an overview of the results of already existing studies on the scale.

Methods

Hungarian participants completed the RIBS within this cross-sectional online survey study and were considered non-clinical individuals based on a cut-off point of the Global Severity Index T score of 63 on the Symptom Checklist-90-Revised. Confirmatory factor analysis, reliability measures, and comparative analyses were performed.

Results

Of the n = 5,701, n = 5,141 participants were included in the analysis. The mean age was 27.8 ± 11.1 years, and 89.2% (n = 4,587) of the sample were female. The unidimensional structure was supported by good model fit indices (RMSEA = 0.031, CFI = 0.999, TLI = 0.996, and WRMR = 0.006). Internal consistency of the RIBS and its test–retest reliability with a 5-month follow-up period were found to be good (Cronbach’s alpha = 0.88 and ICC = 0.838). We found statistically significant differences between the total scores when the ‘Don’t know’ responders were excluded from the sample or when they were coded as neutral as recommended by the scale authors (16 (IQR:13–18) vs. 15 (IQR:13–18) p < 0.0001). There were also statistically significant differences between ‘Neither agree nor disagree’ and ‘Don’t know’ participants in several aspects of lived experiences of mental health problems.

Conclusions

The RIBS demonstrated good psychometric properties and can be transferred to the Hungarian context. It will be a valuable tool in assessing stigmatizing behavior and testing the efficacy of antistigma programs. Our results suggest that ‘Neither agree nor disagree’ and ‘Don’t know’ responses bear different meanings, and coding should account for this.

Peer Review reports

Background

People with mental illness and those who use psychiatric services have long been stigmatized, which impairs their quality of life. Six percent of the world’s population live with a severe mental illness that measurably decreases their quality of life [1]. A good quality of life has two pillars: a safe, independent home and a decent job opportunity. The stigmatization of mental illness worsens the chances of affected people finding a job or housing [2,3,4,5]. A person living with a mental illness may seem strange to the public: sad or awkward, irritable or irritating. Others seem extremely shy, distrustful, or unexpectedly intrusive [6]. The attitude and reaction of others depend on their prior knowledge, personality, and the actual situation and condition in which they meet the other, as well as the social and cultural traditions that lead them to act, react, or neglect [1, 7].

Although mental illness stigma has been defined to include components of knowledge, attitudes, and behavior, only a few psychometrically tested instruments assess behavioral discrimination [8]. Self-reported discriminatory behavior is limited and presumed, or intended behaviors are less often measured than attitudes [9, 10]. The Reported and Intended Behaviour Scale (RIBS) is an instrument based on the Star Social Distance Scale to assess reported (present and past) and intended (future) behavioral discrimination against people with mental health problems in the general population [11]. The RIBS is the only validated questionnaire that analyzes the presence of reported and intended stigmatizing/discriminatory behaviors against people with mental health problems in the general population [12].

The RIBS was developed in the United Kingdom as a brief and feasible instrument consisting of two parts: reported or actual behaviors and intended behaviors assessed in four different contexts: (1) living with, (2) working with, (3) living nearby and (4) continuing a relationship with a person with a mental health problem [8]. The scale items and response options are listed in Table 2. As the first part is designed to assess the prevalence of these behaviors, psychometric studies mainly focus on the intended behavior part, which has a unidimensional factor structure [8]. The internal consistency for the intended behavior items ranged between 0.75 and 0.95 assessed in various populations, including adolescents, in the following countries: the United Kingdom [8, 11], Japan [13], China [14], Italy [12], the Czech Republic [15], Uganda, Sweden [16], Catalonia [17], New Zealand [18], France [19], Brazil [20], Ukraine [21], Columbia [22], Ghana and Kenya [23]. Cronbach’s alpha values for the whole scale were between 0.65 and 0.79 in the aforementioned Czech [15], in Libanese [24] and in Indian studies [25] and in Macedonian, Turkish, Azeri, Kazah and Polish medical students [26].

The scoring system of the scale appears to be easy to follow, the second part is scored on a 5-point Likert scale, and both contain a ‘Don’t know’ (DK) response option. Several questions arise when the optimal number of answer choices are discussed, such as whether an even or an odd number of response alternatives is ideal. When a middle response alternative is provided, people may choose the midpoint to minimize cognitive costs (satisficing)’, or simply due to fatigue or poor motivation to complete the survey, or it might mean a socially desirable hidden DK [27, 28]. Adding the DK alternative makes it possible to distinguish between true neutral opinion holders and those who do not hold an opinion on the issue. Thus, participants are neither forced to choose among the answer choices nor exposed to the selection of middle-alternative rather than explicitly admitting their ignorance by selecting or volunteering DK. Therefore, providing the DK option appears to be beneficial in reallocating ‘face-saving don’t knows’ from the mid-point to the DK category, which significantly alters descriptive and multivariate inferences [28]. In contrast with the benefits, adding a DK option tends to increase nonresponses, does not improve data quality, and prevents respondents from sharing potentially meaningful opinions, particularly those held with lower confidence [29]. As for DK answer applicability, the results appear to be mixed in the existing literature. The meta-analyses of the amount of random measurement error correlate in numerous survey items, including DK or ‘No-opinion’ responses resulting in significantly fewer random errors compared to the omission of these answers [30]. However, there is evidence of the contrary [31]. Moreover, offering DK options had no significant impact on reliability or validity [32].

There are various guidelines in the literature for evaluating DK responses, including coding them as neutral on a Likert scale, omitting these respondents from the entire sample, and scoring them as missing data that could be replaced by various methods such as mean, median or mode, regression imputation, maximum likelihood or multiple imputation methods [33]. It should be noted that we cannot eliminate potential biases if we decide to replace these data; however, ignoring these responses is likely to reduce the effective sample size, yielding reduced statistical power as well [29, 33]. If scale designers decide to offer a DK option, it is beneficial to obtain substantive data from respondents who opted for this answer choice by asking them a follow-up question, for example, whether they tend to choose one of the substantive response options [34]. The RIBS authors suggested scoring the DK answer as neutral on the Likert scale; however, this does not allow separating the DKs from the neutral answers, which would be beneficial. Albeit their solution does not lead to the reduction of the effective sample size; as the scale authors do not provide the respondents with a follow-up question and recommend using 3 points, we fail to get appropriate information from these two separate response categories. It should be highlighted that none of the above-mentioned psychometric studies had specifically examined the suitability of scoring the DK answers, nor did they investigate the possible differences between the subsample of those who chose DK and ‘Neither agree nor disagree’ (NAND) answer choices for the scorable part of the RIBS.

Aims

Our primary aim was to examine the psychometric properties of the RIBS, validate the scale in a Hungarian non-clinical community sample, and examine the psychometric results in contrast to those in other countries. The secondary aim of this study was to assess the appropriateness of the existing scoring recommendations of DK responses being coded as neutral. We finally aimed to examine the differences between the lived experiences of groups who chose NAND and DK answers.

Methods

Study overview

This was a cross-sectional study that used an anonymous online survey to measure the stigmatizing attitudes towards people with mental illness in a non-clinical population within the framework of the Hungarian National Antistigma Program. The research team contacted the participants by e-mail, and the survey link was also distributed on social media platforms as part of a campaign through the network of the Deep Breath Project (Hungarian psychoeducational media content provider presented on various platforms) and the Hungarian Association for Behavioural and Cognitive Therapies. Additionally, forwarding the link of the survey was also an option to the targeted population; therefore, the convenience sampling method contained snowball sampling techniques as well. The questionnaire package included the Symptom Checklist-90-Revised (SCL-90-R) [35], a 90-item questionnaire used to assess psychological problems, whose Hungarian adaptation was found to be a valid and reliable measurement in Hungary [36]. As the aim of the study was to reach people who did not suffer from any mental illness, according to the design of the study and the guidelines of the SCL-90-R, participants were considered to be non-clinical individuals based on a cut-off point of the Global Severity Index T scores of 63 (219 points on SCL-90-R) [37]. This decision was primarily driven by the need to minimize potential confounding factors that could influence the results. By focusing on a non-clinical sample, we aimed to explore attitudes and behaviors in an everyday context, without the potential influence of formally diagnosed and treated mental health conditions. This could lead to the exclusion of those who may not be aware of their mental health problems, possibly indicating undiagnosed and untreated conditions and vice-versa, respondents above the cut-off should not be considered a formal 'clinical' population due to the lack of formal diagnoses. The study was completed with a repeated-measure segment, where a small proportion of the respondents (n = 17) completed the target questionnaire for a second time using code words for identification to allow the investigation of temporal reliability.

RIBS scoring

The first four items calculate the frequencies of ‘reported or actual behaviors’ among participants who are not necessarily engaged in those behaviors. Therefore, these items are not scored. Items 5 to 8 indicate ‘intended behaviors’. These are scored on a Likert scale from 1 to 5 points depending on the level of agreement of the respondent. Strong disagreement with a statement scores 1 point, while strong agreement scores 5 points. The total score is calculated by adding the response values of items 5 to 8. The scale developers suggest coding the DK answer choice as neutral (i.e., 3 points).

Scale translation

English version of the RIBS was first translated into Hungarian by an experienced English–Hungarian clinical psychologist translator, and then translated back into English by a bilingual English-Hungarian qualified medical interpreter according to INDIGO guidelines [38]. An iterative procedure was used to resolve the discrepancies between the original and the back-translated versions of the scale.

The following sociodemographic details were collected from the participants: age, gender; place of residence, marital status, lived experience of mental health problems, including possible knowledge of friends and first- or second-degree relatives affected by any mental health problems.

Statistical analyses

Demographic data points are expressed as sample size (n) and percentage (%). To describe the age of the participants, mean scores and standard deviations were used. Where the data did not follow a normal distribution, the median and interquartile ranges were used. Confirmatory factor analysis (CFA) was performed to examine the model fit. CFA was performed using diagonally weighted least squares (WLSMV) method, which is specifically designed for ordinal data as the variables are nonparametric. To evaluate the model fit, we calculated the following indices and adopted the generally recommended criteria: chi-square (χ2), root mean square error of approximation (RMSEA, < 0.06), comparative fit index (CFI, > 0.95), Tucker-Lewis Index (TLI, > 0.95) [39]. Standardized model estimates, their standard eror (SE), p-values are provided along with the communalities. To measure internal consistency, Cronbach’s alpha coefficients were calculated for the whole scale, and in cases where an item was deleted (0.70–0.95) [40]. To describe the test–retest reliability, the intraclass correlation coefficient (ICC) was calculated along with the 95% confidence intervals based on a mean-rating (k = 2), absolute-agreement, two-way mixed-effects model (ICC < 0.50 poor, 0.50–0.75 moderate, 0.75–0.90 good, > 0.90 excellent) [41]. The Mann–Whitney U test was used to compare the two groups, the Kruskal–Wallis test to more than two groups, whereas the chi-square test was applied to analyze the ratio of personal affection and sociodemographic data with different scorings of DK answers. Effect sizes were measured by eta-squared (η2) (η2 = 0.01 small, 0.06 medium, 0.14 large effect) [42] and Cramer’s V (V for df 1 0: no association between the variables, 0.1 small, 0.3 medium, 0.5 large effect) [43]. We used IBM SPSS 25 (Apache Software Foundation, USA), MPlus 6.12 (Muthen and Muthen, USA), and SAS OnDemand 9.4 (SAS Institute Inc., USA) software for the analyses.

Results

Participants

A total of 5,701 people completed the online survey. To obtain a non-clinical sample, the SCL-90-R’s Global Severity Index T score of 63 was used as cut-off, resulting in a total of 5,141 participants. Participant characteristics are shown in Table 1.

Table 1 Participant characteristics
Full size table

The vast majority (89.2%) of the participants were female; half of them were married or in a relationship. More than half of them (57.8%) had a friend who struggled with mental health problems. More than one in each three had a family member with mental health problems. Approximately one in five participants had been treated for some mental health problem.

Distribution of the responses

Overall, participants appeared to use the full range of response options (see Table 2), although responses skewed toward a higher level of agreement with the given statements, indicating a potential ceiling effect. The frequency of not knowing the level of agreement ranged from 13.0% to 40.9% for the first four items of the questionnaire and from 4.4% to 12.1% for the second part of it.

Table 2 Response frequencies for the RIBS
Full size table

Results of the confirmatory factor analysis

Only the second four items of the scale should be included in the factor analysis; the scale showed a unidimensional structure on which all items loaded appropriately. The fit indices were: Chi2 = 10.553, RMSEA = 0.031 95%CI(0.015–0.051), CFI = 0.999, TLI = 0.996, and WRMR = 0.006. For the model results that include the standardized estimates, standard errors, the corresponding p-values, along with the communalitities, please see Table 3.

Table 3 Standardized model results
Full size table

Internal consistency

Cronbach's alpha value for the second part of the scale ‘intended behavior subscale’ was 0.884, which is considered ‘good’ according to the ranges defined by Cronbach. As presented in Table 4, item reduction did not lead to an increase in Cronbach's alpha. It should be noted that Cronbach’s alpha values were higher when DK answers were excluded from the sample than when coded as neutral (3 points) and pooled with NAND responses.

Table 4 Internal consistency of the RIBS by using two different scorings for don’t know responses
Full size table

Test–retest reliability

In order to calculate test–retest reliability, a subsample of subjects (n = 17) completed the survey twice with a follow-up of five months (median 153 [23–297] days). The Bland and Altman plot demonstrated a good level of agreement, with no proportional bias (t = -0.971, p = 0.347) (See Fig. 1). The ICC was 0.838 95%CI (0.548–0.942) for the intended behaviors of the RIBS, indicating good to moderate test–retest reliability.

Fig. 1
figure 1

Bland Altman plot of consistency of test scores

Full size image

Differences based on different scorings of don’t know answer choices

For the second part of the scale, a total of 824 people chose the DK option for at least one item, and 93 of them indicated not knowing the answer for all four items. The median total scores were statistically significantly higher when the DK responders were excluded from the sample than when they were coded as neutral (16 (IQR: 13–18) vs 15 [13,14,15,16,17,18] p < 0.0001).

There were no statistically significant differences between Hungarian men and women in terms of the number of DK answers (16.4% of Hungarian men and 16.1% of women answered DK for at least one statement of RIBS second part question, Chi-square = 2.34, p = 0.31, η2 = 0.0002). With regard to education, significant differences were found, where the DK answers were the highest among those who graduated from vocational school (at least one DK answer: 27.7%, at least 3 DK answers: 5.56%) (Chi-square = 41.06, p < 0.0001, η2 = 0.007). As for age, a significant difference was also supported. However, no age group emerged from the sample based on the frequency of DK answers (Chi-square: 126.2, p < 0.0001, η2 = 0.024).

Similar to DK answers, no significant gender differences were found in choosing NAND (Chi-square = 2.07, p = 0.35, η2 = 0.0002). Age and education differences were also consistent with the findings of DK answers (Age: Chi-square = 94.66 p = 0.014, η2 = 0.018, Education: Chi-square = 13.57, p = 0.01, η2 = 0.002). Table 5 shows statistically significant differences in several aspects of lived experiences of mental health problems between individuals who chose NAND or DK for at least for one item and those who did not choose either option.

Table 5 Differences between neutral and don’t know responses are based on the lived experience of the participant
Full size table

Discussion

We tested the feasibility and reliability of the RIBS on a large non-clinical sample in Hungary and evaluated several aspects relating to the reliability and validity of the scale. We specifically focused on the differences between participants who answered NAND and DK in the second part of the scale. As the RIBS had not yet been used in Hungary before, weprovide evidence for the psychometric properties of the scale, particularly in terms of test–retest reliability, internal consistency, and goodness of model fit in this country. In addition, we provide more profound insights into the middle point answer choices and challenge the scoring recommendations.

In summary, our results support the good psychometric properties of the RIBS scale. Overall Cronbach’s alpha was 0.87–0.88, suggesting robust internal consistency, regardless of the way of coding the DK answers. Additionally, the confirmatory factor analysis focusing on the intended behavior scale verified a unidimensional structure) as well as good to moderate test–retest reliability, further enhancing the consistency of literature. Table 6 provides an overview of the investigated samples and the results of the factor analysis of the international studies carried out so far, including our results. The RMSEA varied on a wide range, while the incremental fit indices were excellent in each psychometric study. The overall best model fit was detected in Italy [12], followed by the current Hungarian study results, indicating excellent model fit. The RMSEA was 0.06 in the United Kingdom [11] on an adolescent sample and 0.07 in Brazil [20] as well as in Japan [13], which are considered borderline fit. The RMSEA in the French [19] and Columbian [22] studies was higher than the acceptable range. Notably, the scale author did not include CFA results in their article about the development of the RIBS [8].

Table 6 Overview of the results of the factor analyses on the RIBS in international studies
Full size table

It is important to mention that the median total scores were significantly higher when we excluded the DK responders from the sample than coded them as neutral (16 (IQR: 13–18) vs 15 [13,14,15,16,17,18] p < 0.0001). Although this observation does not necessarily have an impact on the results of the examination of psychometric properties, it can most likely influence the results and the interpretation of the scale when used in a clinical trial. Hence, we do not suggest associating these answers with the value of ‘3’ or labeling them as ‘neutral’, as it would mean confusion of holding a neutral attitude and refusing to answer due to other possible reasons. In terms of motivation, respondents giving DK answers and those who tend to provide NAND with answers were found to have more experience (e.g., living or working with) with people suffering from mental health issues, they were more prone to be neutral, and their answer was more likely to be NAND. This explanation is consistent with our earlier claims on the DK answers, as they also allow the respondent not to have any experience with mental illnesses. In addition to experience, it is worth examining the potential role of sociodemographic factors.

As results show, no differences can be supported between those who are more prone to give DK answers and NAND along the sociodemographic characteristics. Differences were found only in the amount of personal experience with mental health issues, although the effect sizes were small. Nevertheless, the resultsemphasize the relevance of differentiating between the two mentioned response types.

The RIBS aims to assess stigma-related behaviors towards people with mental health problems in general and does not differentiate between diagnostic classes. While there is a growing body of literature on stigma associated with different mental health conditions, it is important to note that the extent and nature of stigma may vary across diagnostic classes. Research has shown that certain mental health conditions, such as schizophrenia and bipolar disorder, tend to be highly stigmatized compared to other diagnoses like depression and anxiety disorders [44]. Schizophrenia is often associated with misconceptions, fear, and stereotypes perpetuated by the media and society at large that individuals with schizophrenia are labeled as dangerous or unpredictable [45], so as are people with alcohol or drug dependence [46]. On the other hand, conditions such as depression and anxiety disorders may be relatively less stigmatized due to their higher prevalence and increased public awareness campaigns [47]. Nonetheless, individuals with these conditions may still encounter stigmatization. The stigma towards different diagnostic classes is mainly measured by case vignettes in the literature. The Mental Health Knowledge Schedule [48] assesses the public's comprehension of mental health and records whether various conditions, such as depression and stress etc., are viewed as mental disorders by the respondent. Although there are tools designed specifically for various conditions, such as depression [49] or suicide stigma [50], currently, there is a lack of stigma measures that assess stigma towards different conditions in a single measure. Future research is needed to gain a more comprehensive understanding of differences in stigma across major diagnostic classes, and the scientific community would benefit from a single measure that examines different components of stigma and across different conditions. The RIBS test can be classified as one which aims to measure the stigma toward people with mental health problems; however, it focuses explicitly on the behavioral aspect rather than capturing the underlying attitudes and beliefs that drive stigmatizing behaviors. Although the RIBS does not distinguish between specific diagnoses, which may result in a limited understanding of condition-specific stigmatization, generalist approaches like the RIBS offer the advantage of allowing an overall perspective of how the general public views or intends to behave towards individuals with mental health problems. This provides an opportunity to compare the viewpoints of communities across different countries and cultures, as various diagnoses may have unique aspects of the stigma associated with them due to societal perceptions and cultural factors.

In addition to examining the psychometric properties of the RIBS scale, our results may provide information about the level of stigma in Hungary, for which to date, only one scale, the Opening Minds Stigma Scale for Healthcare Providers, is available and validated, which is designed to measure the attitudes of health care workers [51, 52]. Highlighting the limitation of non-representativeness of each study for their countries, the Hungarian study population reported higher levels of contact with people suffering from psychiatric disorders in comparison with the subjects of English [8], Italian [12], Czech [15], and Japanese [13] studies.

As for the willingness to have future contact with people with mental illness, Hungarian responders showed less willingness only in comparison with English responders. If we take a closer look at the data, it can be seen that the relationship between more personal experience and willingness is not linear. In this context, the lowest ‘willingness’ scores are usually explained by the recent start of the deinstitutionalization process in the relevant countries. However, Hungarian results appear to contradict this explanation, as in this country, this process has barely started [12]. Another probable interpretation of such discrepancy could be based on the nature of antistigma campaigns. Campaigns in Hungary usually focus on the facilitation of contacts between the clinical and non-clinical population, whereas in other places (e.g., Italy) the focus is on the transfer of relevant and accurate information to the general public, or in other countries (e.g., England) both aspects are emphasized [53, 54]. Although these approaches may provide a satisfactory explanation for the results in general, Hungary appears to be still out of line. This country does not have a national-level antistigma program, and the efficacy of local campaigns may vary on a wide scale. Hence, we assume that the characteristics of the sample might have a significant effect on the results. Female responders and people living in the capital city were overrepresented in the sample. It also has to be highlighted that most responders were recruited as part of an antistigma campaign and through the network of the Deep Breath Project and the Hungarian Association for Behavioural and Cognitive Therapies, which could have narrowed the range of responders to those interested in the topic that has a potential impact on the representativeness of the sample and generalizability of the findings. Furthermore, as these results were found in a non-clinical sample (SCL-90-R Global Severity Index T scores of 63), future research should investigate clinical populations to gain deeper insights into the experiences and attitudes of people with diagnosed mental health problems. Our findings can serve as a basis for such investigations, helping to bridge the gap between non-clinical and clinical populations in understanding stigmatising attitudes and behaviours.

Although this study presents robust data on the Hungarian population, limitations should be mentioned. As previously discussed, female city dwellers were overrepresented in the study due to the nature of the recruitment process; hence, our sample cannot be considered nationally representative. Another significant issue is the small sample size in the test–retest reliability evaluation. We could identify two main reasons behind this limitation. First, the responders usually did not remember the code word; thus, we could not match the test–retest surveys in many cases. Second, responders forgot to complete the questionnaires for the second time. To address these limitations, we plan to use an electronic data capture system (e.g., REDCap) in future research, which allows us to avoid using code words and sends frequent reminders to the responders.

In summary, our results are in line with previous literature in terms of providing further evidence that RIBS is a useful and reliable assessment tool to measure stigmatizing behavior in the non-clinical population. Furthermore, as the first validated assessment tool in this regard in Hungary, the RIBS might be an essential component in future research, as it enables measuring the efficacy of existing antistigma programs in our country and thus can aid the development of more effective approaches to fighting mental health stigma.

Conclusions

In this study, the RIBS was translated into Hungarian, and this translation was validated in a sample of non-clinical participants. Although only minor differences were found in psychometric properties based on different coding patterns, our results suggest that neither agree nor disagree (NAND) and ‘don’t know’ (DK) responses bear different meanings, and coding should account for this. Our findings do demonstrate that the RIBS can be transferred to the Hungarian context. The interplay between lived experiences of participants and their differences in choosing NAND or DK responses to the statements has the potential to inform further research and deepen our understanding of processes involved in stigmatization and thus aid the development of antistigma interventions.

Availability of data and materials

The datasets used and analysed during the current study are available from the corresponding author (DŐ) on reasonable request. The data are not publicly available as the stigmatizing attitudes toward people with mental health problems of the large sample of the Hungarian general population is not yet published.

Abbreviations

χ2 :

Chi-square

η2 :

Eta-squared

AGFI:

Adjusted goodness of fit,

CFA:

Confirmatory factor analysis

CFI:

Comparative fit index

df:

Degree of freedom

DK:

Don’t know

GFI:

Goodness of fit

ICC:

Intraclass correlation coefficient

IQR:

Interquartile range

NAND:

Neither agree nor disagree

RIBS:

Reported and Intended Behaviour Scale

RMSEA:

Root mean square error of approximation

SCL-90-R:

Symptom Checklist-90-Revised

SE:

Standard error

TLI:

Tucker-Lewis Index

WLSMV:

Diagonally weighted least squares method

References

  1. Hinshaw SP. The mark of shame: Stigma of mental illness and an agenda for change. New York: Oxford University Press; 2009.

  2. Link BG, Phelan JC. Conceptualizing stigma. Ann Rev Sociol. 2001;27(1):363–85.

    Article  Google Scholar 

  3. Corrigan PW, Kleinlein P. The Impact of Mental Illness Stigma. On the stigma of mental illness: Practical strategies for research and social change. Washington, DC: American Psychological Association; 2005. p. 11–44.

  4. Corrigan PW, Kerr A, Knudsen L. The stigma of mental illness: Explanatory models and methods for change. Appl Prev Psychol. 2005;11(3):179–90.

    Article  Google Scholar 

  5. Thornicroft G. Shunned: Discrimination against people with mental illness. New York: Oxford university press; 2006.

  6. Sztancsik V, Szemán-Nagy A. The stigma of mental illness - conceptions and phenomena. In: Szemán-Nagy A, editor. Bonding and separation Current problems in Clinical Psychology. Debrecen: Debrecen University Press; 2020. p. 107–16.

    Google Scholar 

  7. Weiner B. An attributional theory of achievement motivation and emotion. Psychol Rev. 1985;92(4):548.

    Article  CAS  PubMed  Google Scholar 

  8. Evans-Lacko S, Rose D, Little K, Flach C, Rhydderch D, Henderson C, et al. Development and psychometric properties of the reported and intended behaviour scale (RIBS): a stigma-related behaviour measure. Epidemiol Psychiatr Sci. 2011;20(3):263–71.

    Article  CAS  PubMed  Google Scholar 

  9. Corrigan PW, Morris SB, Michaels PJ, Rafacz JD, Rüsch N. Challenging the public stigma of mental illness: a meta-analysis of outcome studies. Psychiatr Serv. 2012;63(10):963–73.

    Article  PubMed  Google Scholar 

  10. Fox AB, Earnshaw VA, Taverna EC, Vogt D. Conceptualizing and measuring mental illness stigma: The mental illness stigma framework and critical review of measures. Stigma Health. 2018;3(4):348–76.

  11. Mansfield R, Humphrey N, Patalay P. Psychometric validation of the Reported and Intended Behavior Scale (RIBS) with adolescents. Stigma Health. 2020;5(3):284.

    Article  Google Scholar 

  12. Pingani L, Evans-Lacko S, Luciano M, Del Vecchio V, Ferrari S, Sampogna G, et al. Psychometric validation of the Italian version of the Reported and Intended Behaviour Scale (RIBS). Epidemiol Psychiatr Sci. 2016;25(5):485–92.

    Article  CAS  PubMed  Google Scholar 

  13. Yamaguchi S, Koike S, Watanabe Ki, Ando S. Development of a J apanese version of the Reported and Intended Behaviour Scale: Reliability and validity. Psychiatry Clin Neurosci. 2014;68(6):448–55.

  14. Li J, Li J, Thornicroft G, Huang Y. Levels of stigma among community mental health staff in Guangzhou China. BMC Psychiatry. 2014;14(1):1–7.

    Article  Google Scholar 

  15. Winkler P, Csémy L, Janoušková M, Mladá K, Motlová LB, Evans-Lacko S. Reported and intended behaviour towards those with mental health problems in the Czech Republic and England. Eur Psychiatry. 2015;30(6):801–6.

    Article  CAS  PubMed  Google Scholar 

  16. Hansson L, Stjernswärd S, Svensson B. Changes in attitudes, intended behaviour, and mental health literacy in the Swedish population 2009–2014: an evaluation of a national antistigma programme. Acta Psychiatr Scand. 2016;134:71–9.

    Article  PubMed  Google Scholar 

  17. Andrés-Rodríguez L, Pérez-Aranda A, Feliu-Soler A, Rubio-Valera M, Aznar-Lou I, Serrano-Blanco A, et al. Effectiveness of the “What’s Up!” intervention to reduce stigma and psychometric properties of the Youth Program Questionnaire (YPQ): results from a cluster non-randomized controlled trial conducted in Catalan High Schools. Front Psychol. 2017;8:1608.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Deverick Z, Russell LM, Hudson S. Attitudes of Adults Towards People with Experiences of Mental Distress: Results from the 2015 New Zealand Mental Health Monitor. Wellington: Health Promotion Agency; 2017.

  19. Garcia C, Golay P, Favrod J, Bonsack C. French translation and validation of three scales evaluating stigma in mental health. Front Psych. 2017;8:290.

    Article  Google Scholar 

  20. Ribeiro WS, Gronholm PC, Silvestre de Paula C, Hoffmann MS, Rojas Vistorte AO, Zugman C, et al. Development and validation of the Brazilian Portuguese Version of the Reported and Intended Behaviour Scale (RIBS-BP). Stigma Health. 2021;6(2):163.

  21. Quirke E, Klymchuk V, Suvalo O, Bakolis I, Thornicroft G. Mental health stigma in Ukraine: Crosssectional survey. Global Mental Health. 2021;8:E11. https://doi.org/10.1017/gmh.2021.9.

  22. Campo-Arias A, Augusto Ceballos-Ospino G, Herazo E. Performance of the reported and intended behavior scale among colombian adolescents. Psychiatry Clin Psychopharmacol. 2022;32(2):134–9.

  23. Potts LC, Henderson C. Evaluation of anti-stigma social marketing campaigns in Ghana and Kenya: Time to Change Global. BMC Public Health. 2021;21(1):1–14.

    Article  Google Scholar 

  24. Abi Doumit C, Haddad C, Sacre H, Salameh P, Akel M, Obeid S, et al. Knowledge, attitude and behaviors towards patients with mental illness: Results from a national Lebanese study. PLoS ONE. 2019;14(9): e0222172.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Bharti A, Singh H, Singh D, Kumar V. Behavior of general population toward mentally ill persons in digital India: Where are we? Ind Psychiatry J. 2021;30(1):106.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Munir K, Oner O, Rustamov I, Boztas H, Juszkiewicz K, Wloszczak-Szubzda A, et al. Social distance and stigma towards persons with serious mental illness among medical students in five European Central Asia countries. Psychiatry Res. 2022;309: 114409.

    Article  PubMed  Google Scholar 

  27. Krosnick JA. Response strategies for coping with the cognitive demands of attitude measures in surveys. Appl Cogn Psychol. 1991;5(3):213–36.

    Article  Google Scholar 

  28. Sturgis P, Roberts C, Smith P. Middle alternatives revisited: How the neither/nor response acts as a way of saying “I don’t know”? Sociol Methods Res. 2014;43(1):15–38.

    Article  Google Scholar 

  29. Krosnick JA, Holbrook AL, Berent MK, Carson RT, Michael Hanemann W, Kopp RJ, et al. The impact of" no opinion" response options on data quality: non-attitude reduction or an invitation to satisfice? Public Opin Q. 2002;66(3):371–403.

    Article  Google Scholar 

  30. Andrews FM. Construct validity and error components of survey measures: A structural modeling approach. Public Opin Q. 1984;48(2):409–42.

    Article  Google Scholar 

  31. Alwin DF, Krosnick JA. The reliability of attitudinal survey data: the effects of question and respondent characteristics. Sociol Methods Res. 1991;20:139–81.

    Article  Google Scholar 

  32. Scherpenzeel AC, Saris WE. The validity and reliability of survey questions: A meta-analysis of MTMM studies. Sociol Methods Res. 1997;25(3):341–83.

    Article  Google Scholar 

  33. Kang H. The prevention and handling of the missing data. Korean J Anesthesiol. 2013;64(5):402–6.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Bradburn NM, Sudman S. Polls and surveys: Understanding what they tell us. San Francisco: Jossey-Bass; 1988.

  35. Derogatis LR. Scl-90-R: Administration Scoring & Procedures Manual-Ii for the R(Evised) Version and Other Instruments of the Psychopathology Rating Scale Series. 2nd ed. Towson MD: Clinical Psychometric Research; 1992.

  36. Unoka Z, Rózsa S, Ko N, Kállai J, Fábián Á, Simon L. Validity and reliability of the SCL-90 in a Hungarian population sample. Psychiatr Hung. 2004;19:235–43.

    Google Scholar 

  37. Derogatis LR, Fitzpatrick M. The SCL-90-R, the Brief Symptom Inventory (BSI), and the BSI-18. In M. E. Maruish (Ed.), The use of psychological testing for treatment planning and outcomes assessment: Instruments for adults. Lawrence Erlbaum Associates Publishers; 2004. p. 1–41.

  38. Knudsen HC, Vázquez-Barquero JL, Welcher B, Gaite L, Becker T, Chisholm D, et al. Translation and cross-cultural adaptation of outcome measurements for schizophrenia: EPSILON Study 2. Br J Psychiatry. 2000;177(S39):s8–14.

    Article  Google Scholar 

  39. Hu Lt, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model Multidiscip J. 1999;6(1):1–55.

  40. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297–334.

  41. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2016;15(2):155–63.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Lakens D. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Front Psychol. 2013;4:863.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Cohen J. Statistical power analysis for the behavioral sciences (2nd ed). Hillsdale: L. Erlbaum Associates; 1988.

  44. Wood L, Birtel M, Alsawy S, Pyle M, Morrison A. Public perceptions of stigma towards people with schizophrenia, depression, and anxiety. Psychiatry Res. 2014;220(1–2):604–8.

    Article  PubMed  Google Scholar 

  45. Link BG, Phelan JC, Bresnahan M, Stueve A, Pescosolido BA. Public conceptions of mental illness: labels, causes, dangerousness, and social distance. Am J Public Health. 1999;89(9):1328–33.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. Pescosolido BA, Monahan J, Link BG, Stueve A, Kikuzawa S. The public’s view of the competence, dangerousness, and need for legal coercion of persons with mental health problems. Am J Public Health. 1999;89(9):1339–45.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Angermeyer MC, Dietrich S. Public beliefs about and attitudes towards people with mental illness: a review of population studies. Acta Psychiatr Scand. 2006;113(3):163–79.

    Article  CAS  PubMed  Google Scholar 

  48. Evans-Lacko S, Little K, Meltzer H, Rose D, Rhydderch D, Henderson C, et al. Development and psychometric properties of the mental health knowledge schedule. Can J Psychiatry. 2010;55(7):440–8.

    Article  PubMed  Google Scholar 

  49. Griffiths KM, Christensen H, Jorm AF. Predictors of depression stigma. BMC Psychiatry. 2008;8(1):1–12.

    Article  Google Scholar 

  50. Batterham PJ, Calear AL, Christensen H. The Stigma of Suicide Scale. Psychometric properties and correlates of the stigma of suicide. Crisis. 2013;34(1):13–21.

  51. Őri D, Rózsa S, Szocsics P, Simon L, Purebl G, Győrffy Z. Factor structure of The Opening Minds Stigma Scale for Health Care Providers and psychometric properties of its Hungarian version. BMC Psychiatry. 2020;20(1):1–9.

    Article  Google Scholar 

  52. Őri D, Szocsics P, Molnár T, Ralovich FV, Huszár Z, Bene Á, et al. Stigma towards mental illness and help-seeking behaviors among adult and child psychiatrists in Hungary: A cross-sectional study. PLoS ONE. 2022;17(6): e0269802.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Lanfredi M, Rossi G, Rossi R, Van Bortel T, Thornicroft G, Quinn N, et al. Depression prevention and mental health promotion interventions: is stigma taken into account? An overview of the Italian initiatives. Epidemiol Psychiatr Sci. 2013;22(4):363–74.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  54. Henderson C, Corker E, Lewis-Holmes E, Hamilton S, Flach C, Rose D, et al. England’s time to change antistigma campaign: one-year outcomes of service user-rated experiences of discrimination. Psychiatr Serv. 2012;63(5):451–7.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We are thankful to the Hungarian Association for Behavioural and Cognitive Therapies, which hosted the research project and provided us with an institutional background. We are sincerely grateful to the Mélylevegő project (Deep Breath Project) for their valuable contribution to the survey distribution, which considerably helped us approach a large number of people in Hungary. We are grateful to Sándor Rózsa for his valuable help in suggesting using the communalities beside standardized model estimates. We would like to express our gratitude to all participants who completed the questionnaire.

Funding

Open access funding provided by Semmelweis University. No funding to disclose.

Author information

Authors and Affiliations

  1. Institute of Behavioural Sciences, Semmelweis University, Nagyvárad Tér 4., 1089, Budapest, Hungary

    D. Őri

  2. Department of Mental Health, Heim Pál National Pediatric Institute, Budapest, Hungary

    D. Őri

  3. Department of Psychology, Illinois Institute of Technology, Chicago, USA

    D. Őri

  4. Department of Psychiatry and Psychotherapy, Semmelweis University, Budapest, Hungary

    E. Vass & L. Simon

  5. Institute of Psychology, Eötvös Loránd University, Budapest, Hungary

    K. Vajsz

  6. Department of Clinical Psychology, Semmelweis University, Budapest, Hungary

    K. Vajsz

  7. Institute of Environmental Medicine, Karolinska Institute, Solna, Sweden

    K. Vincze

  8. Department of Psychiatry, Clinical Center of the University of Debrecen, Debrecen, Hungary

    V. Sztancsik

  9. Department of Personality and Clinical Psychology, University of Debrecen, Debrecen, Hungary

    A. Szemán-Nagy

Authors
  1. D. Őri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. E. Vass
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. K. Vajsz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. K. Vincze
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. V. Sztancsik
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. A. Szemán-Nagy
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. L. Simon
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

DŐ was the major contributor to manuscript writing and psychometric analyses. EV significantly contributed to the discussion part of the manuscript and analyzed the demographic data. KVajsz, VS, and ASN did an extensive literature review. KVincze helped to contextualize the findings. LS was the principal investigator of the study and supervised the project. All authors contributed to the study design and the data collection; they have read and approved the final version of the manuscript and have given their consent for publication.

Corresponding author

Correspondence to D. Őri.

Ethics declarations

Ethics approval and consent to participate

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. The study protocol was approved by The Regional and Institutional Committee of Science and Research Ethics of Semmelweis University, Budapest, Hungary (approval number: SE-RKEB: 88/2020). Prior to the enrolment, all of the participants provided their informed consent on the online survey. This study was registered online on Open Science Framework (ID: osf-registrations-jp3vq-v1).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Őri, D., Vass, E., Vajsz, K. et al. Psychometric validation of the Reported and Intended Behaviour Scale (RIBS) in Hungary with a particular focus on ‘Don’t know’ responses and further scoring recommendations. BMC Public Health 23, 1773 (2023). https://doi.org/10.1186/s12889-023-16707-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12889-023-16707-3

Keywords

BMC Public Health

ISSN: 1471-2458

Contact us