The case for burnout has been well established over the last few decades [1, 2] and the term is now well-known and used in general conversation. The World Health Organization, in the ICD-11 (2022), has classified burnout as an occupational phenomenon, specifically describing burnout as a “…syndrome conceptualized as resulting from chronic workplace stress that has not been successfully managed” [3]. Most people have also encountered someone who has been said to be affected adversely by work-related factors described as being ‘burned out’. Indeed, burnout has been linked with various concerning individual outcomes, such as: atherosclerosis [4], depression [5], diabetes [6], and hypertension [7], also in South Africa [8]. Additionally, burnout has also been shown to be negatively related to outcomes important to organisations, such as turnover intention, absenteeism, and productivity [1].
Therefore, the evidence indicates that burnout is an increasing public health concern as it not only incurs individual costs but translates into a burden for healthcare systems (both public and private), impeding organisational success and by extension a country’s economic growth. For example, within the South African context, empirical evidence has been presented that employees who score high on burnout claimed approximately double the amount from medical insurance compared to those who scored low on burnout [9]. Furthermore, with the severe impact of COVID-19 and accompanying lockdown strategies in most countries, financial efficiencies for organisations and governments will become even more important which can only be achieved with motivated and well-functioning role-players in the economy. To offset hurdles to achieve this, the accurate identification of burnout risks, for intervention, becomes increasingly important.
Without question, the Maslach Burnout Inventory (MBI) has served the world well and is the go-to instrument for research and also the identification of burnout – described frequently as the “gold standard” [2]. The WHO also seemingly describes burnout’s components in line with the MBI, that is: exhaustion, cynicism and professional (in)efficacy [3]. But, as research has developed over time some glaring theoretical and practical concerns have been raised: Firstly, the MBI was not initially intended as a tool to help in diagnosing burnout, but for use as a research instrument. Secondly, the conceptualisation of burnout has not been universally accepted and different factor structures for the concept’s operationalization have been argued and established ranging from unidimensional, two factor, three-factor, bifactor, and second-order models. [10]. Furthermore, a key component of MBI-assessed burnout, professional (in)efficacy has been shown to be a divergent factor and perhaps more suitable as an outcome of burnout than a core component [10,11,12].
Criticism even extends to burnout as a concept, with researchers questioning the nosological value of burnout and the accompanying prevalence estimates of the syndrome based on the grounds that no set clinical diagnostic criteria is available with different cut-off criteria being used [13]. This has led to research that posits that burnout is likely a syndrome on the depressive spectrum [14]. However, recent meta-analyses have also indicated that: i) burnout and depression are different and robust constructs [15], and ii) that the empirical relation between burnout and depression as a single point estimate may miss the more complex empirical picture [16].
Furthermore, it has been opined that burnout studies are overly concerned with the psychometric properties of instruments measuring burnout, neglecting the necessary development of theory [17]. Indeed, searching for best-fitting factor structures with statistical modelling, although informative, is mostly a data-driven exercise which often neglects theoretical considerations [18], and can also be sensitive to biases in sampling as the divergent research results do show. Nevertheless, it is important that the phenomenon under investigation is measured accurately to increase the veracity of the field – especially when it concerns employee and by extension public health. Because an instrument exists, or because it was developed first (e.g., MBI), does not mean that it necessarily captures a phenomenon more accurately than a newly developed scale. This would be a logical fallacy – especially in the context of a developing research field and advances in statistical implementations that can more accurately identify dynamics and intricacies at play in an instrument for measuring phenomena which may not have been readily available in decades past. Thus, a more useful approach in the context of the conceptualisation of burnout would be implementing a theoretical approach (scientific modelling) to model burnout to the data as opposed to purely data-driven statistical implementations. Burnout has been defined as a syndrome, that is intercorrelated components that make up a unifying concept (score). Although other burnout scales do exist, these scales either reduce and focus only on the exhaustion component (e.g., Shirom-Melamed Burnout Measure [SMBM] and the Burnout Measure [BM]) or they are based on the conceptualization and components of the MBI with some different phrasing of the items (e.g., Bergen Burnout Inventory [BBI]) [19]. All in all, there was a need for an actualised conceptualisation of burnout to investigate if all the important aspects have been covered and included.
The Burnout Assessment Tool (BAT-23) was developed using a combination of an inductive and deductive approach [19]. For the inductive approach a total of 49 practitioners were interviewed that included general practitioners, psychologists and occupational physicians who are involved at the start, middle and end of the burnout process [19]. And for the deductive approach a total of 357 items and 66 dimensions were screened before drafting the initial items of the BAT-23 – for a comprehensive description of the need for and development of the BAT-23 see the seminal article by Schaufeli et al. [19]. BAT-measured burnout is defined as: “a work-related state of exhaustion that occurs among employees, which is characterized by extreme tiredness, reduced ability to regulate cognitive and emotional processes, and mental distancing” [19]. Specifically, BAT-assessed burnout is an overall score based on four scale components. The first two components are the core components of exhaustion (feeling depleted and the inability to expend effort) and mental distance (unwillingness to expend effort, cynicism) in line also with previous research [20]. The two novel components that emerged are closely linked to exhaustion as they indicate a lack of proper management of cognition and emotions – tying into the deductive reasoning of inability and unwillingness. This was also commensurate with the inductive approach taken which included interviews with health professionals who have actively worked with burnout patients: Cognitive impairment (reduced functional capacity to adequately regulate cognitive processes) and emotional impairment (reduced functional capacity to adequately regulate emotional processes) [19, 21]. Thus, the four inter-correlated dimensions constitute one higher-order conception of burnout.
Researchers from different countries are currently conducting research with the BAT-23 and the current evidence for its validity is strong. For example, published research has shown the BAT-23 to be valid and reliable in six European countries with representative samples [22], in Japan [23] and in Korea [24]. Additionally, the BAT-23 has also been shown to be valid with Rasch analysis in both the Netherlands and Belgium (Flanders) [25]. However, no study has been conducted on the validity of the BAT-23 in Africa and the consideration of context in psychometric assessment has been shown to be important in order to offset potential biases and ensure the fair measurement and comparison between groups [26].
South Africa is a diverse multi-cultural context and the Employment Equity Act (EEA) Sect. 8 (Act 55 of 1998) requires that any psychometric instrument used in organisations, or otherwise, must be able to present evidence that it is: 1) valid and reliable, and 2) not be biased against any group or person [27]. Indeed, research has shown that levels of burnout is nuanced between gender and ethnicity and may (or may not) differ [28, 29] – indicating that equivalence of the measuring instrument is important for comparison to be accurate and fair. To this end, the construct-relevant multidimensionality of the BAT-23 is investigated with latent variable modelling to ascertain the most appropriate model in the data. Next, the convergent validity of the global burnout score of the BAT-23 is compared to that of the MBI. Then, to address the second requirement of fairness of the measurement, we also tested the measurement invariance of the BAT-23 for gender and ethnic groups.
Thus, the current study aims to investigate the validity, reliability, and measurement invariance of the Burnout Assessment Tool (BAT-23), a recently developed scale, within the South African context. To achieve these aims the following hypotheses are stated:
-
H1: A hierarchical model of BAT-assessed burnout fits the data.
-
H2: The BAT shows acceptable reliability coefficients.
-
H3: There is convergent validity between the BAT and the MBI.
-
H4a: The BAT shows invariance across gender.
-
H4b: The BAT shows invariance across ethnicity.