Eight-year experience of maternal death surveillance in Morocco: qualitative study of stakeholders’ views at a subnational level
BMC Public Health volume 22, Article number: 2111 (2022)
Since 2009, Morocco has been implementing the Maternal Death Surveillance System (MDSS). The results obtained indicate significant regional variations in terms of implementation stage, completeness of maternal death reporting, and information use for action. The objective of this research is to better understand the contextual factors involved in the implementation process and use of MDSS, with a focus on the facilitators and barriers, as experienced by stakeholders in health regions.
Evaluation research was conducted in 2017 based on a descriptive qualitative study using semi-structured in-depth interviews, in four out of the twelve health regions of Morocco. A total of thirty-one in-depth interviews were held with members of regional committees of maternal death reviews (RC-MDR) and other key informant staff. Interviews focused on participants’ views and their experiences with the MDSS since the introduction in 2009. We conducted thematic analysis relied on inductive and deductive approaches. Applying the Consolidated Framework for Implementation Research guided data analysis and reporting findings.
Engaging leadership at all health system levels, regular training of district and regional MDSS coordinators and supportive supervision at a national level were the most important MDSS implementation facilitators. Reported barriers were essentially related to the review system: Irregular review meetings, blame culture, high turn-over of RC-MDR members, lack of analytical capacity to inform the review process and formulate recommendations, finally limited accountability for recommendation follow-up. While financial incentives boosted MDSS adoption, they were nonetheless a substantial barrier to its sustainability.
The MDSS is a complex process that requires taking numerous steps, including the commitment of multiple stakeholders with varying roles as well as information sharing across health system levels. Contextual factors that influence MDSS implementation at the sub-national level are to be considered. Horizontal and vertical communication about MDSS goals and feedback is crucial to strengthen stakeholders’ commitment, hence improving quality and use of MDSS. Furthermore, health regions should place emphasis on making high-quality recommendations in partnerships between the regional management teams, RC-MDR members and external stakeholders.
Contributions to literature
• Barriers and facilitators of MDSS implementation were deeply analyzed at a subnational level using the Consolidated Framework for Implementation Research (CFIR).
• The research highlighted stakeholders’ requirement for implementing MDSS, And their subjective experiences with sustaining such a system
• Ethical and legal issues should be anticipated to ensure effective and sustainable MDSS implementation.
• The study indicates that the main factors enabling MDSS implementation include MOH support, skilled and trained staff, communication regarding MDSS goals and feedback and sub national stakeholders’ early involvement in sustainability discussions.
As part of the maternal mortality reduction strategy, low and middle-income countries (LMICs) have been working to establish comprehensive Maternal Death Surveillance and Response (MDSR) systems since 2013. However, glaring gaps exist between the development of the national MDSR policies and their in-country implementation [1, 2]. Operationalize maternal death review committees and implementing actions in response to recommendations are a main shortcoming in this system . In 2017, LMICs were urged to include perinatal deaths (MPDSR) in their surveillance systems, which added to the challenges [3, 4].
For more than a decade, Morocco had no prevalent harmonized or standardized process for examining and studying maternal deaths. This depended on individual health facilities performing clinical audits (audit research studies or pilot projects funded by the Ministry of Health (MOH) and its partners). Following their apparent success, the MOH called for scaling up voluntary clinical audits in obstetrics in March 2001. However, few maternity hospitals began audits and even fewer have continued them [5,6,7,8].
With the announcement of the health policy for 2008–2012 and the increasing national focus on accelerating maternal mortality reduction, the MOH sought to operationalize a National Maternal Death Surveillance System (MDSS) . In 2008, the MOH mandated the completion of notification forms for every death of women of reproductive age (WRA). It instructed health regions and districts to adhere to the national MDSS guidelines, issued in 2009, based on training provided to all district and regional MDSS coordinators . The Minister of Health appointed a national experts committee (NEC) on confidential inquiries into maternal deaths (CIMD) and RC-MDR. Three national CIMDs (2009, 2010 and 2015) have been conducted to date, and all emphasized that most deaths could be prevented if proper actions are taken in health care facilities [10, 11].
Nevertheless, it is important to examine the weaknesses in the current MDSS. In 2017, the MDSS was still operational in ten of the twelve regions. In seven regions, all MDSS components were implemented. Only four regions successfully integrated MDSS activities into routine practices . Significant under-reporting, lack of investigation and review of maternal deaths were barriers to a well-established national system [1, 11, 12].
For such a public health surveillance system to have a true impact, stakeholder commitment throughout the surveillance process is necessary. Stakeholders can help interpret the reported data based on their knowledge of the health outcome or the environment where the data is collected. They may also respond to the information generated by the surveillance system and recommend or influence the surveillance system evaluation to ensure that it meets its goals .
Stakeholders and their relationships are at the heart of the implementation process, as are implementation climates and communication channels . Yet, there is a lack of structured research based on stakeholders’ needs in building MPDSR and their own subjective experiences related to sustaining it .
The purpose of this study is to understand stakeholders’ views at the regional health level related to facilitators and barriers of MDSS implementation and its use between 2009 and 2017. It is Morocco’s first qualitative MDSS evaluation with the aim of generating evidence-based recommendations to strengthen the MDSS. The findings can thereby guide policymakers in Morocco and other LMICs, and help plans and adjustments to make this system more useful at the subnational level.
We adopted a descriptive qualitative research design [15, 16] to describe and explore facilitators and barriers to MDSS implementation in health regions, as well as to make evidence-based recommendations for MDSS improvement.
Such descriptive qualitative research is relevant for summarizing and understanding a topic by recognizing its subjective nature as well as the viewpoints of those involved [15, 17]. The focus on producing rich descriptions of a phenomenon from experienced people offers a unique opportunity to get inside knowledge and learn from how they perceive their context [18, 19]. The findings closely reflect the initial research question and are of particular interest to practitioners and policymakers [15, 20].
Because they build on prior exploratory work, descriptive qualitative studies may be utilized to conduct a more focused investigation. They may, however, generate unexpected results when a phenomenon is explored in new contexts . This flexibility allows for adjustments in research focus and methods as empirical findings emerge. The concepts or phenomena studied can guide decision-making throughout the research process, allowing the most relevant methods to answer the research question .
Study setting and recruitment
Considering the nationwide implementation of MDSS, we selected four regions using purposive sampling among the ten regions that demonstrated evidence of maternal death surveillance practice in 2017. An additional Excel file shows this in more detail (see Additional file 1). To maximize data heterogeneity, we examined MDSS implementation level and monitoring MDSS indicators as well as population’s vulnerability. The characteristics of the study regions selected are described in Table 1.
The regional MDSS coordinators (RCo) supplied a list of stakeholders involved in MDSS management and/or conducting MDRs (RC-MDR members). We defined stakeholders at the health region level as those individuals who contribute to the maternal death surveillance and/or review process, use the results obtained, or have influence due to their position over the MDSS’s implementation and sustainability [13, 21]. The stakeholders include district and regional maternal health managers (tasked with data collection and/or MDRs), practitioners who are members of the RC-MDR and district or regional public health officials. Among forty-seven potential participants, we could not find contact information for eight health workers due to retirement, departure for private practice, or transfer to another region. Thirty-nine were invited all to take part in the scheduled interviews with facilitation from RCo. All agreed in principle to attend, except for one anesthesiologist-resuscitator (AN-RE), who declined the invitation and no longer wanted to be a RC-MDR member. The principal investigator (PI) met with the interviewees at the health region’s office. She called the ten people who did not show up that day to schedule another appointment. Three participants were interviewed via phone: a regional health director (RHD), an obstetrician-gynecologist (OB-GYN), and a head of the regional health observatory (RHO). Even after constant callbacks, seven participants were unable to attend: One RHD, three OB-GYNs, one of whom left for the private sector, two AN-REs, and one midwife.
Data collection and reflexivity
The data collection method chosen was in-depth interviews (IDIs). This method is particularly suitable for letting the respondent express him/herself in his/her own words, to be able to bring out original research material .
To further understand the MDSS implementation process, a topic semi-structured guide was prepared after reviewing the national MDSS guidelines. Steps of the maternal death surveillance cycle were all covered (Fig. 1).
The topic guide addressed the following key themes: How was the MDSS initiated in the region? How were participants initiated and introduced to their roles? How was maternal death notified and investigate? How were MDR meetings held, and aggregate data analyzed? How were recommendations formulated and implemented?
The interview guide focused on barriers and facilitators of MDSS implementation and use. However, due to the IDI’s interactive nature, questions could be changed or added considering data obtained from earlier interviews . The full topic guide can be found in Additional file 2.
The PI (SA) collected data between May 2017 and August 2017. She led IDIs in person at the health region office in a private place, depending on respondent’s availability and willingness to participate. Interviews were conducted in French and tape-recorded with the participants’ verbal consent. The interview was concluded once no added information regarding MDSS seemed to emerge, and the interviewer had a full understanding of the participant’s perspective. Only four stakeholders did not give their verbal consent to be recorded. In this case, the interviewer took detailed notes during the discussion and reconstructed them shortly after completing the interview.
Researchers also reviewed additional documentation when available (i.e., MDSS action plans, monitoring indicators, RC-MDR meeting notes, regional reports on CIMD) to complete interviews and acquire an in-depth understanding of participants’ key statements.
Two student midwives transcribed recorded interviews into French in Microsoft text format. The PI (SA) cross-examined all transcriptions for consistency and accuracy before she imported them to N VIVO 11 Starter for data management.
The exploitation of the data was that of thematic analysis using inductive and deductive approaches. In 2006, Braun and Clarke clarified the conceptual basis and practical aspects of thematic analysis in the field of psychology . Although various authors were already making extensive use of it in the field of sociology. The aim was to generate new knowledge from inductive reasoning, reasoning that consists of moving from the specific to the general. The mix of an inductive and deductive qualitative approach combines the advantages of inductive reasoning: generating new knowledge and consolidating it with a theoretical framework, as Pearse states: “deductive qualitative research, the theoretical propositions derived from a review of the literature serve as its departure point, informing how the data is collected” . As stated by Casula and al, to encompass a deductive approach to inductive may increase the scientific rigor, while allowing new theory or new hypotheses to emerge .
Two authors (SA and VDB) separately coded first inductively the facilitators and barriers influencing MDSS implementation. Then, they deductively mapped them to the CFIR domains and constructs. Data reliability was confirmed by comparing the encoding produced independently. Agreements and discrepancies were documented, and any divergence was resolved by consensus, through discussion within the research team.
The theoretical framework for research was adapted from the Consolidated Framework for Implementation Research (CFIR) [26,27,28,29]. Thirty-nine CFIR constructs are organized into five domains: (1) intervention characteristics, (2) outer setting, (3) inner setting, (4) individual characteristics, and (5) process . The CFIR constructs by domain along with a brief description are listed in Additional file 3.
Domain 4 (individual characteristics) was not applied because our analysis unit was the health region rather than the individual. A codebook was developed and refined to describe the constructs selected, those rejected or merged, as well as the underlying justification (See Additional file 4). Figure 2 illustrates the conceptual framework used to guide data coding, data analysis and findings reporting.
The adapted CFIR contained four domains (intervention characteristics, outer setting, inner setting, and process) and fourteen constructs. The MDSS is a multifaceted intervention with six interacting components. Its characteristics (relative advantage, adaptability, complexity, and quality design) may influence implementation in a health region (inner setting). The healthcare system is hierarchically organized with interrelationships between health regions and the national level. Changes in the outer setting (extra organizational communications, political and hierarchical pressure, and external policies and incentives) can have an impact on implementation, often mediated through changes in the inner setting (structural characteristics, networks and communications, culture, implementation climate, and readiness for implementation). Successful implementation usually requires achieving individual and organizational level use of the intervention as designed. Figure 2 represents the implementation process by an arrow that symbolizes the engaging, executing, reflecting, and evaluating implementation, aimed at the direction of the inner setting for an effective implementation.
Results were presented in narrative format following the CFIR Framework stages. We determined what qualified as a theme when a similar finding came from different respondent profiles and when it could shed interesting light on the research question and thus offer an angle of analysis for our results. The tree node structure developed for thematic analysis is summarized in Additional file 5. Verbatim that best illustrated the significant points shared by most respondents were selected and translated into English by the PI (SA). Co-authors discussed the translation, which has been revised by a professional translator. The Consolidated criteria for Reporting Qualitative research (COREQ) (See Additional file 6) were used to structure reporting methods and the findings .
A total of 31 participants were interviewed: 14 women and 17 men. They were aged from 30 to 60 years old (median 46) and had an average of five-year experience with MDSS (min: 1, max: 8). The interviews lasted 67 min on average, although they ranged from 20 to 120 min (Table 2).
Facilitators and barriers to MDSS implementation
Findings are based on four investigated CFIR domains: (i) Intervention characteristics, (ii) Outer Setting, (iii) Inner Setting, (iv) and Process. Accordingly, we present respondents’ understanding of facilitators and barriers to initiating and sustaining MDSS, as well as suggestions for future improvement.
The first MDSS characteristics domain includes three constructs: relative advantage, adaptability, and complexity and quality design.
The MOH developed and implemented the MDSS in 2009. Almost all interviewed stakeholders agreed that the MDSS was better than the status quo or clinical audit. The most significant MDSS advantages were increased awareness of maternal mortality issues among health personnel, improved understanding of maternal deaths and families’ experiences, and enhanced communication between health facilities and districts. The MDSS also contributed to improve working conditions and reduce intra-hospital maternal mortality.
- “This system has truly benefited our facilities. The hospital manager has upgraded the surgery room. Regional reviews were quite valuable. For example, there was no death this year (2017) in the aftermath of nine deaths in 2016.” (An OB-GYN–M–3 years (Y) of MDSS experience).
Allowing districts and health regions to adjust the MDSS to local constraints was identified as a facilitator for implementation. In some districts, the DCo performed preliminary inquiries and verbal autopsies before families left the hospital, despite the allowed 6-week limit set by national guidelines. In all regions, RC-MDR now meets once or twice a year instead of quarterly.
- “Municipalities report women’s deaths to the health districts, but not to the primary healthcare facilities ... I have to perform preliminary investigations and verbal autopsies at the hospital, because no one else will.” (A DCo–F–7 Y).
Complexity and quality design
Stakeholders perceived the MDSS’s quality design and complexity as implementation barriers. Data collection was time-consuming and labor-intensive due to difficulties in identifying maternal deaths and filling out excessively long questionnaires. The lack of a high-performance IT application was to blame for delays in data aggregation and analysis across districts.
- “The system is complex, and the questionnaires are lengthy. The MOH must revise and reduce them to strictly the most important variables.” (A RCo–M–8 Y).
This domain considers factors which are external to the health region: political and hierarchy pressure, national policy and incentives, and extra organizational communication.
Political and hierarchy pressure
From 2008 to 2012, there emerged a strong government determination to reduce the maternal mortality, which was an essential facilitator of on-the-ground implementation. All stakeholders involved in MDSS during this period were convinced that the national political commitment and pressure from top Health Ministry Officials inspires regional and local “leadership commitment”.
- “We always follow the Minister’s strategy. Human resources and maternal mortality are the two top priorities.” (A RHD–M–6 Y).
National policy and incentives
In 2009, 2011 and 2015–2016, the MOH provided MDSS training for DCo and RCo, with support of development partners such as WHO, UNFPA and UNICEF. Financial incentives were reserved to DCo and professionals in primary healthcare facilities to fund costs related to data collection on WRA and pregnancy-related deaths. The same applied to RC-MDR members traveling to the regional health directorate to review maternal death records. However, these incentives are provided as part of projects involving partners, which implies that they were not given to all districts on a regular basis. While financial motivation was acknowledged as an implementation facilitator, stakeholders stressed the negative effect of the one-off reward on MDSS sustainability and integration into routine practice.
Several stakeholders also raised concerns about failure to implement the overall healthcare system development policy. As a result, women do not receive prompt life-saving healthcare, which is critical to the MDSS's relevance and, hence, its sustainability.
- “Maternity wards meeting standards and other planned interventions as part of maternal mortality reduction action plan (2008-2012) were foiled. Nothing materialized in practical! The MDSS is one part of the strategy to reduce maternal mortality. It cannot be developed on a standalone basis; it is part of a full package! ” (A DHD–M–6 Y).
Extra organizational communication
The appointment of a national coordinator at the MOH level facilitated intensive communication and strong networking with DCo and RCo during early MDSS implementation stages.
- “ In 2009, the MOH ensured that MDSS adoption was closely monitored. DCo have frequent telephone conversation and email communication with the national MDSS coordinator. Districts and regions are now more connected with one another.” (A regional maternal health program manager–F–7 Y).
Informal channels of communication improve collaboration with external actors, particularly local authorities, and thus WRA death reporting. However, because of the risk of penalties for health practitioners, no feedback is provided to territorial collectivities.
- “ We organize meetings to make local authorities aware of certain issues, but we cannot share the full report with them. It is a double-edged sword. They would focus on the report statement that “95% of deaths are avoidable”. Therefore, fault is attributed to the healthcare system and health professionals.” (A RCo–M–8 Y).
The third domain consisted of five constructs referring to factors that were internal to the health region: structural characteristics, internal communications, cultural environment, implementation climate, and readiness for implementation.
Most stakeholders mentioned staffing shortages as a major barrier to MDSS implementation, especially those related to applying recommendations. Respondents in one region with low staff turnover reported an important level of workplace camaraderie, and teamwork positively influenced MDSS implementation and sustainability.
- “Most of the personnel come from neighboring areas. We have been working together for a long time. We are knowledgeable about the regional specificities. This familiarity makes work easier.” (A HFNS manager–M–8 Y).
The important formal and informal communication links between health regions and districts are influencing MDSS implementation and sustainability and cannot be overstated. Regular health managers’ meetings at distinct levels of the health system were considered as implementation facilitators, just as the absence of these meetings was viewed as a barrier by hospital medical and nursing personnel.
- “It is simply a matter of lack of communication. They inform us of a meeting on the same day or the day before. So, I am no longer able to attend most of these meetings because I cannot arrange for them on such short notice. Moreover, we do not receive a copy of the regional report in advance.” (An OB-GYN–M–4 Y).
The blame culture was a major source of concern and still is today in public health services. Health workers feel afraid of maternal deaths when managers refer to them as murders. Midwives are unable to speak openly about causes and circumstances of maternal deaths.
- “ Every time a woman dies, we are afraid of being punished. The head nurse was turning people against us. Doctors were trying to blame us. This is no longer the case; the current hierarchy, as well as doctors support us.” (A Head nurse of a maternity hospital–F–3 Y).
The RC-MDR meetings were not open and constructive, and members sought to find the culprit. Health professionals’ defensive attitude made it difficult to accept the questioning of practices. The external experts’ (university hospital professors) presence in meetings was valuable in that regard.
- “It is very important to have an external view to guarantee transparent and beneficial reviews, especially when the hospital staff has a defensive attitude...” (A GYN-OB–M–4 Y).
While some argue for MDSS incompatibility with the urgent need for immediate maternal death information because they face immense pressure from policymakers, the community, and the media, all stakeholders recognize that MDSS is an added workload. They experience difficulties incorporating MDSS activities into their workflow and information system.
- “ We are adding new processes, therefore the personnel must work harder… We need to combine efforts and use a single data collection form. Teams lack time! “ (A DHD–M–6 Y).
The relative priority, i.e., the shared perception of the importance of the implementation within the health region, had a detrimental effect on MDSS sustainability. At the start, national strategic and political priorities drove the “push” towards adopting the MDSS. When the MOH introduced other interventions, a lower level of priority was given to MDSS.
- “ We spent our days in 2016 and the first quarter of 2017, collecting data from health programs to update the national health information system. We also had to prepare the regional healthcare provision plan.” (A RCo–F–8 Y).
The lack of internal organizational incentives and rewards was a strong barrier to MDSS integration and sustainability. As health workers saw MDSS as requiring extra efforts, they expected some form of compensation.
- “ Due to a staffing shortage in one district, environmental health coordinators agreed to perform preliminary home inquiries into WRA deaths rather than primary healthcare workers doing it. Initially, the MOH offered a cash incentive. But, when such a payment was removed, they refused to complete the task since it was outside of their scope of work.” (A RCo–F–8 Y).
All stakeholders involved in the MDSS implementation were not aware of the MDSS goals. Managers were more informed than healthcare providers. They prioritize counting of maternal deaths over MDRs and recommendation implementation.
- “ The problem is not just that maternal deaths are under-reported in comparison to expectations. We must also discuss what we did in response to the identified maternal deaths.” (An OB-GYN–F–7Y).
Feedback was given through regional coordination and task force meetings. Several interviewees expressed disappointment regarding the time spent at these meetings on reflection on the issue and collective evaluation of the MDSS. This limited the opportunity for organizational learning based on MDSS data.
- “What task force? They did not comment on the regional report! This kind of committee only complicates matters.” (A RCo–F–6 Y).
Readiness for implementation
Engaged leadership, access to adequate time, financial and material resources, as well as access to knowledge and information supported the initial stages of MDSS implementation.
- “Our partner gave us around 3,000 euros in 2015. It was enough to meet expenses. MDSS benefited from this financial support.” (A RCo–F–8 Y).
The lack of additional resources for MDSS was a barrier to sustainability. A minimum amount of funds is required to cover field investigators’ travel expenses, organize RC-MDR meetings, analyze data, organize training, and edit the regional report.
- “We meet in an unpleasant setting, a small room. We hold meetings in the afternoons and go beyond 4:30 pm. I have nothing to offer RC-MDR members except chocolate that I bought myself. They are pleasant people with whom I get along well. So, they agree to coming back anyway” (A RCo–F–8 Y).
On one hand, almost all DCo and RCo agreed that successive training sessions (in 2009, 2011 and 2015–2016) helped in dissemination of information and knowledge. The continuous implementation support from the national MDSS coordinator participated in keeping them informed and building momentum. On the other hand, health professionals do not appear to have the same level of knowledge and information. Several RC-MDR members have expressed an interest in MDR and data analysis training. They also sought guidelines for reporting, presenting, and wording of recommendations.
- “ I worked as a ward midwife in 2009... At the HFNS meeting, only the hospital’s maternity ward team leaders were present. Midwives were unaware of MDSS. When I was appointed to manage the maternal health program at HFNS in 2013, they handed me the MDSS guidelines. Then, in 2015, I attended the DCo’s training.” (A DCo–F–4 Y).
The implementation process was considered crucial in terms of engaging relevant individuals and executing and evaluating the MDSS.
Several respondents emphasized the importance of formally appointing key stakeholders, particularly RCo, DCo and RC-MDR members. They recognized the RCo as an essential actor based on the backing of hierarchy, and his/her relationships with district teams and external actors. RCo were all trained and had worked, except for one, on MDSS for a long time (at least six years). However, when the MDSS depends on a single person, continuity is no longer ensured if she/he leaves.
- “The stability of the RCo implies that he has a wide range of interpersonal relationships. He benefits from hierarchy’s support, regional personnel’s trust, and a well-developed network.” (A HFNS manager–M–8 Y).
The maternal health program manager and the HFNS manager have a substantial effect on the success of MDSS implementation at the district level. Training, regular monitoring via quarterly reports, supportive supervision, coordination meetings, and other strategies, such as mentorship, are all needed to maintain commitment.
- “The district maternal health program manager and the head of HFNS are key players of the MDSS at the district level. Those trained in 2009 and 2011 were committed. Others who came afterwards were not. They invented excuses: "I am not trained". The training in 2016 helped revitalize MDSS in districts.” (A RCo-F–1 Y).
Many RC-MDR members saw their involvement in MDSS as theoretical. They stated how the RCo just approached them by phone to join the RC-MRD. They were discouraged when not invited to meetings in timely manner, when they did not receive the report for review or were not asked to provide feedback prior to the task force meeting. Some interviewees questioned the members’ selection process and underlined the role of university hospital academics in ensuring an objective review of deaths at regional meetings.
- “RD-MDR members should be unanimously chosen with the support of their colleagues. Furthermore, due to recurring confrontations, the presence at the meeting of individuals from outside the region, with scientific credentials such as a university hospital senior professor, is preferred.” (A RHD–M–6 Y).
Executing, reflecting, and evaluating
The level of MDSS implementation varied within and across the four regions, but all demonstrated evidence of practice. Both WRA and maternal deaths were under-reported according to DCo and RCo. The preliminary inquiries to identify suspected maternal deaths were not systematic. Fear of maternal complications or blame for a maternal death led to unnecessary referrals to the next level of care, increasing the workload on the referral hospital.
According to managers, RC-MDR members blamed deaths on factors that were frequently beyond their control, such as lack of prenatal care, community challenges, and infrastructural concerns. Although hierarchy contested those conclusions, RC-MDR members were frustrated that their recommendations had not been adopted.
- “What frustrates me is that after all these years, I still do not see any added value due to my work nor any progress to justify my sacrifices. The committee detects flaws and suggests feasible solutions that we are unable to implement. So, year after year, the same problems and death causes reoccur.” (A RCo–F-8 Y).
This study provides an in-depth qualitative evaluation of the MDSS implementation process using the CFIR as a theoretical lens. Potential facilitators of MDSS implementation included the intervention’s adaptability and its relative advantage compared to current practices (intervention characteristics), political and hierarchy pressure and communication with the national level (outer setting), access to knowledge and information (inner setting), and RCo and DCo level of commitment (process). Potential barriers included perception of the intervention as complex (intervention characteristics), not always compatible with current workflows and a prevalent blame culture (inner setting), competing priorities of the MOH (outer setting), weak RC-MDR and insufficient support for implementing recommendations (process).
A complex surveillance system
Political commitment and hierarchy pressure were the main reasons for the early MDSS adoption in less than a year. In several LMICs, the “top-down approach” backed by strong political and administrative buy-in and national leadership enabled the MDSR implementation [31,32,33,34]. However, both top-down and bottom-up approaches are required to ensure successful implementation and sustainability [31,32,33, 35].
Building a sustainable MDSS was difficult in the four regions of our study due to its complexity, concurrent political priorities, and lack of resources. Complexity arises from two MDSS components that are interrelated. The district surveillance and MDRs are combined to inform regional and national CIMD. The review system, which is the responsibility of the health region, should be kept separate from the district’s surveillance system. Districts must prioritize the use of medical certificates of death cause, particularly in hospitals and communes served by health offices. Because more than half of pregnancy-related deaths occurred in intensive care units, operating theaters, maternity wards, and emergency services, physicians and nurses working in these settings should be targeted .
Maternal death surveillance is time-consuming at the district level because of the lengthy data collection tool. The questionnaire covers detailed medical information about the woman and her death, which data is collected through hospital records, staff interviews, and/or verbal autopsies involving family members. This information is essential for the RC-MDR to determine whether the death was caused by avoidable factors. Furthermore, once the file is de-identified at the district level and centralized in the region, it is no longer possible to obtain additional information from DCo. This issue must be addressed: How to balance competing demands for simplicity, quality, and exhaustivity when the district does not directly use the collected data? A brief questionnaire might be integrated into daily work and timely completed with other specific forms, depending on regional priorities. Since 2015, a web-based system is in place in seven regions. However, it is not yet fully functioning. Countries with well-established MDR systems use technology to facilitate data analysis, allowing for faster aggregation and analysis of data sourced from districts or states [2, 36, 37].
The ability to maintain trained staff on surveillance and MDRs was a concern recognized by all health regions. Getting funding to keep training and supplies going was challenging. This highlights the importance of regions launching local programs of training, including web-based training, to address issues such as under-trained staff and high staff turnover. Continuous technical skill development planning is required, as is a mechanism to assess stakeholders’ knowledge level [38, 39].
Overcoming blame culture
Lack of confidentiality and blame culture continue to be big issues for efficiently engaging stakeholders in MDRs, sharing MDSS results, and implementing recommendations. The use of a systems-theoretical approach to MDRs should help identify causal factors at all levels of the system, learning from mistakes, and designing system-level changes to avoid these concerns in the future . Despite consensus that the “no blame” model is far more successful in making health care facilities safer for women giving birth, managers have yet embraced the model. A punitive malpractice regime still dominates the error landscape [8, 41,42,43,44]. The healthcare system should strive towards a culture that accepts human errors as part of human nature and encourages people to discuss them without fear of repercussions. Achieving the right balance between “no blame” and accountability is challenging. Debate should address rather than a “no blame” culture, a “just culture,” which distinguishes between blameworthy and blameless conducts [45, 46]. Women’s safety requires ongoing efforts to improve practices, training, information technology, and culture throughout the hospital, not only in the maternity ward. Professional norms that see patient harm as a social problem should guide efforts. Solutions can be found if healthcare workers and managers work together [46,47,48].
Buy-in by health professionals and external actors
Our study revealed that most healthcare providers did not recognize MDSS benefits in their work, MDSS being too slow to produce practical results. They struggle to distinguish between the regional CIMD and the clinical audit approach. The lack of feedback from RC-MDR and NEC regarding MDSS performance hinders the implementation process, highlighting the benefit of establishing feedback and communication mechanisms. Both healthcare providers and managers are frustrated because community, medias and policymakers do not appreciate efforts to reduce maternal deaths. The MOH should encourage regions to share maternal death surveillance findings at international and national conferences and meetings, to increase awareness and understanding of maternal deaths occurring locally. Regional communication that includes external stakeholders is crucial for supporting motivation and building a learning culture. This suggests the importance of managing expectations and finding ways to engage with district and regional staff.
Incentives were crucial factors in both outer and inner settings. In 2009 and 2010, the MOH provided technical and financial incentives to DCo in all districts to encourage them to collect data for MDSS. In 2015, only six regions got onetime cash incentives. This short-lived support was not sufficient to ensure MDSS sustainability, which should lead to consideration and discussion early in the planning process of MDSS feasibility and sustainability and the need to engage effectively field actors. Morocco’s political pressure implied a rapid MDSS launch and a large-scale implementation. The MOH is responsible for maintaining programs such as MDSS that meet the government’s public health priorities. Difficulty in monitoring MDSS performance and implementing a proper incentive mechanism for data collection and MDRs is still one of the system’s most serious sustainability challenges.
Putting recommendations into practice
The CIMD makes recommendations based on elaborate procedures. Unfortunately, the MDSS dissemination and response component appear to be underdeveloped. Stakeholders are confident in MDSS usefulness, yet the lack of implementing recommendations is discouraging. Region's health officials expressed doubts related to MDRs’ reliability and criticized the recommendation-making process. The current monitoring system focuses on the completeness of maternal death reporting. No indicators track progress in implementing recommendations. Although each region conducted an annual evaluation, the format differed, with some producing an overall summary report and others summarizing information in a slide presentation. Few health regions developed strategies for implementing recommendations and no systematic follow‐up of previous recommendations was organized.
In numerous countries, a large gap exists between data production and the ability to translate that data into usable information, formulate appropriate recommendations, and subsequently initiate relevant public health actions [44, 49]. In South Africa, each triennial report included recommendations. However, no specific indicators, targets, or timelines were defined, and no tracking system was used . Several studies specifically mention the challenge of not having funds to execute MDR recommendations .
Strengths and limitations of the study
Our study is the first qualitative study in Morocco, using an implementation framework to analyze factors influencing MDSS implementation and use. The CFIR was used to guide data coding, analysis, and reporting of findings. Each decision and rationale for selecting and reporting of CFIR constructs is documented. Since the stakeholders interviewed were from diverse levels (health region, district and health facility), the qualitative data was more comprehensive. A check by a research member contributed to reduce the bias in description and interpretation of IDIs. We also reviewed MDSS documentation from 2009 to 2017 in each region.
Nevertheless, our study has some limitations. First, to make the interview guide more understandable to the respondents, the CFIR was not used explicitly. Hence, some CFIR constructs may be missing information.
Second, we limited the IDIs to healthcare professionals who were RC-MDR members. External stakeholders and other healthcare professionals offering maternal healthcare services should translate results and recommendations into quality improvement actions. However, due to a lack of research resources and challenges in reaching out to these people, it was impossible to include them in our study. This is not a significant limitation, as various participants reported previous MDSS experiences while they were not RC-MDR members.
Another potential limitation is related to memory bias because we conducted the study between 2017 and 2018. However, the fact that the PI conducted the interviews herself, supervised and verified the transcripts, and took detailed field notes all contributed to reducing the memory bias.
Finally, the PI was the national MDSS coordinator between 2009 and 2013, then head of the national unit of maternal death surveillance when she conducted the research. This position had no influence on the selection of study participants. According to inclusion criteria, RCo compiled the list of names of potential participants, not the PI. Furthermore, as national coordinator, she mostly communicated with RCo, with whom she had a good relationship, based on trust and respect. This connection was extremely helpful in conducting the study. For most participants, the interviews were their first contact with the PI.
The MDSS is a complex process that utilizes a continuous cycle from identifying, reporting, and reviewing deaths to actions to improve healthcare services and avoid future deaths. This system has yielded three national reports on maternal deaths and only five regional reports. MDSS data and those reports are either unavailable or difficult to acquire. This shows that the appropriation of the MDSS, even 10 years after its launch is still a problem. Much effort is still required to produce high-quality routine analyses.
The MOH also planned to launch a similar neonatal deaths surveillance system in 2009. Despite repeated attempt, it has not been done because neonatal deaths are more frequent and more difficult to investigate. Our study is the only one to explore MDSS implementation factors in Morocco. Given our results, the MOH should carefully consider whether to adopt the MPDSR approach and combine perinatal and MDRs at the health facility level, or to first aim to improve the existing system.
Whatever approach is adopted, some important key issues must be addressed: MOH support, skilled and trained staff, horizontal and vertical communication related to MDSS goals and feedback and early involvement of subnational stakeholders in sustainability discussions.
Availability of data and materials
All relevant data are available within the paper and its Supporting Information files.
Consolidated criteria for Reporting Qualitative research
Consolidated Framework for Implementation Research
Confidential inquiries into maternal deaths
District MDSS coordinator
District Health director
Demographic and Health Survey
Healthcare facility network service
Low and middle-income countries
Maternal Death Surveillance System
Maternal death review(s)
Maternal (Perinatal) Death Surveillance and Response
Ministry of Health
National experts committee
Public health and epidemiological surveillance service
Regional MDSS coordinator
Regional health director
Women of reproductive age
World Health Organization. Time to respond: a report on the global implementation of maternal death surveillance and response (MDSR). Geneva: World Health Organization; 2016. https://apps.who.int/iris/handle/10665/249524.
Smith H, Ameh C, Roos N, Mathai M, van den Broek N. Implementing maternal death surveillance and response: a review of lessons from country case studies. BMC Pregnancy Childbirth. 2017;17:1–11. https://doi.org/10.1186/s12884-017-1405-6.
World Health Organization. Making every baby count: audit and review of stillbirths and neonatal deaths. Geneva: World Health Organization; 2016. https://apps.who.int/iris/handle/10665/249523.
World Health Organization. Maternal and perinatal death surveillance and response: materials to support implementation. Geneva: World Health Organization; 2021. https://apps.who.int/iris/handle/10665/348487.
Blaise P, Gruénais M, Nani S, Sahel A, De Brouwere V. Does clinical audit promote professional reflexivity? The experience of Casablanca public hospitals in Morocco with quality of care improvement. In: Proceedings of the 9th Toulon-Verona Conference Quality in Services Paisley, Scotland, 7–8 September 2006. http://www.strengtheninghealthsystems.be/doc/2/ref%202.25%20Does%20clinical%20audit%20promote%20professional%20reflexivity%20-%20the%20experience%20of%20Casablanca%20public%20hospital_PB.pdf.
Filippi V, Brugha R, Browne E, Gohou V, Bacci A, De Brouwere V, et al. Obstetric audit in resource-poor settings: lessons from a multi-country project auditing ‘near miss’ obstetrical emergencies. Health Policy Plan. 2004;19:57–66. https://doi.org/10.1093/heapol/czh007.
Muffler N, Trabelssi MEH, De Brouwere V. Scaling up clinical audits of obstetric cases in Morocco. Tropical Med Int Health. 2007;12:1248–57. https://doi.org/10.1111/j.1365-3156.2007.01911.x6.
Rousseva C, Kammath V, Tancred T, Smith H. Health workers’ views on audit in maternal and newborn healthcare in LMICs: a qualitative evidence synthesis. Tropical Med Int Health. 2020;25:525–39. https://doi.org/10.1111/tmi.13377.
Abouchadi S, Belghiti Alaoui A, Meski FZ, De Brouwere V. Implementing a maternal mortality surveillance system in Morocco – challenges and opportunities. Tropical Med Int Health. 2013;18(3):357–65. https://doi.org/10.1111/tmi.12053.
Abouchadi S, Belghiti Alaoui A, Meski FZ, Bezad R, De Brouwere V. Preventable maternal mortality in Morocco: the role of hospitals. Tropical Med Int Health. 2013;18(4):444–50. https://doi.org/10.1111/tmi.12065.
Abouchadi S. Le système de surveillance des décès maternels au Maroc: Pour optimiser la prise de décision au niveau des régions [Dissertation on internet]. Brussels: Université Libre de Bruxelles (ULB); 2022. http://hdl.handle.net/2013/ULB-DIPOT:oai:dipot.ulb.ac.be:2013/341974.
Abouchadi S, Zhang W-H, De Brouwere V. Underreporting of deaths in the maternal deaths surveillance system in one region of Morocco. PLoS ONE. 2018;13:e0188070. https://doi.org/10.1371/journal.pone.0188070.
Groseclose SL, Buckeridge DL. Public Health Surveillance Systems: Recent Advances in Their Use and Evaluation. Annu Rev Public Health. 2017;38:57–79. https://doi.org/10.1146/annurev-publhealth-031816-044348.
Kinney MV, Walugembe DR, Wanduru P, Waiswa P, George A. Maternal and perinatal death surveillance and response in low- and middle-income countries: a scoping review of implementation factors. Health Policy Plan. 2021;36:955. https://doi.org/10.1093/heapol/czab011.
Bradshaw C, Atkinson S, Doody O. Employing a Qualitative Description Approach in Health Care Research. Glob Qual Nurs Res. 2017;4:2333393617742282. https://doi.org/10.1177/2333393617742282.
Rendle KA, Abramson CM, Garrett SB, Halley MC, Dohan D. Beyond exploratory: a tailored framework for designing and assessing qualitative health research. BMJ Open. 2019;9(8):e030123. https://doi.org/10.1136/bmjopen-2019-030123.
Caelli K, Ray L, Mill J. “Clear as mud”: Toward greater clarity in generic qualitative research. Int J Qual Methods. 2006;2(2):1–23. https://doi.org/10.1177/160940690300200201.
Sullivan-Bolyai S, Bova C, Harper D. Developing and refining interventions in persons with health disparities: The use of qualitative description. Nurs Outlook. 2005;53(3):127–33. https://doi.org/10.1016/j.outlook.2005.03.005.
Lambert VA, Lambert CE. Qualitative descriptive research: An acceptable design. Pacific Rim Int J Nurs Res. 2012;16(4):255–6 (https://he02.tci-thaijo.org/index.php/PRIJNR/article/view/5805).
Sandelowski M. Focus on research methods: Whatever happened to qualitative description? Res Nurs Health. 2000;23(4):334–40. https://doi.org/10.1002/1098-240X(200008)23:4%3c334::AID-NUR9%3e3.0.CO;2-G.
Balane MA, Palafox B, Palileo-Villanueva LM, McKee M, Balabanova D. Enhancing the use of stakeholder analysis for policy implementation research: towards a novel framing and operationalised measures. BMJ Glob Health. 2020;5:e002661. https://doi.org/10.1136/bmjgh-2020-002661.
Boyce C, Neale P. Conducting in-depth interviews: a guide for designing and conducting in-depth interviews for evaluation input. Watertown: Pathfinder International. Pathfinder International Tool Series, Monitoring and Evaluation; 2006. https://www.pathfinder.org/publications/guide-designing-conducting-depth-interviews-evaluation-input/.
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101. https://doi.org/10.1191/1478088706qp063oa.
Pearse N. An Illustration of a Deductive Pattern Matching Procedure in Qualitative Leadership Research. EJBRM. 2019;17:3. https://doi.org/10.34190/jbrm.17.3.004.
Casula M, Rangarajan N, Shields P. The potential of working hypotheses for deductive exploratory research. Qual Quant. 2021;55(5):1703–25.
Damshmroder LJ. Clarity out of chaos: Use of theory in implementation research. Psychiatry Res. 2020;283:112461. https://doi.org/10.1016/j.psychres.2019.06.036.
Means AR, Kemp CG, Gwayi-Chore M-C, Gimbel S, Soi C, Sherr K, et al. Evaluating and optimizing the consolidated framework for implementation research (CFIR) for use in low- and middle-income countries: a systematic review. BMC Implementation Science. 2020;15:1–19. https://doi.org/10.1186/s13012-020-0977-0.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. BMC Implementation Science. 2009;4:1–15. https://doi.org/10.1186/1748-5908-4-50.
Keith RE, Crosson JC, O’Malley AS, Cromp D, Taylor EF. Using the Consolidated Framework for Implementation Research (CFIR) to produce actionable findings: a rapid-cycle evaluation approach to improving implementation. BMC Implementation Science. 2017;12:1–12. https://doi.org/10.1186/s13012-017-0550-7.
Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19:349–57. https://doi.org/10.1093/intqhc/mzm042.
Abebe B, Busza J, Hadush A, Usmael A, Zeleke AB, Sita S, et al. ‘We identify, discuss, act and promise to prevent similar deaths’: a qualitative study of Ethiopia’s Maternal Death Surveillance and Response system. BMJ Glob Health. 2017;2:e000199. https://doi.org/10.1136/bmjgh-2016-000199.
Kalter HD, Mohan P, Mishra A, Gaonkar N, Biswas AB, Balakrishnan S, et al. Maternal death inquiry and response in India - the impact of contextual factors on defining an optimal model to help meet critical maternal health policy objectives. Health Research Policy Systems. 2011;9:1–14. https://doi.org/10.1186/1478-4505-9-41.
Said A, Sirili N, Massawe S, Pembe AB, Hanson C, Malqvist M. Mismatched ambition, execution and outcomes: implementing maternal death surveillance and response system in Mtwara region. Tanzania BMJ Global Health. 2021;6:e005040. https://doi.org/10.1136/bmjgh-2021-005040.
Smith H, Ameh C, Godia P, Maua J, Bartilol K, Amoth P, et al. Implementing Maternal Death Surveillance and Response in Kenya: Incremental Progress and Lessons Learned. Global Health: Science and Practice. 2017;5:345. https://doi.org/10.9745/GHSP-D-17-00130.
Melberg A, Mirkuzie AH, Sisay TA, Sisay MM, Moland KM. ‘Maternal deaths should simply be 0’: politicization of maternal death reporting and review processes in Ethiopia. Health Policy Plan. 2019;34:492. https://doi.org/10.1093/heapol/czz075.
Moodley J, Pattinson RC, Fawcus S, Schoon MG, Moran N, Shweni PM. The confidential enquiry into maternal deaths in South Africa: a case study. Br J Obstet Gynaecol. 2014;121:53–60. https://doi.org/10.1111/1471-0528.12869.
Purandare C, Bhardwaj A, Malhotra M, Bhushan H, Shah PK. Every death count: Electronic tracking systems for maternal death review in India. Int J Gynecol Obstet. 2014;127:S35–9. https://doi.org/10.1016/j.ijgo.2014.09.003.
Kinney MV, Ajayi G, Graft-Johnson J de, Hill K, Khadka N, Om’Iniabohs A, et al. “It might be a statistic to me, but every death matters.”: An assessment of facility-level maternal and perinatal death surveillance and response systems in four sub-Saharan African countries. PLOS ONE. 2020;15:e0243722. https://doi.org/10.1371/journal.pone.0243722.
Jayakody H, Knight M. Implementation assessment in confidential enquiry programmes: A scoping review. Paediatr Perinat Epidemiol. 2020;34:399. https://doi.org/10.1111/ppe.12604.
Lewis G. The cultural environment behind successful maternal death and morbidity reviews. Br J Obstet Gynaecol. 2014;121:24–31. https://doi.org/10.1111/1471-0528.12801.
Tura AK, Fage SG, Ibrahim AM, Mohamed A, Ahmed R, Gure T, et al. Beyond No Blame: Practical Challenges of Conducting Maternal and Perinatal Death Reviews in Eastern Ethiopia. Global Health: Science and Practice. 2020;8:150. https://doi.org/10.9745/GHSP-D-19-00366.
Melberg A, Teklemariam L, Moland KM, Aasen HS, Sisay MM. Juridification of maternal deaths in Ethiopia: a study of the Maternal and Perinatal Death Surveillance and Response (MPDSR) system. Health Policy Plan. 2020;35:900. https://doi.org/10.1093/heapol/czaa043.
Scott H, Dairo A. Maternal Death Surveillance and Response in East and Southern Africa. J Obstet Gynaecol Can. 2015;37:915–21. https://doi.org/10.1016/S1701-2163(16)30030-5.
Cahyanti RD, Widyawati W, Hakimi M. “Sharp downward, blunt upward”: district maternal death audits’ challenges to formulate evidence-based recommendations in Indonesia - a qualitative study. BMC Pregnancy Childbirth. 2021;21:1–14. https://doi.org/10.1186/s12884-021-04212-7.
Parker J, Davies B. No Blame No Gain? From a No Blame Culture to a Responsibility Culture in Medicine. J Appl Philos. 2020;37:646. https://doi.org/10.1111/japp.12433.
Wachter RM, Pronovost PJ. Balancing “No Blame” with Accountability in Patient Safety. N Engl J Med. 2009;361:1401–6. https://doi.org/10.1056/NEJMsb0903885.
Wolvaardt E. Blame does not keep patients safe. Community Eye Health. 2019;32(106):36 PMID: 31649433; PMCID: PMC6802475.
Elmqvist KO, Rigaudy MT, Vink JP. Creating a no-blame culture through medical education: a UK perspective. J Multidiscip Healthc. 2016;9:345. https://doi.org/10.2147/JMDH.S111813.
Mutabazi M, Leshabari S. Maternal Death Surveillance and Response: Factors Affecting its implementation in Mtwara Region, Tanzania. JWHG. 2020;19(2):556007. https://doi.org/10.19080/JGWH.2020.18.556007.
We would like to express our sincere thanks to all RC-MDR members and the participants in this study for their willingness to share their experiences. We would like also to address special thanks to regional directors of the MOH and regional coordinators of MDSS for their valuable collaboration.
We thank Mr. Abdelatif Belmehdi, editor and translator, for his careful English editing of this paper, as well as Ms. Liza Santorini for her proofreading.
The study is part of an Individual PhD Scholarship Program of the Institute of Tropical Medicine of Antwerp (ITM) funded by the Belgian Development Cooperation (DGD) under the Framework Agreement (FA3-III). Support was also provided by the Moroccan MOH and from FP7-PEOPLE-2013-IRSES Marie Curie Actions project funded by the European Union, grant agreement no. 612216. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
Ethics approval and consent to participate
All methods were conducted in accordance with relevant guidelines and regulations. The Institutional Review Board of the Institute of Tropical Medicine of Antwerp, Belgium (No 1165/17) and the Ethics Committee for Biomedical Research of the Faculty of Medicine and Pharmacy of Rabat, Morocco (No 84/17) approved the study on ethical ground including the process for obtaining verbal consent to participate in research. Before beginning interviews, the PI informed the participant about the objective of the study, his/her participation on a voluntary basis, and that anonymity would be ensured. She asked participants for verbal consent before recording discussions. Two student midwives were recruited and paid as freelancers to transcribe the interviews at the PI’s office under her supervision. To preserve the study participants’ anonymity, we have chosen to refrain from giving more details on the study participants and the IDIs study settings. Each region has been given a code.
Consent for publication
The authors declare that they have no competing interests. The position of SA as the head of Morocco’s National Unit of Maternal Mortality Surveillance was not a hierarchical one. In that position, she was officially assigned the responsibility for monitoring MDSS activities at the national level. This role was extremely helpful in accessing the data and obtaining authorization to conduct interviews. We should mention that her thesis promoter (VDB) verified the whole data collection and analysis processes.
Interviews were conducted with regional and district-level actors whose names were drawn up by health-region officials. For most participants, the interviews were their initial contact with the PI. She gave no incentives to compensate for routine activities. Incentives are the responsibility of the managers in the health regions. If funds were available, it offered to cover the travel costs of personnel once surveillance and review activities were completed.
The position of SA does not alter her adherence to all BMC Public Health policies on sharing data and materials.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Assessment of health region level MDSS in Morocco. Figure S1. Score by health region for each level of implementation. Figure S2. Implementation progress on the schematic scoring scale.
In-depth interview guide.
Dimensions and constructs of the CFIR and their description.
CFIR Codebook adapted to MDSS.
Summary of the node tree developed for thematic analysis (NVivo Starter 11).
COREQ 32 item Checklist.
About this article
Cite this article
Abouchadi, S., Godin, I., Zhang, WH. et al. Eight-year experience of maternal death surveillance in Morocco: qualitative study of stakeholders’ views at a subnational level. BMC Public Health 22, 2111 (2022). https://doi.org/10.1186/s12889-022-14556-0