Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Public engagement in health technology assessment in Brazil: the case of the public consultation on National Clinical Guidelines for Care in Normal Birth

BMC Public Health volume 21, Article number: 1825 (2021) Cite this article

Abstract

Background

There is a growing body of literature that recognizes the importance of public engagement in health technology assessment. However, there is still uncertainty regarding how the results should be recorded, analyzed, and used by decision makers.

Objective

Synthesize the contributions of the Brazilian public (women, health professionals, managers, educational institutions, and companies) about the implementation of the National Clinical Guidelines for Care in Normal Birth from the public consultation carried out in Brazil.

Method

IRaMuTeQ software was used to organize and summarize the corpus based on three types of analysis: descriptive statistics; descending hierarchical classification; and specificities analysis. The public consultation was conducted in 2016 by the National Committee for Health Technology Incorporation (CONITEC) in the Brazilian public health system as part of the guideline development process.

Results

The corpus consisted of 303 texts, separated into 1233 text segments, 1081 of which were used, corresponding to retention of 87.67%. Five classes emerged from our analyses: mandatory presence of an obstetrician during labor and delivery in hospital settings; barriers and facilitators for guideline implementation; use of evidence—based practices by health professionals; progression of labor and delivery and women’s rights; and mobilization to promote the guideline For each class, the most frequent words and sentences with the highest chi-squared scores were presented. Barriers were associated with lack of financial resources, training and professional motivation, and facilitators with training to change the practices of health professionals. Obstetric nurses emerged as an alternative for supervising normal births as well as the mandatory presence of an obstetrician during childbirth in hospital settings.

Conclusion

Our findings summarize the contributions provided by the Brazilian public and shed some light on the barriers and facilitators of clinical guidelines for care in normal birth. These topics are not typically explored by quantitative studies. Including this information in the decision-making process would not only increase public engagement, but provide greater evidence for implementing the clinical guidelines nationwide.

Peer Review reports

Background

There is a growing body of literature that recognizes the importance of public engagement in health technology assessment (HTA) [1, 2], whereby communities are involved in decision-making as well as the planning, design, governance and delivery of services [3]. Public engagement, such as patient-citizens [4], for example, can improve accountability, transparency, and social inclusion in addition to providing a real-world understanding of the benefits and adverse effects of using technology to manage the process [5,6,7]. Public involvement, such as any member of society, can ensure the participation of individuals without personal interests, a broader representation of society and add more impartiality in decision-making in HTA [8]. It is important to highlight that the public and patient interests not always will be aligned [9], because patients could have personal interests related with the technology. However, there is still uncertainty regarding how the results should be recorded, analyzed, and used by decision makers [9,10,11,12,13,14], as well as about the impact of public, patient [12] or stakeholders involvement at HTA process [15]. According to a survey conducted by HTA agencies worldwide, barriers to involving the public in HTA processes include the potential tension between social and scientific considerations, lack of expertise in qualitative research, the mismatch between the demand for timely HTA and the time required to conduct high-quality public engagement, and the decision around who to engage in order to avoid potential biases or conflicts of interest [16].

Public consultation is frequently used by HTA agencies, whereby members of the public provide feedback on a specific technology under consideration [16]. Recently, a step-by-step approach was proposed to summarize public contributions in a systematic, transparent, reproductive, objective, and timely manner [17]. This method combines a case study based on grounded theory with the use of IRaMuTeQ software. IRaMuTeQ is used to organize, code and group words by similarity. Pearson’s chi-squared test was used at a 5% significance level to assess whether the frequency of a word was statistically associated with another [18]. It is important to apply this approach to technologies other than drug coverage decisions [17], such as clinical practice guidelines.

The Ministry of Health proposed new recommendations for care during normal birth, aimed at promoting improvements in procedures and outcomes and standardizing the most common practices in assisting normal birth. Based on the best available evidence and other international guidelines for care during normal childbirth, and using the ADAPTE methodology [19], the Brazilian National Clinical Guidelines for Care in Normal Birth reflected the consensus reached by several technical departments of the Ministry of Health, and a group of experts, professional associations (physicians and nurses), and social movements [20]. The guidelines were submitted to the National Committee for Health Technology Incorporation (CONITEC) in the public health system in December 2015. A favorable preliminary report was made available on the CONITEC website for public consultation from January to February 2016, in order to gather contributions from the public before the final decision [21].

The aim of this study is to synthesize the contributions of the Brazilian public (women, health professionals, managers, educational institutions, and companies) who participated in the consultation regarding the Brazilian National Clinical Guidelines for Care in Normal Birth, carried out by CONITEC in 2016. As proposed by Carvalho et al., 2019 [17], we used four questions that help us to summarize our main findings. The first question is about whether public categories shared some point of views on the guidelines under consultation. The second question seeks to identify whether there is public support to implement or not the guidelines under consideration. The third question refers to the main arguments (against or for) used by the public to support a decision of implementing or not the guidelines under consideration. The fourth question identifies the main issues raised by the members who participated in the consultation, stratifying by public categories. The summary provided could help both CONITEC and other HTA agencies or governments worldwide that aim to implement clinical guidelines for care in normal birth.

Method

Study design

A qualitative approach was used [22] to identify and synthesize the main contributions of the public consultation on implementing Brazilian National Clinical Guidelines for Care in Normal Birth. An exploratory case study was carried out to qualify the composition of the corpus, which contained all the contributions from the public consultation studied. IRaMuTeQ software was used for data processing, coding, separating and organizing information [23]. The program allows users to access text segments quickly and provides a transparent, systematic and reproducible system for processing qualitative data. IRaMuTeQ is free software that uses Python programming language and R software to statistically analyze a group of different texts combined into a corpus [24].

Based on the methodology proposed by Carvalho et al., 2019 [17], content analysis was performed using IRaMuTeQ software for data mining and to organize the corpus based on three types of analyses: descriptive statistics, descending hierarchical classification (DHC); and specificities analysis [25]. Descriptive statistics identify aspects such as the number of words, average frequency and number of hapaxes (words that only occur once) and the level of retention (percentage of text segments retained in DHC) [24, 25]. These indicators are good candidates for a measure of reproducibility of the results [17]. Descending hierarchical classification (DHC) categorizes text segments as a function of their respective vocabularies and separates them based on the frequency with which words and classes occur as well as their chi-squared scores [17, 24, 25]. Finally, specificities analysis allows text from databases to be associated with variables of interest, which in our case are the key concepts related to HTA [17, 24, 25]. These last two analyses use the chi-squared test (χ2) to verify whether there is correlation between any variable and words. The score of χ2 indicates how strong is the association between words and classes [21]. The use of chi-squared test is a default configuration on IRaMuTeQ and we decided for not change it [26].

Data processing and analysis

Data were analyzed in four stages: First, the corpus was skim read to determine whether an alternative form of prior analysis was necessary, such as grouping contributions according to the discourse categories. We opted to work with the classification that contributors themselves declared on the public consultation form, which included nine categories: i) company; family member, friend or caregiver; iii) patient groups, associations and organizations; iv) educational institutions; v) stakeholders; vi) others; vii) patients; viii) health professionals; and ix) medical societies.

In the second stage, the corpus was prepared according to the specificities of IRaMuTeQ software, including correcting typing and punctuation mistakes, standardizing acronyms and combining compound words by adding ‘underscores’ (for example medical_obstetrician).

In the third stage, three types of analysis were performed using the IRaMuTeQ program: descriptive statistics, descending hierarchical classification (DHC) and specificities analysis (specificities and factorial correspondence analysis – FCA). For specificities analysis, 15 words related to the key concepts of HTA were selected. These were defined based on the list of the most frequently cited words in the public consultation.

In the final stage, the results were systematized and interpreted. Each class was interpreted based on the most frequent words generated using the chi-squared test. The synthesis of our main results followed the structure proposed by Carvalho et al., 2019 [17].

Data set

The data analyzed were from the public consultation [21] about implementing National Clinical Guidelines for Care in Normal Birth [27], conducted in 2016 on the website of the National Committee for Health Technology Incorporation (CONITEC). There were a total of 396 contributions from different Brazilian states. Members of the public were asked to indicate what information they would change or include in the text; describe what would hinder implementation of the guideline, according to their own reality; explain what would facilitate its implementation, also based on their own reality; and provide additional comments on any other aspects. The fields ‘how have you contributed’ and ‘what do you think of the proposed protocol or guideline’ were used as variables in the analysis.

According to the recommendation report made available after the public consultation [27], contributions were distributed as follows: 66 from women; 24 from family members, friends or caregivers; 233 from health professionals; 63 from stakeholders; and 10 from legal entities, including companies [2], educational institutions [1], medical societies [3], patient groups/associations/organizations [2] and others [2]. Based on CONITEC’s report [28], most contributions (84%) were from women and 79% of contributors considered the guidelines good or very good, 7% fair and 14% inadequate of highly inadequate.

Researchers in the present study adopted non-participant observation, since contributions to the public consultation were collected by CONITEC. Despite the large volume of contributions, 93 were excluded from analysis because the text consisted solely of a “yes” or “no” answer. This was done to improve the quality of the corpus and the analysis itself, since these words are not related to other text segments.

Ethical aspects

We used secondary data provided by CONITEC, which are publicly available on the internet. On this basis, there is no need for ethical approval according to the Resolution 510/16 of the Brazilian National Health Council [29].

Results

Characteristics of the corpus – descriptive statistics

We identified 303 texts, divided into 1233 text segments (TS), of which 1081 were used, corresponding to 87.67% TS retention (Table 1).

Table 1 Characterization of the corpus
Full size table

Due to the heterogeneity of text size, a TS size of 30 occurrences was adopted to guarantee greater TS retention. In order to ensure that DHC did not provide a partial classification, minimal retention between 70 and 75% was adopted [26, 30]. There were a total of 33,185 word occurrences, with 4248 different word forms and 1.272 words (3.72% of the total occurrences) that occurred only once.

Descending hierarchical classification

The content analyzed was classified by IRaMuTeQ software into five word classes (Fig. 1): class 1, with 389 TS (35.99%), class 2, with 339 TS (31.36%), class 3, with 135 TS (12.49%), class 4, with 159 TS (14.71%) and class 5, containing 59 TS (5.46%). The five classes were divided into three branches with four sub-branches: subcorpus A (class 4), subcorpus B (class 1), subcorpus C (classes 2 and 3), and subcorpus D (class 5). The main topic (subject) of each word class was identified by reading the corpus and extracting the most significant excerpts for each class (Fig. 1).

Fig. 1
figure 1

Main classes and subclasses resulting from DHC of the corpus. LEGEND: Classe 1: class 1; Classe 2: class 2; Classe 3: class 3; Classe 4: class 4 Source: compiled by the authors based on an analysis performed in IRaMuTeQ software

Full size image

Based on analyses performed in IRaMuTeQ software, a table was compiled listing the main words, sentences, type of contribution and opinion on the guidelines for each word class (Table 2), generated using the chi-squared test (χ2). To extract the excerpts showed in Table 2, we first selected the typical text segment from each class. After that, we opted for displaying 50 TS and selected the excerpts with higher absolute score. Absolute score is provided by the IRaMuTeQ, considering the sum of the χ2 values from all words within a class.

Table 2 Main words, type of contribution, opinion and excerpts per class - classes 1 to 5
Full size table

Factorial correspondence analysis

Figure 2 shows the distribution of the 15 words related to health technology assessment (HTA) with the contributor categories provided in the public consultation. The words were selected in accordance with the HTA terms that emerged in the consultation.

Fig. 2
figure 2

Distribution of key concepts related to health technology assessment by contributor category, Brazil, 2016. Source: Elaborated by the authors. LEGEND: 1 - company; 2 - family member, friend or caregiver; 3 - patient association; 4 educational institution; 5- stakeholders; 6 - other; 7 - patient; 8 - health professional; 9 - medical society.. Notes: From top to bottom – study, impact, quality of life, efficiency, efficacy, incorporation, technology, expenses, side effects, cost, benefit, safety, safe, risk, scientific evidence

Full size image

The word “study” obtained the highest score under the “company” category for type of contributor (category 1), that is, the χ 2 calculation by IRaMuTeQ software revealed a strong statistical association between this contributor category and the word “study”, which refers to research on aspects related to childbirth, such as the benefits of giving birth at home, transfer rates to hospitals and the best position for childbirth. The words with the lowest score in this category were “scientific evidence”, “risk” and “benefit”.

The categories that showed the greatest use of key concepts related to HTA were health professionals (category 8), followed by patients (category 7) and stakeholders (category 5). The highest scoring word in the patient category was “safety”, followed by “benefit” and “efficiency”. The first two words are associated with information and guidelines on safety during labor and delivery and the benefits of natural childbirth, the presence of a doula during childbirth and the possible interventions that may occur. According to the contributions, these interventions should be properly explained to the mother to ensure she can make an informed decision about the care she wants to receive. In the public consultation, “efficiency” was related to training health professionals, particularly doctors, based on scientific evidence and communicating with patients and family members to avoid conduct based preconceptions and expert opinions.

The stakeholder category contained the words “expense”, “benefit” and “risk”. Expense and “cost” are in the same semantic field and associated with the idea that implementing the guidelines is an attempt to lower costs at health units. The word “risk” was related to the risks associated with childbirth and obtained the highest score in the medical society category.

Under health professionals (category 8), the words “side effects”, “efficacy” and “technology” were only associated with this category. “Efficacy” was related to inconclusive evidence about the use of prophylaxis for infants with chlamydial conjunctivitis, whereas “technology” and “side effects” were linked to interventions during labor. The side effects of alcohol consumption by pregnant women were also cited, as was the lack of a specific care protocol for these women.

The words with the highest score in the family member, friend or caregiver category (category 2) were “benefit” and “scientific evidence”, with the first pertaining to evidence from research that should be used to support clinical decision making.

The patient association category (category 3) showed no strong statistical association with any of the HTS words selected, whereas educational institution (category 4) was strongly associated with the words “cost” and “safe”, related to the costs of childbirth and health professionals, and labor, respectively.

According to contributors, the decision to undergo a home birth or childbirth without a doctor present is used as an alternative to lower the costs of healthcare services. The “other” category exhibited a greater statistical association with the word “incorporation”, linked to implementing the guidelines in clinical practice.

Summary of the public consultation

A difference was observed between health professionals for the question “is there convergence/divergence of opinion between different discourse categories about the guideline under consultation?”. Some professionals argued that childbirth should be the sole domain of doctors and performed only in hospital settings, while others advocated the inclusion of obstetric nurses and midwives in normal births. In general, although the remaining categories contained different ideas, these did not characterize differences of opinion. While some stakeholders argued in favor of mobilizing to promote the guidelines, others advocated for allowing labor to progress naturally and respecting women’s right to choose. Despite being different topics, they are not opposing ideas, as occurred for health professionals.

With regard to the question “is there public support for including/excluding the guideline under consultation?”, in general there were more arguments in favor of implementing the guidelines. A portion of contributors from both categories (health professionals and family members, friends or caregivers) were more emphatic in opposing the guidelines because they felt the presence of a doctor was vital during childbirth to address any complications that might arise.

Some of the barriers cited under the question “what are the main pros/cons raised by members of the public about including/excluding the guideline under consultation?” were the resistance of health professionals, lack of teams willing to perform home births and shortage of human resources and materials. The main arguments in favor of the guidelines are related to the inclusion of obstetric nurses and midwives, respecting women’s right to choose and mobilization to promote the guidelines. Some contributors viewed the guidelines as an instrument to support professional practices and strengthen sexual and reproductive rights within healthcare policies. Facilitators mentioned were determination, training programs and professional awareness.

Finally, for the question “what are the main issues related to the opinion and type of contributor who participated in the public consultation”, only a small portion of health professionals, patient associations, stakeholders and patients viewed the guidelines as negative, with opinions of “highly inadequate”, “fair” and “inadequate” predominating. The opinion of the remaining contributors ranged from “fair” to “very good” (Fig. 3).

Fig. 3
figure 3

Guideline assessment by type of contributor. Source: compiled by the authors based on 303 contributions included in analyse

Full size image

The main arguments in favor of the guidelines were related to introducing obstetric nurses and midwives to maternity wards and creating a law or ordinances to regulate the guidelines. The inclusion of obstetric nurses, offering fair salaries and a mandatory requirement for all nurses caring for pregnant and laboring women to hold a specialty in obstetrics were highlighted by health professionals and medical societies. Stakeholders and companies emphasized that the inclusion of obstetric nurses and midwives could contribute to respecting and guaranteeing women’s right to choose. Contributors who identified as stakeholders and others addressed issues related to the support of research groups and health committees in mobilizing to promote the guidelines.

Discussion

In global terms, a recent systematic review found that clinical guidelines for uncomplicated birth are generally diverse, exhibit poor methodological quality and low agreement in terms of recommendations [31]. The lack of international consensus may explain the different opinions among participants in the public consultation. A key point of the consultation was the conflict between health professionals about the possibility of obstetric nurses or midwives supervising the birth. The International Confederation of Midwives (ICM) defines a “midwife” as someone with the necessary qualifications to be registered and/or legally licensed to practice midwifery, in accordance with the ICM Global Standards for Midwifery Education [32]. In the United Kingdom, most care during childbirth is provided by midwives working in partnership with doctors in vaginal births and cesarean sections to support the mother and baby [33].

Issues related to a lack of understanding regarding the rights of pregnant women were also mentioned. Women can experience different forms of obstetric violence and its causes are multifactorial, requiring the combined efforts of different health professionals to address the problem [34]. Creating an environment of care is one of the most challenging aspects and suggests that midwifery acts in defense of women within the healthcare system [35]. For example, some women associate overcoming their fears and doubts and building confidence in their ability to give birth without pharmacological pain relief to the relationship of trust with midwives [36]. In Brazil, the Ministry of Health launched the Stork Network in 2011 at the regional level to provide a care network aimed at guaranteeing women the right to reproductive planning and humanized care through pregnancy, childbirth and the postpartum period [37]. The Stork Network consists of four components that involve a series of healthcare initiatives: prenatal care; childbirth; postpartum and comprehensive child care; and logistics, sanitary transport and regulation. Its implementation in Brazilian states and municipalities is based on epidemiological criteria such as population density and infant and maternal mortality rates [37].

The key HTA concepts were little used by participants in the public consultation. Of the nine contributor categories, only three used more than four HTA-related words in their contributions, suggesting that both these terms and the importance of evidence-based decision making need to be better disseminated among public consultation participants. The need for HTA education and training strategies for patients and the public was also identified in research performed by HTA agencies that evaluated their public and patient engagement processes [38]. Additionally, it has been reported that patient organizations generally do not receive training from HTA agencies [39].

Strengths and limitations of the study

Our study contributed by applying the method proposed by Carvalho et al. 2019 [17] to a clinical practice guideline, and proved to be effective with satisfactory results. This method was previously applied to a public consultation regarding the incorporation of a drug in the public health system. This method combined with IRaMuTeQ software was also faster than conventional content analysis, which is beneficial for HTA agencies, given the short time period they have available to conduct this type of assessment. Descriptive statistics were also used as a likely indicator of reproducibility should other researchers be interested in replicating the analysis. Reproducibility is also a vital attribute for health technology assessment. In addition, the present study progressed in such a manner that the opinions, experiences and interests of contributors to the public consultation on National Clinical Guidelines for Care in Normal Birth were systematized.

However, the study also exhibited some limitations. First, 12.33% of text segments were not used in the DHC analysis conducted in IRaMuTeQ software, obtaining retention of 87.67%. Although this loss may have influenced the results to some degree, retention was still the minimum 75% recommended by the IRaMuTeQ manual [26]. Second, the contributor categories used were those defined by CONITEC for the public consultation and some of these may have overlapped for some contributors, such as “pregnant woman” and “health professional”. However, we opted to use contributors’ self-reported categories since they are better equipped to know which categories best represent them. Third, limitations regarding the public engagement process in Brazil were also identified. Based on the dissemination strategies of the public consultation used by CONITEC (website and email lists), the participants probably had some interest in the subject or knew about the processes adopted by CONITEC. It is likely that participants attracted by the public consultation may not represent the Brazilian society, both because of the small number of contributions received and because of the limited forms of dissemination adopted by CONITEC. People who do not have access to the internet, for example, are not able to participate in the public consultation. Although these limitations are not caused by the method used, we consider that the analysis is valid to highlight aspects about this group of people involved with CONITEC process. It is important to mention that the Brazilian engagement process in HTA needs to be improved. Finally, we used a public consultation carried out in 2016, after CONITEC had made the final decision on National Clinical Guidelines for Care in Normal Birth. Although this study lost its potential to contribute to decision making at the time, it is still relevant in terms of highlighting aspects of the guidelines that should be monitored following their implementation. The information summarized here can also be used in other contexts or countries that initiate discussions regarding normal birth.

Implications for public policies

Identifying and addressing the barriers to implementation is critical to the success of a healthcare policy or program [40]. The National Clinical Guidelines for Care in Normal Birth are a valuable tool for integrating scientific knowledge into the practices of health services and care providers in order to improve health outcomes and people’s lives. However, their implementation requires knowledge of the importance of qualitative evidence in this process [41,42,43]. Qualitative evidence is increasingly valued worldwide, including global academic efforts to make it more systematic, transparent and reliable [44]. The present study contributes to this field by providing a practical application of a tool to support the analysis and synthesis of colloquial qualitative evidence, as well as a set of important elements for decision makers to consider when planning the implementation of guidelines for normal birth in Brazil and similar contexts.

Conclusions

This study systematized opinions, experiences, and interests of contributors from the public consultation on the National Clinical Guidelines for Care in Normal Birth in Brazil. Based on that, we could achieve our objective of identifying barriers and facilitators related to the clinical guidelines’ implementation. The inclusion of obstetric nurses and midwives in normal births was a point of divergence between healthcare workers. Few healthcare workers argued that childbirth should be performed by doctors only at hospital setting. We identified more arguments in favor of implementing the guidelines than in opposing, and only a small portion of contributors viewed the guidelines as negative. Some barriers identified were related to human resources and healthcare worker’s resistance to perform home births. Determination, training programs and healthcare worker awareness, as for example about the need for change, were mentioned as facilitators.

The low use of key HTA concepts by the public consultation participants was identified, what suggests a lack in knowledge or in training from the participants. To qualify the HTA social participation, the training and dissemination of information about HTA process and evidence-based decision making to public consultation participants must be improved by HTA agencies.

Finally, our study explored topics that are not typically explored by quantitative studies. This kind of information can help to improve the HTA process, increase public engagement and provide greater evidence for implementing the clinical guidelines in Brazil and other counties with similar context.

Availability of data and materials

The datasets analysed during the current study are available in the CONITEC repository, [http://conitec.gov.br/images/Consultas/Contribuicoes/2016/CP_CONITEC_01_2016_PCDT_Diretriz_Nacional_de_Assistência_ao_Parto_Normal.pdf].

Abbreviations

CONITEC:

National Committee for Health Technology Incorporation

DHC:

Descending hierarchical classification

FCA:

Factorial correspondence analysis

HTA:

Health Technology Assessment

IRaMuTeQ:

Interface de R pour les Analyses Multidimensionnelles de Textes et de Questionnaires)

SUS:

National Health System

TS:

Text segments

χ2:

Chi-squared

References

  1. Gagnon M. Introducing patients ’ and the public ’ s perspectives to health technology assessment : a systematic review of international experiences. INTL J Technol Assess Heal CARE. 2011;27(1):31–42. https://doi.org/10.1017/S0266462310001315.

    Article  Google Scholar 

  2. Castro R, Elias FTS. Envolvimento dos usuários de sistemas de saúde na avaliação de tecnologias em saúde (ATS): Uma revisão narrativa de estratégias internacionais. Interface Commun Heal Educ. 2018;22(64):97–108.

    Google Scholar 

  3. O’Mara-Eves A, Brunton G, McDaid D, Oliver S, Kavanagh J, Jamal F, et al. Community engagement to reduce inequalities in health: a systematic review, meta-analysis and economic analysis. Public Heal Res [Internet]. 2013;1(4):1–526. Available at: https://www.journalslibrary.nihr.ac.uk/phr/phr01040/. https://doi.org/10.3310/phr01040.

    Article  Google Scholar 

  4. Alami H, Gagnon M-P, Fortin J-P. Involving Citizen-Patients in the Development of Telehealth Services: Qualitative Study of Experts’ and Citizen-Patients’ Perspectives. J Particip Med [Internet]. 2018;10(4):e10665 Available at: http://jopm.jmir.org/2018/4/e10665/.

    Article  Google Scholar 

  5. Facey K, Boivin A, Gracia J, Hansen HP, Lo Scalzo A, Mossman J, et al. Patients’ perspectives in health technology assessment: a route to robust evidence and fair deliberation. Int J Technol Assess Health Care. 2010;26(3):334–40. https://doi.org/10.1017/S0266462310000395.

    Article  PubMed  Google Scholar 

  6. Oxman AD, Lewin S, Lavis JN, Fretheim A. SUPPORT tools for evidence-informed health policymaking (STP) 15: engaging the public in evidence-informed policymaking. Chinese J Evidence-Based Med. 2010;10(5):520–6.

    Google Scholar 

  7. Hailey D. Involvement of patients in health technology assessment: further perspectives for informing decision-makers. Hosp Pract Res [Internet]. 2017;2(3):58–62. Available at: http://jhpr.ir/article_49622.html. https://doi.org/10.15171/hpr.2017.16.

    Article  Google Scholar 

  8. Street J, Stafinski T, Lopes E, Menon D. Defining the role of the public in Health Technology Assessment (HTA) and HTA-informed decision-making processes. Int J Technol Assess Health Care [Internet]. 2020/03/10. 2020;36(2):87–95. Available at: https://www.cambridge.org/core/article/defining-the-role-of-the-public-in-health-technology-assessment-hta-and-htainformed-decisionmaking-processes/292CF251A57392C24EFF3E186E97A9E1

  9. Menon D, Stafinski T. Role of patient and public participation in health technology assessment and coverage decisions. Expert rev Pharmacoecon outcomes res [internet]. 1 de fevereiro de 2011;11(1):75–89. Available at: https://doi.org/10.1586/erp.10.82, 2011.

  10. Martin GP. Policy and practice: Whose health , whose care , whose say? Some comments on public involvement in new NHS commissioning arrangements. Crit Public Health 2009;19(1):123–132, Whose health, whose care, whose say? Some comments on public involvement in new NHS commissioning arrangements, https://doi.org/10.1080/09581590802385672.

  11. Aslin HJ (Heather J., Brown VA, Murray-Darling Basin Commission. Towards whole of community engagement : a practical toolkit. Murray-Darling Basin Commission; 2004. 146 p.

  12. Boothe K. “Getting to the table”: changing ideas about public and patient involvement in Canadian drug assessment. J health Polit policy law [internet]. 1 de agosto de 2019;44(4):631–63. Available at: https://doi.org/10.1215/03616878-7530825, 2019.

  13. Wortley S, Tong A, Howard K. Community views and perspectives on public engagement in health technology assessment decision making. Aust heal rev [internet]. 2017;41(1):68–74. Available at: https://doi.org/10.1071/AH15221, 2017.

  14. Haji Ali Afzali H, Street J, Merlin T, Karnon J. The representation of public values in health technology assessment to inform funding decisions: the case of Australia’s national funding bodies. Int J Technol Assess Health Care [Internet]. 2021/01/18. 2021;37(1):e22. Available at: https://www.cambridge.org/core/article/representation-of-public-values-in-health-technology-assessment-to-inform-funding-decisions-the-case-of-australias-national-funding-bodies/6697D7BC9DC865D98AE39C1F8FEDFB3B

  15. Luhnen M, Ormstad SS, Willemsen A, Schreuder-Morel C, Helmink C, Ettinger S, et al. Developing a quality management system for the European Network for Health Technology Assessment (EUnetHTA): toward European HTA collaboration. Int J Technol Assess Health Care [Internet]. 2021/04/27. 2021;37(1):e59. Available at: https://www.cambridge.org/core/article/developing-a-quality-management-system-for-the-european-network-for-health-technology-assessment-eunethta-toward-european-hta-collaboration/07C5596698A1A72212370F52C3288484

  16. Whitty JA. An international survey of the public engagement practices of health technology assessment organizations. Value Heal [Internet]. 2013;16(1):155–63 Available at: https://doi.org/10.1016/j.jval.2012.09.011.

    Article  Google Scholar 

  17. Carvalho VK da S, Sousa MSA De, Barreto JOM, Silva EN da. Public engagement in health technology assessment in Brazil : the case of the Trastuzumab public consultation. BMC Health Serv Res 2019;19(762):1–11, DOI: https://doi.org/10.1186/s12913-019-4555-6.

  18. de Oliveira JF, Pescarini JM, de Souza RM, de Araujo AB, Pessanha Henriques CM, Gouveia FC, et al. The global scientific research response to the public health emergency of Zika virus infection. PLoS One [Internet]. 2020;15(3):1–16 Available at: https://doi.org/10.1371/journal.pone.0229790.

    Google Scholar 

  19. ADAPTE Collaboration. Guideline adaptation : a resource toolkit [internet]. Update. 2009; Available at: http://www.g-i-n.net/document-store/working-groups-documents/adaptation/adapte-resource-toolkit-guideline-adaptation-2-0.pdf.

  20. Grilo Diniz CS, Rattner D, Lucas d’Oliveira AFP, de Aguiar JM, Niy DY. Disrespect and abuse in childbirth in Brazil: social activism, public policies and providers’ training. Reprod health matters [internet]. 2018;26(53):19–35. Available at: https://doi.org/10.1080/09688080.2018.1502019, 2018.

  21. Comissão Nacional de Incorporação de Tecnologias no SUS (Conitec). Contribuições da Consulta Pública - PCDT - Diretriz Nacional de Assistência ao Parto Normal - CONITEC. Consult Pública No 1, 8 Janeiro 2016 [Internet]. 2016;264. Available at: http://conitec.gov.br/images/Consultas/Contribuicoes/2016/CP_CONITEC_01_2016_PCDT_Diretriz_Nacional_de_Assistência_ao_Parto_Normal.pdf

  22. Denzin NK, Lincoln YS. The SAGE handbook of qualitative research. 3rd ed. Denzin NK: Lincoln YS, organizadores. SAGE Publications; 2005. 968 p.

    Google Scholar 

  23. de MAR S, Wall ML, de MC TAC, IMV L, Peres AM. O uso do software IRAMUTEQ na análise de dados em pesquisas qualitativas. Rev da Esc Enferm da USP. 4 de outubro de. 2018;52:e03353.

    Google Scholar 

  24. Camargo B V., Justo AM. IRAMUTEQ: Um software gratuito para análise de dados textuais. Temas em Psicol [Internet]. 2013;21(2):513–8. Available at: http://pepsic.bvsalud.org/pdf/tp/v21n2/v21n2a16.pdf

  25. Justo AM, Camargo BV. Estudos qualitativos e o uso de softwares para análises lexicais. In: Novikoff, C; Santos, S R M & Mithidieri, O B (Orgs) Caderno de artigos: X SIAT & II Serpro (2014: Duque de Caxias, RJ) (p. Duque de Caxias: Universidade do Grande Rio “Professor José de Souza Herdy” - UNIGRANRIO; 2014. p. 37–54.

  26. Camargo BV, Justo AM. Tutorial para uso do software (Interface de R pour les Analyses Multidimensionnelles de Textes et de Questionnaires) [Internet]. Florianópolis; 2018. Available at: www.laccos.com.br

    Google Scholar 

  27. Brasil. Ministério da Saúde. Secretaria de Ciência Tecnologia e Insumos Estratégicos. Departamento de Gestão e Incorporação de Tecnologias em Saúde. Diretrizes nacionais de assistência ao parto normal: versão resumida [Internet]. Ministério da Saúde, Secretaria de Ciência, Tecnologia e Insumos Estratégicos, Departamento de Gestão e Incorporação de Tecnologias em Saúde. Brasília; 2017. Available at: https://doi.org/10.1016/B978-84-9113-358-2/00012-9

  28. CONITEC. Diretrizes Nacionais de Assitência ao Parto Normal - Relatório de recomendação no 211. Brasília; 2017.

    Google Scholar 

  29. CNS CN de S, Brasil. Ministério da Saúde. CONSELHO NACIONAL DE SAÚDE. RESOLUÇÃO No 510, DE 07 DE ABRIL DE 2016. RESOLUÇÃO No 510, DE 07 DE ABRIL DE 2016 Brazil; 2016 p. 10.

  30. Salvador PTC de O, Gomes AT de L, Rodrigues CCFM, Chiavone FBT, Alves KYA, Bezerril MDS, et al. Uso do software iramuteq nas pesquisas brasileiras da área da saúde: uma scoping review. Rev Bras em Promoção da Saúde. 29 de novembro de 2018;31.

  31. Zhao Y, Lu H, Zang Y, Li X. A systematic review of clinical practice guidelines on uncomplicated birth. BJOG An Int J Obstet Gynaecol. 2020;127(7):789–97. https://doi.org/10.1111/1471-0528.16073.

    Article  CAS  Google Scholar 

  32. ICM. ICM Definitions - definition of midwife [Internet]. 2018 [citado 26 de janeiro de 2020]. Available at: https://www.internationalmidwives.org/our-work/policy-and-practice/icm-definitions.html

  33. Hunter B. Implementing a national policy initiative to support normal birth: lessons from the all wales clinical pathway for normal labour. J Midwifery Women’s Heal. maio de. 2010;55(3):226–33. https://doi.org/10.1016/j.jmwh.2009.12.014.

    Article  Google Scholar 

  34. Martins A de C, Barros GM. Will you give birth in pain? Integrative review of obstetric violence in Brazilian public units. Rev Dor. 2016;17.

  35. Kennedy HP, Shannon MT, Chuahorm U, Kravetz MK. The landscape of caring for women: a narrative study of midwifery practice. J Midwifery Women’s Heal janeiro de. 2004;49(1):14–23.

    Google Scholar 

  36. Leap N, Sandall J, Buckland S, Huber U. Journey to confidence: Women’s experiences of pain in labour and relational continuity of care. J Midwifery Women’s Heal maio de. 2010;55(3):234–42. https://doi.org/10.1016/j.jmwh.2010.02.001.

    Article  Google Scholar 

  37. Brasil. Ministério da Saúde. Gabinete do Ministro. PORTARIA No 1.459, DE 24 DE JUNHO DE 2011. Institui, no âmbito do Sistema Único de Saúde - SUS - a Rede Cegonha. No 1.459, 2011.

  38. Weeks L, Polisena J, Scott AM, Holtorf AP, Staniszewska S, Facey K. Evaluation of patient and public involvement initiatives in health technology assessment: a survey of international agencies. Int J Technol Assess Health Care. 2017;33(6):715–23. https://doi.org/10.1017/S0266462317000976.

    Article  PubMed  Google Scholar 

  39. Scott AM, Wale JL. Patient advocate perspectives on involvement in HTA: an international snapshot. Res Involv Engagem [Internet]. 2017;3(2):17. Available at: http://researchinvolvement.biomedcentral.com/articles/10.1186/s40900-016-0052-9

  40. Panisset U, Koehlmoos TP, Alkhatib AH, Pantoja T, Singh P, Kengey-Kayondo J, et al. Implementation research evidence uptake and use for policy-making. Heal Res Policy Syst. 2012;10(1):1–7. https://doi.org/10.1186/1478-4505-10-20.

    Article  Google Scholar 

  41. Carroll C. Qualitative evidence synthesis to improve implementation of clinical guidelines. BMJ [Internet]. 2017;356:4–7 Available at: https://doi.org/10.1136/bmj.j80.

    Google Scholar 

  42. Glenton C, Lewin S, Lawrie TA, Barreix M, Downe S, Finlayson KW, et al. Qualitative evidence synthesis (QES) for guidelines: paper 3-using qualitative evidence syntheses to develop implementation considerations and inform implementation processes. Heal Res Policy Syst. 2019;17(1):1–14. https://doi.org/10.1186/s12961-019-0450-1.

    Article  Google Scholar 

  43. Langlois EV, Tunçalp Ö, Norris SL, Ghaffar A. Qualitative evidence to improve guidelines and health decision-making. Bull World Health Organ. 2018;96(2):79–79A. https://doi.org/10.2471/BLT.17.206540.

    Article  PubMed  PubMed Central  Google Scholar 

  44. The Cochrane Collaboration. Cochrane Handbook for Systematic Reviews of Interventions. 2o ed. Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., organizadores. Cochrane Handbook for Systematic Reviews of Interventions. 2019. 703 p.

Download references

Acknowledgements

We would like to thank the Pan American Health Organization (PAHO/WHO), the Special Programme for Research and Training in Tropical Diseases (TDR), and the Alliance for Health Policy and Systems Research (AHPSR) for the grant to the project entitled “Barreras y estrategias para la implementación de la Guía Nacional de Parto Normal en Brasil. Proyecto Parto Cuidadoso: monitoreo y adopción de buenas prácticas para el cuidado en el parto [Call for proposals ‘Embedding Research for the Sustainable Development Goals (ER-SDG)’]”. We also would like to thank the Secretary of Science, Technology Innovation and Strategic Inputs of the Ministry of Health of Brazil.

Funding

This study was supported by the Pan American Health Organization (PAHO/WHO), the Special Programme for Research and Training in Tropical Diseases (TDR), and the Alliance for Health Policy and Systems Research (AHPSR) through a grant to the project entitled “Barreras y estrategias para la implementación de la Guía Nacional de Parto Normal en Brasil. Proyecto Parto Cuidadoso: monitoreo y adopción de buenas prácticas para el cuidado en el parto [Call for proposals ‘Embedding Research for the Sustainable Development Goals (ER-SDG)’]”. This study was also supported by a grant from the Ministry of Health of Brazil (TED MS/SCTIE-Fiocruz #43/2016) The funders had no role in the design, analysis, data interpretation or writing of the manuscript,

Author information

Authors and Affiliations

  1. Campus Universitário Darcy Ribeiro, Brasília, DF, CEP 70910-900, Brazil

    Viviane Karoline da Silva Carvalho & Everton Nunes da Silva

  2. Oswaldo Cruz Foundation, Brasília, DF, Brazil

    Jorge Otávio Maia Barreto

Authors
  1. Viviane Karoline da Silva Carvalho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Everton Nunes da Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jorge Otávio Maia Barreto
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Study conception and design: VKSC, JOMB, ENS. Analysis: VKSC. Data interpretation: VKSC, ENS. Drafting: VKSC, ENS. Critical revision: VKSC, JOMB, ENS. Final approval of the version to be published: VKSC, JOMB, ENS. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Viviane Karoline da Silva Carvalho.

Ethics declarations

Ethics approval and consent to participate

This study did not have to be submitted to the Research Ethics Committee, in accordance with Resolution 510/16 of the Brazilian National Health Council.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

da Silva Carvalho, V.K., da Silva, E.N. & Barreto, J.O.M. Public engagement in health technology assessment in Brazil: the case of the public consultation on National Clinical Guidelines for Care in Normal Birth. BMC Public Health 21, 1825 (2021). https://doi.org/10.1186/s12889-021-11855-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12889-021-11855-w

Keywords

BMC Public Health

ISSN: 1471-2458

Contact us