- Research
- Open Access
- Published:
Psychometric properties of public trust in Covid-19 control and prevention policies questionnaire
BMC Public Health volume 22, Article number: 1959 (2022)
Abstract
Background
Public trust is a crucial concept in the COVID-19 pandemic, which determines public adherence with preventive rules as a success factor for disease management. This study aimed to develop and validate a tool to measure public trust in COVID-19 control and prevention policies (COV-Trust tool).
Methods
This is a psychometric study that was conducted in 2020 (March-August). A primary tool was developed through literature review, in-depth interviews with experts and expert panel meetings. Content and construct validity was evaluated using content validity index (CVI) and content validity ratio (CVR) indexes and exploratory and confirmatory factor analysis, respectively. Cronbach α coefficient was calculated to determine the internal consistency.
Results
A 28-item questionnaire with seven factors was developed. Factors included macro policy-making and management of pandemic, pandemic control policies implementing at all levels and their effectiveness, providing protective equipment and medicine for hospitals and public, prevention of negative socio-economic consequences of the pandemic, public participation, informing and public education and public behavior. The questionnaire reliability was calculated to be α = 0.959. Based on the experts’ opinion, tool content validity was estimated to be CVR = 0.73, CVI = 0.89. RMSEA = 0.07 revealed a good model fit as the confirmatory factor analysis results for the tool.
Conclusion
COV-Trust tool is a well-fit tool to be used during this pandemic for improving policies effectiveness and could be used in similar situations as it determines the success of public health interventions.
Highlights:
Investigating public trust during the outbreak of Covid-19 is of utmost important for both research and policy making perspectives.
COV-Trust tool (Seven factors and 31 items) was developed and validated.
COV-Trust tool comprehensively assesses public trust in various aspects of policies and their determinants and can provide valuable information for promoting the effectiveness of policies.
Introduction
The World Health Organization (WHO) identified COVID-19 as an epidemic in 2020 and considered it worrying and high-risk for countries with vulnerable health systems [1]. This is a serious threat to public trust in the government and the health system, leading to considerable negative consequences if appropriate action is not taken. Trust is one of the Achilles heels of governments and health systems [2]. Trust is the basis of the relation between people, the health system, and health providers. Patients often disclose their personal information and are sure that it will remain confidential[3]. To achieve this level of trust, health care systems should convince people that patients’ interests are more important than their personal or financial interests [4]. However, studies have shown that trust in health systems has declined over the past fifty years or more [5].
Public trust is an important concept that refers to the degree of public confidence and belief in the ability and competence of the health system to meet the needs and response ideally [6]. The higher level of trust, the easier to manage the unfavourable effects of an epidemic which are largely controlled by some health intervetions such as hand washing, staying home, and travel restrictions [6,7,8]. Public reactions to governments’ restrictive policies may depend on the level of their trust in health policymakers. However, little information is known about the effect of trust on adherence to health and safety regulations [9]. Most countries have taken several measures against this pandemic, from severe repression methods such as quarantine, forced social distancing, schools and unnecessary economic businesses and activities closures [10]. Public trust is more crucial when health rules require the contribution and cooperation of all members of the community [11, 12]. The lesson learnt from the recent outbreak of Ebola in 2018–2019 suggests that public trust in each country’s health system is vital to safety measures establishment [7]. Trust not only determines the success or failure of a risk management strategy, but the selected policy can also affect public trust and determine the success rate of future policies [13]. Investigations in diseases similar to COVID-19 have shown that it is essential to recognize community responses and insights into controlling and prevention policies [14, 15]. Therefore, public trust during the outbreak of Covid-19 is as important as clinical, epidemiological, and genomic studies. Most studies on trust during COVID-19 have examined political, organizational, information or individual trust related to Covid-19 disease [11, 16,17,18,19]. Farzanegan et al. (2022) concluded that the high mortality rate due to COVID-19 was negatively associated with public trust in government [18]. COVID-19 is slightly different from infectious diseases as it is a prolonged disease with various types of variants that may need different policies to be well controlled. This situation is required a high level of public trust in the government and policies to reduce the negative consequences of the disease. Considering that public trust in Covid-19 control and prevention policies is one of the topics that have been less addressed and comprehensive tools have not been provided to measure it, this study aimed to develop and validate a public trust in Covid-19 control and prevention policies questionnaire.
Methods
This is a quantitative study conducted to develop and validate public trust in Covid-19 control and prevention policies (COV-Trust tool) questionnaire (annex 1). The study was developed and conducted in Iran from March to August 2020.
Item generation and selection:
To identify the questionnaire items, a comprehensive literature search was conducted through Scopus, PubMed and Web of Sciences databases using the keywords of COVID-19, Corona, trust, policy, public, and government. Retrieved articles were reviewed, and two research team members extracted preliminary items. Parallelly, in-depth interviews were conducted with five experts (n = 5) in the field to achieve more items. Experts were academics in field of Health Services Management (n = 2), Health Policy (n = 1), Psychology (n = 1), and Epidemiology (n = 1). Experts were selected through purposive sampling. As a result of these two steps, 41 initial items were generated. Members of the research team reviewed the initial list during a meeting, and after merging similar items and excluding the duplicates, 35 items remained.
Furthermore, an 1.5-hour expert panel session was held in the faculty in April 2020 and generated items were reviewed, and required editions were made in items content. Moreover, items were categorized into seven categories based on their content similarity. Experts included individuals with the related field of study such as Health Education (n = 1), Health Policy (n = 3), Health Services Management (n = 2), Epidemiology (n = 2), experts who were engaged in COVID-19 policymaking at the provincial level (n = 2).
Content validity
To determine the content validity of the questionnaire, two methods were used, including qualitative (experts’ opinions and suggestions) and quantitative (content validity index (CVI) and content validity ratio (CVR) methods. At this stage, the content validity form was provided to the experts and they were asked to evaluate each item based on the four criteria of “simplicity”, “relevance”, “transparency” and “necessity” (Table 1).
According to the experts’ opinions, the content validity index (CVI) and content validity ratio (CVR) were calculated for each questionnaire item and the whole questionnaire. CVR and CVI were used to assess the content validity quantitatively. To calculate the CVI, three criteria of simplicity, relevance and transparency were used for each item, and the CVI score was calculated by summing the agreeing scores for each item ranked third and fourth on the total number of experts. Items with a score above 0.79 were approved. To calculate the CVR, the criterion of necessity in a 3-part Likert scale was used and calculated based on the following formula.
nE: Number of experts who have responded to the “necessary” option.
N = Total number of experts.
At this stage, experts’ inclusion criteria were related academic education, research experience on COVID-19 policy and management issues/ public trust, and managerial experience in the health system at the city level or higher since the onset of the COVID-19 pandemic.
Construct validity
To evaluate the construct validity of the Covid-Trust measurement tool, the data were randomly divided into two groups of 395 (15 cases out of 805 were removed because of missing data). In the first set of data, exploratory factor analysis (EFA) and in the second set, confirmatory factor analysis was applied. Factors were extracted through principal component with varimax rotation. Eigenvalue > 1 and theoritical interpretability was used for factors extraction. Model fit was examined using X2, χ2 / df, root mean square error of approximation (RMSEA), Goodness of Fit (GFI), Tucker–Lewis index (TLI), Normed Fit Index (NFI), and comparative fit index (CFI) parameters. In examining the indices, if the result of X2 analysis is significant, it indicates that the model does not fit. Values close to zero in RMSEA index indicate more fit of the model and TLI, NFI and CFI indices values more than 0.95 indicate a good model fit. For the GFI index, a cut-off point of 0.0.95 was reported [20].
Reliability
Cronbach α coefficient was calculated to evaluate and determine the internal consistency of the questionnaire. The internal consistency of the items and scales of the questionnaire was assessed independently by calculating the Cronbach α coefficient for the whole questionnaire and each of its components.
Sampling
Based on the number of questionnaire items (29 items) and the norm provided to determine the sample size of factor analysis (10 to 15 times the number of items), 805 samples participated in the study. The study statistical population included members of virtual networks (Whatssapp and Telegram) throughout the country which were the most popular in Iran. A convenience random sampling was used to select the samples. As channels wih countrywide members were selected and the online questionaire link was published in channels in 4 different days in a week including weekend, first, third and fifth days of the week. Data collection was done from May to july 2020.
Statistical analysis
Cronbach α index was calculated to assess the reliability by the internal consistency method. Values greater than 0.7 were considered as good reliability. To investigate the tool construct validity, exploratory factor analysis (EFA) was run and then confirmatory factor analysis (CFA) was used. Analyses were performed using SPSS 26 and Amos 26 software.
Results
Population characteristics
A total of 805 people participated in the study. Most of the participants in the study were female (53.5%) and had a bachelor’s degree (32.7%) and a master’s degree (32.5%). The majority of the participants were in the age group of 21 to 39 years (60.3%). In terms of income level, most people (60.1%) had declared a moderate-income level (Table 2).
Content validity
Seven factors and 31 items were selected based on expert opinions. After reviewing experts’ opinions, content validity indices were calculated for all items and two items were excluded due to CVR score < 0.42 and CVI < 0.79. Finally, 29 items were analysed for construct validity.
Construct validity
Model fit indices in exploratory factor analysis confirmed the 28-item, seven-factor model (Table 3). One item was eliminated bacause of lower loading than 0.3 in EFA. Extracted factors include macro policy-making and management of pandemic, pandemic control policies implementing at all levels and their effectiveness, providing protective equipment and medicine for hospitals and public, prevention of negative socio-economic consequences of the pandemic, public participation, informing and public education and public behavior (Appendix 1). CFA approved the extracted factors with good model fit indices (Fig. 1). Also, the relationship between all items and components and the correlation between the seven components was statistically significant (P < 0.001) and the standardized regression weight of all items with related factors was more than 0.3.
Reliability
Examining the internal consistency of questionnaire structures showed that all scales and subscales of the questionnaire have the minimum reliability standard (0.7) and Cronbach α coefficients were calculated to confirm the reliability of this questionnaire at the desired level (Table 4).
Discussion
The present study developed and validat ed public trust (COV-Trust tool) in the context of Covid-19 control and prevention policies. The COV-Trust tool was designed to measure trust in heath policies through 28 items in 7 components with the reliability of α = 0.959 and validity of CVR = 0.73, CVI = 0.89. Its components include macro policy-making and management of pandemic, pandemic control policies implementing at all levels and their effectiveness, providing protective equipment and medicine for hospitals and public, prevention of negative socio-economic consequences of the pandemic, public participation, informing and public education and public behavior. The exploratory and confirmatory factor analysis results showed good model fit and construct validity of this tool.
Trust is the cornerstone of the relationship between people and public institutions such as the health system. It has been introduced as a determining factor in public adherence to social and even personal health protocols [4, 12, 21]. Studies have shown that public trust in the health system has declined over the past decades [22] and the outbreak of the Coronavirus pandemic caused a significant threat in this regard [4]. During the pandemic, policies to control and prevent coronavirus disease, including social distancing, quarantine, using a mask, and handwashing, was recommended by the WHO, which required public adherence in their personal and social life [1]. Public trust in these policies is the weak point of this issue, and governments must make the necessary predictions about this issue since people trust in the policies adopted by governments and their adherence will determine the future of the Covid-19 pandemic [6, 13].
Prior studies have investigated the impact of political trust on the social behaviour of the public during the Covid-19 pandemic and have shown that the political trust was associated with public adherence to the required social behaviours during the pandemic, especially in non-pharmaceutical interventions [9, 16, 17]. Low public trust leads to people’s violations of quarantine rules and regulations [18, 23]. It is also true about the coronavirus vaccination, as evidence suggests that lack of trust in vaccines’ safety and efficacy has caused hesitancy among the public and health professionals[24]. Continuous assessment of public trust in this area will provide helpful information for policymakers to make necessary changes and reforms in current policies based on the people’s preferences.
The questionnaire, in the dimension of macro policy-making and management of pandemic (α = 0.906), assesses public trust in the integrity, credibility and competence of participants in the process [25] by asking the people on the use of scientific and up-to-date evidence, experts engagement, prioritization of vulnerable individuals and groups, as well as benchmarking and using the experiences of other countries. The second component of the questionnaire examines public trust in the implementation and effectiveness of policies at all national, regional and local levels (α = 0.778). This dimension measures the integrity of policies at all management levels of the country. Lack of coordination in policies and executive programs at national and local levels leads to low effectiveness [26]. Trust in the equitable supply and distribution of medical and protective equipment is one of the most critical issues in coronavirus control policies, which is assessed by the third dimension of this tool (α = 0.842). Coordination in the system of preparation and distribution of this equipment and its feedback at the community level will lead to a change in the level of public trust [27].
Prevention of pandemic negative socioeconomic consequences and Public behavior are the other two dimensions of the questionnaire, which includes topics that affect people’s lives especially vulnerable groups. It affects people peace of mind and their trust in the proper functioning of the government in this regard. In addition to creating social peace and preventing harmful movements of groups, it can increase people adherence to announced health guidelines [28, 29]. Public participation is another dimension assessed by the designed tool (α = 0.777). Public assurance of the government’s intention to use the local potentials and public participation in its policies to control the pandemic could facilitate the government [30]. Participation can occur in a range of activities and at different levels, from informing participation in planning and evaluation of policies [31]. Continuous monitoring of public trust in information channels and preventing inaccurate information dissemination is an issue assessed by the developed tool. The importance of informing and public education in the coronavirus pandemic and the involvement of different individuals and groups in this issue led to the creation of “infodemic” terminology [32]. Paving the way for the dissemination of accurate and precise information and gaining people trust in the determined paths is one of the necessities to increase trust, which results in increased people adherence to control and prevention policies, especially in the social aspect [19, 33].
The study has some limitations which should be considered when generalizing the results. This study was done during the pandemic, and the estimations may be affected by the peoples’ mental and psychological status. The study samples were individuals who could use online virtual networks, which may not represent the population well. Moreover, we did not use any external criterion instrument to assess the concurrent validity. Also, for reliability analysis, we did not use test-retest analysis.
Conclusion
The COV-Trust tool developed in this study could be used as a standard tool by researchers and governments to independently assess public trust in Covid-19 control and prevention policies. This tool comprehensively assesses public trust in various aspects of policies and their determinants and can provide valuable information for promoting the effectiveness of policies.
Availability of data and material
Interested author can contct Mohammad Saadati (saadati_m@khoyums.ac.ir, hcm.2020@gmail.com) for relevant data and information.
Abbreviations
- CFI:
-
Comparative fit in.
- CVI:
-
Content validity index.
- CVR:
-
Content validity ratio.
- GFT:
-
Goodness of Fi.
- NFI:
-
Normed Fit Index.
- RMSEA:
-
Root mean square error of approximation.
- TLI:
-
Tucker–Lewis index.
- WHO:
-
World Health Organization.
References:
World Health Organization. Novel Coronavirus (2019-nCoV): situation report, 3. 2020.
Huang EC-H, Pu C, Chou Y-J, Huang N. Public trust in physicians—health care commodification as a possible deteriorating factor: cross-sectional analysis of 23 countries. INQUIRY: The Journal of Health Care Organization Provision and Financing. 2018;55:0046958018759174.
Nouri M, Ghaffarifar S, Sadeghi Bazargani H, Ghaffari R. Patients’ Satisfaction with Medical Residents’Communication Skills at the Largest Teaching and Treatment Center in North West Iran in 2016. Shiraz E-Med J. 2017;18(4):e60032.
Baker DW. Trust in Health Care in the Time of COVID-19. JAMA. 2020;324(23):2373–5.
Melhado EM. Health Planning in the United States and the Decline of Public-interest Policymaking. Milbank Q. 2006;84(2):359–440.
Earle T, Siegrist M. Trust, confidence and cooperation model: a framework for understanding the relation between trust and risk perception. Int J Global Environ Issues. 2008;8(1–2):17–29.
Blair RA, Morse BS, Tsai LL. Public health and public trust: Survey evidence from the Ebola Virus Disease epidemic in Liberia. Soc Sci Med. 2017;172:89–97.
Paek H-J, Hove T. Risk perceptions and risk characteristics. Oxford research encyclopedia of communication; 2017.
Bargain O, Aminjonov U. Trust and compliance to public health policies in times of COVID-19. J Public Econ. 2020;192:104316.
Flaxman S, Mishra S, Gandy A, Unwin HJT, Mellan TA, Coupland H, et al. Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe. Nature. 2020;584(7820):257–61.
Davies B, Lalot F, Peitz L, Heering MS, Ozkececi H, Babaian J, et al. Changes in political trust in Britain during the COVID-19 pandemic in 2020: integrated public opinion evidence and implications. Humanit Social Sci Commun. 2021;8(1):1–9.
Wong CML, Jensen O. The paradox of trust: perceived risk and public compliance during the COVID-19 pandemic in Singapore. J Risk Res. 2020;23(7–8):1021–30.
Meredith LS, Eisenman DP, Rhodes H, Ryan G, Long A. Trust influences response to public health messages during a bioterrorist event. J health communication. 2007;12(3):217–32.
Leung GM, Ho L-M, Chan SK, Ho S-Y, Bacon-Shone J, Choy RY, et al. Longitudinal assessment of community psychobehavioral responses during and after the 2003 outbreak of severe acute respiratory syndrome in Hong Kong. Clin Infect Dis. 2005;40(12):1713–20.
Vaughan E, Tinker T. Effective health risk communication about pandemic influenza for vulnerable populations. Am J Public Health. 2009;99(S2):324-S32.
Siegrist M, Zingg A. The role of public trust during pandemics: Implications for crisis communication. Eur Psychol. 2014;19(1):23.
Olsen AL, Hjorth F. Willingness to distance in the COVID-19 pandemic. Retrieved from: osf io/uanhg. 2020.
Farzanegan MR, Hofmann HP. A Matter of Trust? Political Trust and the Covid-19 Pandemic. CESifo Working Paper No 9121. 2021.
Ipsen C, Myers A, Sage R. A cross-sectional analysis of trust of information and COVID-19 preventative practices among people with disabilities. Disabil health J. 2021;14(2):101062.
D I. Structural equations modeling: Fit indices, sample size, and advanced topics. Journal of consumer psychology. 2010;1(20):90–8.
Alijanzadeh M, Ahorsu DK, Alimoradi Z, Mahmoudi N, Griffiths MD, Lin C-Y, et al. Fear of covid-19 and trust in the healthcare system mediates the association between individual’s risk perception and preventive covid-19 behaviours among iranians. Int J Environ Res Public Health. 2021;18(22):12146.
Blendon RJ, Benson JM, Hero JO. Public trust in physicians–U.S. medicine in international perspective. N Engl J Med. 2014;371(17):1570–2.
Saechang O, Yu J, Li Y, editors. Public Trust and Policy Compliance during the COVID-19 Pandemic: The Role of Professional Trust. Healthcare: Multidisciplinary Digital Publishing Institute; 2021.
Daly M, Jones A, Robinson E. Public Trust and Willingness to Vaccinate Against COVID-19 in the US From October 14, 2020, to March 29, 2021. JAMA. 2021.
Cairney P, Wellstead A. COVID-19: effective policymaking depends on trust in experts, politicians, and the public. Policy Des Pract. 2021;4(1):1–14.
Iverson T, Barbier E. National and Sub-National Social Distancing Responses to COVID-19. Economies. 2021;9(2):69.
Castro-Sánchez E, Alexander C, Atchison C, Patel D, Leung W, Calamita M, et al. Evaluation of a personal protective equipment support programme for staff during the COVID-19 pandemic in London. J Hosp Infect. 2021;109:68–77.
Montoya-Torres JR, Muñoz-Villamizar A, Mejia-Argueta C. Mapping research in logistics and supply chain management during COVID-19 pandemic. International Journal of Logistics Research and Applications. 2021:1–21.
Josephson A, Kilic T, Michler JD. Socioeconomic impacts of COVID-19 in low-income countries. Nat Hum Behav. 2021;5(5):557–65.
Farida I, Setiawan R. Public Participation In Preventing Covid-19 Outbreaks In Lampung Province, Indonesia. RUDN J Public Adm. 2021;8(3):297–311.
Kotus J, Sowada T. Behavioural model of collaborative urban management: extending the concept of Arnstein’s ladder. Cities. 2017;65:78–86.
Purnat TD, Vacca P, Czerniak C, Ball S, Burzo S, Zecchin T, et al. Infodemic Signal Detection During the COVID-19 Pandemic: Development of a Methodology for Identifying Potential Information Voids in Online Conversations. JMIR infodemiology. 2021;1(1):e30971.
Llewellyn S. Covid-19: how to be careful with trust and expertise on social media. BMJ. 2020;368.
Acknowledgements
It is our honour to appreciate all participants for their contribution. Moreover, we appreciate Dr. Mehrdad Karimi for his contribution in data analysis.
Funding
The study was supported by Student Research Committee of Tabriz University of Medical Sciences. However, researchers have received their salary from their respective universities.
Open access funding provided by Mid Sweden University.
Author information
Authors and Affiliations
Contributions
RAK: Conceptualisation, Data Collection, Writing first draft.
SAA: Data collection, writing.
RR: Data collection, writing.
MN: Data collection, writing.
ND: Data collection, writing.
MS: Conceptualisation, Data collection, Data analysis, writing and Data curator. * Same role as corresponding author and contact for data.
KD: Conceptualisation, writing, Critical review.
All authors have contributed, read and approved the manuscript.
Corresponding authors
Ethics declarations
Ethical approval and consent to participate
The study has received ethical permission from ethical committee of Tabriz University of Medical Sciences (IR.TBZMED.REC.1399.185). Written informed consent received from the respondents. All methods were carried out in accordance with relevant guidelines and regulations.
Consent for publication
Not applicable.
Competing interests
Declared none.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Kalajahi, R.A., Saadati, M., Aghdash, S.A. et al. Psychometric properties of public trust in Covid-19 control and prevention policies questionnaire. BMC Public Health 22, 1959 (2022). https://doi.org/10.1186/s12889-022-14272-9
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12889-022-14272-9
Keywords
- Trust
- COVID-19
- Policy
- Management
- Prevention