The U.S. COVID-19 County Policy Database: a novel resource to support pandemic-related research
BMC Public Health volume 22, Article number: 1882 (2022)
It is increasingly recognized that policies have played a role in both alleviating and exacerbating the health and economic consequences of the COVID-19 pandemic. There has been limited systematic evaluation of variation in U.S. local COVID-19-related policies. This study introduces the U.S. COVID-19 County Policy (UCCP) Database, whose objective is to systematically gather, characterize, and assess variation in U.S. county-level COVID-19-related policies.
In January-March 2021, we collected an initial wave of cross-sectional data from government and media websites for 171 counties in 7 states on 22 county-level COVID-19-related policies within 3 policy domains that are likely to affect health: (1) containment/closure, (2) economic support, and (3) public health. We characterized the presence and comprehensiveness of policies using univariate analyses. We also examined the correlation of policies with one another using bivariate Spearman’s correlations. Finally, we examined geographical variation in policies across and within states.
There was substantial variation in the presence and comprehensiveness of county policies during January-March 2021. For containment and closure policies, the percent of counties with no restrictions ranged from 0% (for public events) to more than half for public transportation (67.8%), hair salons (52.6%), and religious gatherings (52.0%). For economic policies, 76.6% of counties had housing support, while 64.9% had utility relief. For public health policies, most were comprehensive, with 70.8% of counties having coordinated public information campaigns, and 66.7% requiring masks outside the home at all times. Correlations between containment and closure policies tended to be positive and moderate (i.e., coefficients 0.4–0.59). There was variation within and across states in the number and comprehensiveness of policies.
This study introduces the UCCP Database, presenting granular data on local governments’ responses to the COVID-19 pandemic. We documented substantial variation within and across states on a wide range of policies at a single point in time. By making these data publicly available, this study supports future research that can leverage this database to examine how policies contributed to and continue to influence pandemic-related health and socioeconomic outcomes and disparities. The UCCP database is available online and will include additional time points for 2020–2021 and additional counties nationwide.
The COVID-19 pandemic has resulted in over 1 million deaths in the U.S. as of May 2022 . It has also caused tremendous financial hardship with nearly 15% unemployment at its peak in April 2020 , and millions falling into poverty . It is increasingly recognized that health and social policies played a role in both alleviating and exacerbating the health and economic consequences of the pandemic. For example, mandated business closures likely contributed to decreased levels of transmission [4, 5], but also job and income loss that disproportionately affected low-income groups and women . Meanwhile, economic policies like eviction moratoria ensured that families had the resources to stay healthy at home .
With limited coordination of federal guidance in COVID-19 policymaking relative to other high-income countries [8, 9], there has been substantial variation in state- and county-level COVID-19-related policies. The COVID-19 U.S. State Policy database at Boston University has systematically documented longitudinal variation in state policies on closures, shelter-in-place orders, housing protections, and more . Similarly, the Oxford COVID-19 Government Response Tracker documents national-level policies over time and collects subnational data for countries with substantial local variation (including the U.S.) . This database has documented substantial state-level variation in the comprehensiveness of containment and closure policies. Researchers have subsequently examined state policies’ effects on a variety of health and related outcomes, including whether minimum wage and paid sick leave policies affected food insufficiency during the pandemic .
In contrast, there has been little systematic documentation of county-level policies in the U.S., despite the fact that counties have been deeply involved in COVID-19 policymaking since early in the pandemic and that there are substantial differences in county-level COVID-19 rates even within a given state . For example, six counties in the San Francisco Bay Area issued the first coordinated shelter-in-place orders in the U.S. on March 16, 2020 even before an analogous state policy was implemented . A few studies have undertaken assessments of county policies for a handful of counties or for single type of policy (e.g., mask requirements) [15, 16]. Other studies seeking to explain differences in COVID-19-related outcomes have focused on county demographic characteristics such as population density or racial/ethnic composition, despite acknowledging the importance of policymaking [17, 18]. Without systematic and comprehensive documentation of county-level policies, it will be difficult to evaluate policies’ effects on health and economic outcomes.
In this study, we sought to fill this gap by gathering data on a range of county-level COVID-19-related policies from 171 counties in 7 states as the first phase of the new U.S. COVID-19 County Policy (UCCP) Database. We then characterize these policies both within and across states, testing the hypothesis that there is geographic variation in both the number and stringency of policies passed at the county and state levels.
Overview of data collection
The objective of the U.S. COVID-19 County Policy (UCCP) Database is to systematically gather, characterize, and assess variation in U.S. county-level COVID-19-related policies. This study summarizes the first phase of data collection, in which the research team gathered policy data for 171 counties in California, Louisiana, Mississippi, New Jersey, New York, Texas, and Utah (Supplemental Table 1). The total population of these counties is 90.4 million. While these counties are not nationally representative, they include over a quarter of the U.S. population and are diverse with respect to geography, race/ethnicity, and politics . The reasons for the selection of the given counties is described in the Supplemental Methods.
For these counties, we gathered data in January-March 2021 on COVID-19-related policies that were in effect at that time, capturing a cross-sectional picture of county policies during that period. Data collection proceeded for approximately eight weeks. Data were also collected for the corresponding states in which these counties are nested, with two waves of state policy data collection conducted in January 2021 just before county data collection, and again in February–March 2021 just after county data collection was completed.
We gathered data on 22 policies within three overarching domains: containment and closure, economic support, and public health measures (Table 1). These were in part modeled on national and state policy data currently collected through the Oxford COVID-19 Government Response Tracker, excluding those not applicable to counties (e.g., border control), and including additional policies that are primarily relevant at the county level that may affect health (e.g., housing support). For each policy, the study team assessed a sample of current policies across rural and urban counties in several states, and developed scoring criteria to assess comprehensiveness of the policy. These were aligned with the way in which policies and associated restrictions were framed at the county level (Supplemental Table 2). For example, one policy indicator captured public events with the following categories: minimal (≥ 50% capacity) limitations, major (< 50% capacity) limitations, recommended cancellation, and required cancellation. Each indicator also included a category to capture the scenario where there was explicitly no relevant restriction or program in place, and a category for missing where there was no information available to determine the policy/program in place. For school closures in this first phase of data collection, in counties with more than one school district or university, data collectors coded the district or university with the most stringent/comprehensive policy.
Data collectors used the Research Electronic Data Capture (REDCap) data entry and management platform . For each policy, data collectors abstracted scoring related to comprehensiveness, the effective date of the policy in place at the time of data collection, and source documentation. Data collectors gathered data from a variety of sources, including government websites, policy and government response summaries and databases, press releases, news articles, and social media posts by government organizations. Details on data collection are provided in the Supplemental Methods.
Imputation of missing data
Despite a thorough search of multiple sources, in some cases there was no information on the county policies of interest. This ranged from 4.7% for school closures to 50.3% for utility support (Supplemental Table 3). This was especially the case in rural counties (39.9% missingness across all policies), which are less likely to have robust public health departments than urban counties (23.8% missing) (Supplemental Table 4). For these, data on state policies were used to impute county policies. Since state data were gathered in two waves—just before and after county data collection—we used the closest state survey date to fill in the corresponding policy for missing county data. If there was no information on the policy at the state level either, then the policy remained coded as missing.
Additionally, we combined policies that were missing and those with clear documentation of no relevant restrictions, in essence assuming that there was no policy in place for those counties with no documented policy. While this assumption may not always be accurate, if policies were difficult for our trained staff to find after a thorough search, they were also likely to be difficult for residents to locate, resulting in no substantive restrictions in place. For example, counties with no clear documentation of a face covering policy and those with clear documentation that no face coverings were required were combined.
First, we calculated univariate distributions of each policy within each of the three overarching policy domains, documenting the range of comprehensiveness of each policy.
We then calculated bivariate Spearman’s correlations between each pair of policies. This type of non-parametric analysis examines the extent to which two ordinal ranked variables are associated with one another. In this case, it assessed the extent to which policies co-occurred in a given county, reflecting the fact that governments often implement bundles of policies on related issues [21, 22].
Next, we examined geographic distribution of the policies. First, for each state, we tabulated the mean number of policies implemented by counties in that state in each of the three policy domains. We then produced heatmaps for each state, documenting the number of policies present in each county (range 0–22). For these plots, policies were coded as binary (i.e., no policy versus any policy).
We next conducted several secondary analyses to account for possible bias introduced by the imputation process. To do so, for each analysis above, we calculated the results using data obtained directly from counties only, without imputation using state data.
Second, to allow for greater variation and nuance in policy landscapes, we calculated an overall index of comprehensiveness, rather than simply the number of policies implemented. For this analysis, if there was no policy, this was coded as 0, the most comprehensive were coded as 1, and intermediate categories were fractions thereof. For example, for public events, no restriction was coded as 0, minimal (≥ 50% capacity) limitations was 0.25, major (< 50% capacity) limitations was 0.50, recommended cancellation was 0.75, and required cancellation was 1. When summing the policies to achieve a total policy score for each county, the range was again 0–22, with non-integer values possible.
Finally, we used principal component analysis (PCA) as an alternative technique to create a composite index of policy comprehensiveness (see Supplemental Methods for details). We also examined the contributing policies to each of the principal components created by this technique to assess the relationships between the different policies in a different manner than the pairwise correlations described above.
Overall, policy data were most readily available for states and urban and suburban counties. Rural county data collection was more challenging since policies often were not documented or implemented, or websites were not updated regularly. Policy information was most often derived from health department and other government websites.
Variation in county-level COVID-19-related policies
Univariate analyses demonstrated substantial variation in the presence and comprehensiveness of county policies that were in effect during January-March 2021 (Table 2). For containment and closure policies (Panel A), the percent of counties with no restrictions in place for a given policy ranged from 0% for public events, to more than half for public transportation (67.8%), hair salons (52.6%), and religious gatherings (52.0%). For counties with restrictions, there were fewer restrictions in effect for schools, workplaces, restaurants, and childcare settings, with more restrictions for private gatherings and bars. Few policies demonstrated lack of documentation in this main analysis.
For economic response policies (Table 2, Panel B), 76.6% of counties had housing support, while 64.9% had utility relief. For public health policies (Table 2, Panel C), most were comprehensive, with 70.8% of county public information campaigns coordinated across traditional and social media, and 66.7% requiring facial coverings outside the home at all times. The percent of counties with no documentation for a given policy was less than 5%.
Results for analyses using county data only, without imputation using state data, are available in the Supplemental Results.
Correlations between county-level COVID-19-related policies
We next examined correlations between each pair of policies (Table 3). Containment and closure policies were more likely to be correlated with each other, with nearly half (38.1%) of correlations positive and moderate (0.4–0.59) to strong (> 0.6) . Most negative correlations were very weak (< 0.2). Meanwhile, correlations with and between economic response and public health policies included both positive and negative relationships, although these tended to be weak (0.2–0.39) or very weak, indicating less bundling for these policies. All correlations in this analysis were statistically significant at p < 0.05.
Geographic variation in county-level COVID-19-related policies
When comparing across states (Table 4), there was substantial variation in the mean number of COVID-19-related policies that counties in each state had in place. For containment and closure policies, this ranged from 5.5 policies passed per county in Mississippi to 14.1 in New York (out of 15 total possible). For economic response policies, this ranged from 1.1 in New York to 2.0 in Mississippi (out of 2 total possible). For public health policies, this ranged from 3.5 in Mississippi to 5.0 in California and New Jersey (out of 5 total possible).
Analyses also demonstrated variation in the number of county-level policies within states (Fig. 1; note that counties without shading were not included in our data collection). Most states had counties that fell into all three tertiles, with California, New York, and Utah having more counties in the top tertile. There were no counties with zero policies implemented.
When we examined scores for comprehensiveness of policies instead of summing binary indicators for whether any type of policy had been implemented (Supplemental Fig. 2), the variation across states again became even more pronounced. For example, counties in California were more likely to be in the top tertile, and counties in Louisiana were more likely to fall into lower tertiles.
We found that the first four principal components produced by the PCA loaded on combinations of policies that could be described as follows: (1) entity closures (e.g., business closures), (2) individual restrictions (e.g., on private gatherings), (3) state-funded programs (e.g., housing support), and (4) public health measures (e.g., vaccination). These were similar to the relationships we observed in the pairwise correlations described above. See Supplemental Methods and Supplemental Table 7 for additional details.
PCA results also indicated that counties in different states varied in the values of each of the different principal components (Supplemental Table 8). For example, counties in California and New York had the highest mean scores for the first principal component (entity closure). Meanwhile, counties in Texas and Mississippi had the highest mean scores on the second principal component (individual restrictions) followed by the counties in Mississippi, while New Jersey and Utah had the highest scores on the third principal component (state funding). Counties in New York and Texas had the highest scores on the fourth principal component (public health measures). The composite score shows that, overall, comprehensiveness of policies was greatest in counties in New York and California. Each principal component and the composite policy index also demonstrated substantial variation within states (Supplemental Fig. 4).
Assembling data from a variety of government and media websites, the U.S. COVID-19 County Policy (UCCP) Database is among the first to provide granular data on local governments’ policy responses to the COVID-19 pandemic. In the current study, we present findings from the first phase of data collection, including data from 171 counties across 7 states in the U.S. We document variation across and within states on 22 types of containment, economic, and public health policies, finding that there were important geographic differences in the types and comprehensiveness of policies to address the COVID-19 pandemic’s health and economic consequences. This adds to a body of literature that has examined variation in COVID-19 policymaking globally across countries and domestically at the state level [11, 24, 25], and suggests that future research examining the impact of policies on health during the pandemic needs to consider local variation that may result in substantial heterogeneity within states.
Several factors may explain the differences in county-level policymaking. These include health-related factors (like COVID-19 case rates, vaccines, or underlying morbidity and age distributions that may spur county policymakers to act), or sociodemographic characteristics (like population density or urbanicity that may have heightened concern for transmission). Local economic factors—e.g., unemployment or poverty rates—may have also driven policymaking around economic support for local residents. Alternatively, a stronger history of public health policymaking—e.g., around HIV/AIDS in the San Francisco Bay Area—may have led to more active COVID-19-related policymaking. Prior work has examined state characteristics associated with public health policymaking to address obesity , and future work could similarly examine the predictors of COVID-19-related policymaking.
It is also likely that the variation in policymaking in this study resulted in county variation in health outcomes. Studies before COVID-19 have often demonstrated the impacts of public health and economic policies on health and social outcomes [27,28,29], and a handful of studies during the COVID-19 pandemic have addressed similar questions, e.g., finding that state shelter-in-place policies affected mobility . This is an important area of future research, to inform policymaking during the later phases of this pandemic and future crises. Notably, research on the health impacts of COVID-19-related policies will be complicated by the collinearity of some policies (demonstrated here) and confounding by other unobserved place-based factors.
While patterns of policy comprehensiveness were similar when using county-only versus imputed county-plus-state data, the number of policies per county in each state varied much more. Specifically, some states were more likely to have policies enacted at the state level rather than by counties. There are several possible explanations for these findings. During this phase of the pandemic, these counties—which tend to be located in more rural states—may have been experiencing more limited transmission relative to counties in more urban states, such that county policymakers felt less urgency to act. Alternately, these counties are also located in states that are more right-leaning on indicators of political conservativeness , and which have been previously documented to have a less forceful policy response to COVID-19 in part perhaps due to a political philosophy supporting a more limited role of government . Relatedly, these states are more likely to engage in preemption of local policies, in which state governments restrict county governments’ abilities to act on a given issue . We did not capture state pre-emption policies, but pre-emption is known to affect other types of health policies like tobacco control and firearm safety . Future research can examine the extent to which this may have played a role in constraining county COVID-19-related policies. Interestingly, these same states also tend to be bigger proponents of federalism, in which authority over local affairs is ceded to states rather than the federal government, including for public health issues . It appears that the same logic does not extend to counties being afforded more flexibility when managing their own local affairs, despite the proliferation of local public health agencies in recent decades .
This study has several strengths, including the collection of policy data spanning 171 diverse counties in 7 states. We collected 22 policies across numerous domains, painting a detailed portrait of the policy landscape during this period of active policymaking and COVID-19 transmission. This study also has limitations. Counties were selected for inclusion based on the criteria of the parent project (see Supplement), and therefore do not constitute a generalizable or representative sample nor cover all 50 states. Similarly, data were collected in January-March 2021, and do not capture earlier or later phases of the pandemic. Nevertheless, even in this sample that is limited in geographic and temporal scope, we demonstrate substantial variation, and provide evidence that such data collection endeavors of county policies are critical to understanding how the pandemic has unfolded. Moreover, in ongoing work, we are conducting retrospective longitudinal weekly data collection for the period 2020–2021 from a larger swath of U.S. counties in all 50 states and Washington D.C. to fill this gap, and the current database will be updated with new data as it becomes available. Additionally, while the initial data collection presented here involved 22 policies hypothesized to be important for health, in ongoing work we are collecting additional data on a more robust set of social and economic support policies that may be important for addressing health disparities. Finally, policies are not always implemented or enforced as intended; importantly, we did not intend to capture actual enforcement or on-the-ground conditions, but rather the policies that were passed by governments during this time.
This study provides among the first evidence of variation in county-level COVID-19-related policies across multiple states, demonstrating substantial differences that reflect an active and diverse policymaking landscape. Assembling such data is the first step in developing evidence on how these policies contributed to and may continue to influence pandemic-related health and socioeconomic outcomes and disparities therein, and this study thereby informs future research on the health impacts of these policies. We encourage other investigators to leverage these data, which we are making publicly available, to advance research in this realm.
Availability of data and materials
The datasets used for the current study are available to any investigator at openICPSR, a service of the Inter-university Consortium for Political and Social Research: https://doi.org/10.3886/E180482V1.
Principal component analysis
U.S. COVID-19 County Policy
U.S. Centers for Disease Control and Prevention. Covid-19 Death Data And Resources. Daily Updates Of Totals By Week And State [https://www.cdc.gov/nchs/nvss/vsrr/covid19/index.html].
Falk G, Carter J, Nicchitta I, Nyhof E, Romero P. Unemployment Rates During The COVID-19 Pandemic: In Brief. In. Washington, D.C.: Congressional Research Service; 2021:2–16.
Parolin Z, Curran M, Matsudaira J, Waldfogel J, Wimer C. Monthly poverty rates in the United States during the COVID-19 pandemic. New York City: Columbia University Center On Poverty & Social Policy; 2020.
Song H, Mckenna R, Chen At, David G, Smith-Mclallen A. The Impact Of The Non-Essential Business Closure Policy On COVID-19 Infection Rates. Int J Health Econ Manag. 2021;21:387–42.
Hsiang S, Allen D, Annan-Phan S, Bell K, Bolliger I, Chong T, Druckenmiller H, Huang L, Hultgren A, Krasovich E, et al. The effect of large-scale anti-contagion policies on the COVID-19 pandemic. Nature. 2020;584(7820):262–7.
Ewing-Nelson C. After a full month of business closures, women were hit hardest by April’s job losses. Washington, D.C: National Women’s Law Center; 2020.
Raifman J, Bor J, Venkataramani A. Association between receipt of unemployment insurance and food insecurity among people who lost employment during the covid-19 pandemic in the United States. JAMA Netw Open. 2021;4(1):E2035884–E2035884.
Carter DP, May PJ. Making sense of the U.S. COVID-19 pandemic response: a policy regime perspective. Adm Theory Prax. 2020;42(2):265–77.
Nuzzo JB, Bell JA, Cameron EE. Suboptimal U.S. response to COVID-19 despite robust capabilities and resources. JAMA. 2020;324(14):1391–2.
Raifman J, Nocka K, Jones D, Bor J, Lipson S, Jay J, Chan P, Galea S. COVID-19 U.S. State Policy Database. Ann Arbor: Inter-University Consortium For Political And Social Research; 2020.
Hale T, Angrist N, Goldszmidt R, Kira B, Petherick A, Phillips T, Webster S, Cameron-Blake E, Hallas L, Majumdar S, et al. A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker). Nat Hum Behav. 2021;5(4):529–38.
Raifman J, Nsoesie E, Dean LT, Gutierrez K, Raderman W, Skinner A, Shafer P. State Minimum Wage, Paid Sick Leave, And Food Insufficiency During The COVID-19 Pandemic. Medrxiv 2021. https://doi.org/10.1101/2021.03.01.21252723.
Messner W, Payson SE. Variation in COVID-19 outbreaks at the U.S. state and county levels. Public Health. 2020;187:15–8.
San Francisco Chronicle. The Bay Area's Battle Against Coronavirus [https://projects.sfchronicle.com/2020/coronavirus-timeline/].
Lasry A, Kidder D, Hast M, Poovey J, Sunshine G, Winglee K, Zviedrite N, Ahmed F, Ethier KA, CDC Public Health Law Program, et al. Timing of community mitigation and changes in reported COVID-19 and community mobility-four U.S. metropolitan areas, February 26–April 1, 2020. Morb Mortal Wkly Rep. 2020;69(15):451–7.
Guy GP Jr, Lee FC, Sunshine G, Mccord R, Howard-Williams M, Kompaniyets L, Dunphy C, Gakh M, Weber R, Sauber-Schatz E. Association of state-issued mask mandates and allowing on-premises restaurant dining with county-level COVID-19 case and death growth rates—United States, March 1–December 31, 2020. Morb Mortal Wkly Rep. 2021;70(10):350.
Ives AR, Bozzuto C. Estimating and explaining the spread of COVID-19 at the county level in the USA. Commun Biol. 2021;4(1):60.
Pan W, Miyazaki Y, Tsumura H, Miyazaki E, Yang W. Identification of county-level health factors associated with COVID-19 mortality in the United States. J Biomed Res. 2020;34(6):437–45.
American Community Survey [https://www.census.gov/programs-surveys/acs/].
Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (redcap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81.
Grumbach JM. From backwaters to major policymakers: policy polarization in the states, 1970–2014. Perspect Polit. 2018;16(2):416–35.
Riley AR, Collin D, Grumbach JM, Torres JM, Hamad R. Association of Us State policy orientation with adverse birth outcomes: a longitudinal analysis. J Epidemiol Community Health. 2021;75:689–94.
Evans JD. Straightforward Statistics For The Behavioral Sciences. Pacific Grove, California: Thomson Brooks/Cole Publishing Co; 1996.
Hale T, Atav T, Hallas L, Kira B, Phillips T, Petherick A, Pott A. Variation In U.S. States Responses To COVID-19. Working Paper 2020/034. Oxford: Blavatnik School Of Government, Oxford University; 2020.
Covid-19 U.S. State Policy Database [https://statepolicies.com/].
Arons A, Pomeranz J, Hamad R. Identifying novel predictors of state legislative action to address obesity. J Public Health Manag Pract. 2021;27(1):E9–E18.
Collin DF, Shields-Zeeman LS, Batra A, White JS, Tong M, Hamad R. The effects of state earned income tax credits on mental health and health behaviors: a quasi-experimental study. Soc Sci Med. 2021;276:113274.
Lee BC, Modrek S, White JS, Batra A, Collin DF, Hamad R. The effect of California’s paid family leave policy on parent health: a quasi-experimental study. Soc Sci Med. 2020;251:112915.
Wehby G, Dave D, Kaestner R. Effects of the minimum wage on infant health. J Policy Anal Manag. 2019;39(2):411–43.
Feyman Y, Bor J, Raifman J, Griffith KN. Effectiveness of Covid-19 shelter-in-place orders varied by state. Plos One. 2021;15(12):E0245008.
Pomeranz JL, Pertschuk M. State preemption: a significant and quiet threat to public health in the United States. Am J Public Health. 2017;107(6):900–2.
Turnock BJ, Atchison C. Governmental public health in the United States: the implications of federalism. Health Affairs. 2002;21(6):68–78.
The authors would like to acknowledge Erica Johnson for her administrative support of this project, as well as the data collectors: Jordan Bodzin, Kyrsten Jones, Claire Purcell, Cady Smith, Margarethe Sylvester, and Nicholas Villalobos.
Funding for this work was provided by the Patient-Centered Outcomes Research Institute (grant number COVID-2020C2-10761) and the National Institute of Mental Health (grant number U01MH129968). The funders had no role in the design of the study; collection, analysis, and interpretation of data; or writing the manuscript.
Ethics approval and consent to participate
This study did not involve human subjects and ethical approval was therefore not required.
The authors declare that they have no competing interests.
Publisher’ s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Hamad, R., Lyman, K.A., Lin, F. et al. The U.S. COVID-19 County Policy Database: a novel resource to support pandemic-related research. BMC Public Health 22, 1882 (2022). https://doi.org/10.1186/s12889-022-14132-6