Skip to main content
Download PDF

Methodological and reporting quality of systematic reviews on health effects of air pollutants were higher than extreme temperatures: a comparative study

BMC Public Health volume 23, Article number: 2371 (2023) Cite this article

Abstract

Background

An increasing number of systematic reviews (SRs) in the environmental field have been published in recent years as a result of the global concern about the health impacts of air pollution and temperature. However, no study has assessed and compared the methodological and reporting quality of SRs on the health effects of air pollutants and extreme temperatures. This study aims to assess and compare the methodological and reporting quality of SRs on the health effects of ambient air pollutants and extreme temperatures.

Methods

PubMed, Embase, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), Cochrane Library, Web of Science, and Epistemonikos databases were searched. Two researchers screened the literature and extracted information independently. The methodological quality of the SRs was assessed through A Measurement Tool to Assess Systematic Reviews 2 (AMSTAR 2). The reporting quality was assessed through Preferred Reporting Items of Systematic reviews and Meta-Analyses (PRISMA).

Results

We identified 405 SRs (286 for air pollution, 108 for temperature, and 11 for the synergistic effects). The methodological and reporting quality of the included SRs were suboptimal, with major deficiencies in protocol registration. The methodological quality of SRs of air pollutants was better than that of temperature, especially in terms of satisfactory explanations for any heterogeneity (69.6% v. 45.4%). The reporting quality of SRs of air pollution was better than temperature, however, adherence to the reporting of the assessment results of risk of bias in all SRs (53.5% v. 34.3%) was inadequate.

Conclusions

Methodological and reporting quality of SRs on the health effect of air pollutants were higher than those of temperatures. However, deficiencies in protocol registration and the assessment of risk of bias remain an issue for both pollutants and temperatures. In addition, developing a risk-of-bias assessment tool applicable to the temperature field may improve the quality of SRs.

Peer Review reports

Background

The environmental consequences of climate change such as increasing air pollution and extreme temperature events are impacting human health and lives [1]. Ambient air pollution, including sulfur dioxide (SO2), carbon monoxide (CO), nitrogen dioxide (NO2), particulate matter (PM), such as PM2.5, PM10 PM1, black carbon (BC) and ultrafine particles (UFP), and Ozone(O3), is the fourth contributing factor to death worldwide [2, 3]. According to the World Health Organization (WHO), nine out of ten people around the world breathe polluted air, contributing to 7.0 million deaths each year [3]. In addition, the 2017 Global Burden of Disease study (GBD) reported 4.6 million deaths from PM and 500,000 deaths from O3 [4]. Compared to 2017, the burden of PM increased by 44.6% in 2019 [5]. Therefore, health effects attributable to short-term and long-term air pollutants exposure have been a major threat shared by people worldwide [1].

Extreme temperature events, such as heat waves, cold spells, extreme heat, and extreme cold, are significant contributors to climate change and major risk factors for human health, causing increasing concern among governments and the general public [6]. GBD 2019 adds three risk factors: non-optimal temperature, high temperature, and low temperature, with the non-optimal temperature being the tenth risk factor for the death of women in 2019 (940,000 deaths) [5]. Due to rapid temperature rise, vulnerable populations have been exposed to 3.7 billion more person-days of heatwaves in 2021 than annually in 1986–2005 [7]. Extreme temperature events can directly affect health, and they can also affect physical and mental well-being through less direct pathways such as the recurrence of infectious diseases [8]. Additionally, there are synergistic effects between air pollution and temperature. Some studies have shown that extreme heat significantly increases the impacts of air pollutants such as PM10 and O3 on mortality, and extreme cold increases the health impacts of PM10 on respiratory disease [9,10,11].

A large number of SRs in the environmental field have been published recently as a result of the global concern about the health impacts of air pollution and temperature. Systematic reviews are regarded as high-level evidence and can provide accurate, succinct, credible, and comprehensive summative evidence for policy making [12]. As an evidence-based practice, it is one of the key tools for guideline developers and policy-makers [13]. Methodology and reporting are two crucial steps for SRs, the quality of which is key to producing a high-quality systematic review. Methodological quality determines whether the evidence is robust while reporting quality reflects the completeness and comprehensiveness of the SRs [14, 15].

Some previous studies [16,17,18,19] have found that the methodological and reporting quality of published SRs were suboptimal. Sheehan et al. have reviewed environmental health SRs published from 1990 to 2013 and ambient air pollution SRs published from 2009 to 2015, using a questionnaire consisting of major items shared by several guidelines, such as the PRISMA checklist and Meta-Analysis of Observational Studies in Epidemiology (MOOSE) statement [20, 21]. A limitation of systematic reviews conducted by Sheehan et al. is that they merely focused on reporting quality of environmental health SRs. However, the methodological quality determines the strength of evidence. The number of SRs for air pollutants and temperatures has significantly expanded, with more than 300 SRs published in 2015 and beyond. No recent review that we are aware of has updated the research.

We don’t know how recently the quality of SR in environmental health has been. Therefore, this study aims to (i) evaluate the methodological and reporting quality of SRs in environmental health, focusing on the health effects of ambient air pollution and temperature, using AMSTAR 2 and PRISMA checklist; (ii) compare the methodological and reporting quality of SRs on health effects of ambient air pollution and temperature.

Methods

Literature search

Six electronic databases were searched to identify relevant literature: PubMed, Embase, CINAHL, the Cochrane Library, Web of Science, and Epistemonikos. The search was limited to English language publications and covered articles published from database inception until July 1, 2022, and then updated to October 9, 2023. Medical Subject Headings (MeSH) and free text were reviewed along with specific keywords to construct a comprehensive search strategy, based on the PEOS principles (Population, Exposure, Outcome, and Study design). The full search strategies are available in Additional file 1. Table S1.

Literature selection

Literature screening was performed independently by two researchers. When the opinions of the two reviewers differed, differences were resolved through consultation with a third reviewer. Two reviewers screened all titles and abstracts and any full-text retrieved, to determine eligibility. Endnote X9 software was used to identify and reject duplicates. The inclusion and exclusion criteria were based on PEOS principles.

The inclusion criteria

  1. i

    Population: no restriction on disease types;

  2. ii

    Exposure: temperature (e.g., extreme heat, extreme cold, heat waves, cold waves) and/or ambient air pollutants (e.g., PM10, PM2.5, PM1, SO2, NO2, O3, CO, BC, UFP) and/or greenhouse gases from climate change (e.g., CO2, CH4, N2O, HFCs, PFCs, SF6);

  3. iii

    Outcome: morbidity or mortality;

  4. iv

    Study design: systematic reviews and/or meta-analyses.

The exclusion criteria

(i) Studies focusing only on indoor exposure or special site exposure; (ii) Studies that considered only seasonal effects rather than temperature impacts ; (iii) Animal studies; (iv) Articles not published in English.

Data extraction

Data extraction was performed independently by two reviewers using a predesigned table in Microsoft Excel 2021, with disagreements resolved by consultation with a third reviewer. Extractions included authors, year of publication, corresponding author’s country, journal of publication, type of exposure, type of disease, study outcome, type of systematic reviews (narrative or meta-analysis), and the Impact Factors (IF) of the journals in Journal Citation Reports (JCR) in 2021.

Quality assessment

The methodological and reporting quality evaluation processes were conducted independently by two trained researchers, with disagreements discussed with a third researcher to reach a consensus.

The methodological quality of the included SRs was evaluated through the AMSTAR 2 tool, which was published in the British Medical Journal (BMJ) in 2017 to evaluate the methodological quality of SRs [13]. AMSTAR 2 consists of 16 items (with 7 critical domains: items 2, 4, 7, 9, 11, 13, 15), and each item was evaluated as “Yes”, “Partial Yes”, “No”, or “Not Applicable”. The quality level was rated as “High”, “Moderate”, “Low”, and “Critically Low” [13].

The reporting quality of included studies was evaluated using the PRISMA checklist, which was disseminated in 2009 in BMJ as a reporting guideline for SRs [22]. Since its publication, PRISMA has gained international endorsement. PRISMA contains 7 modules with 27 items: title, abstract, introduction, methods, results, discussion, and funding. Each of the items was assigned a value of 1 if it was “total compliance”, a value of 0.5 if it was “partial compliance”, and a value of 0 if it was “no compliance”. The values of each item were added up to provide a final score, with a total maximum score of 27. An SR with a score ≤ 15 was considered to have major flaws, 15.5–21 as having minor flaws, and ≥ 21.5 as having minimal flaws [23].

Data synthesis and analysis

Data were extracted, managed, and analyzed using Microsoft Excel 2021. The Chi-square test was performed using IBM SPSS Statistics 26.0.0.2 for differences in proportions of the evaluation results of each item (or Fisher’s exact test if a contingency table contained cells with five or fewer events).

Results

A total of 33,292 records were identified. Of the 405 SRs that met the inclusion criteria, 286 were for air pollution, 108 for temperature, and 11 for interaction between air pollution and temperature (Additional file 1. Fig. S1). No articles on the health effects of greenhouse gases were included.

Characteristics of included systematic reviews

Table 1 provides an overview of the key features of the included studies. (i) The studies were published between 2001 and 2023. More than half of them were published during the last 5 years (n = 250), suggesting that the major of them were proposed after the publication of AMSTAR 2 and PRISMA. (ii) Of the included studies, 68.4% were more likely to conduct meta-analyses, rather than qualitative systematic reviews. (iii). The average IF of the included 302 studies was 8.546 ± 1.480, whereas 17 papers [24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40] had no impact factor. SRs of air pollution were most frequently published in journals with an IF of 5 to 10 (44.1%), while SRs of temperature were mostly published in journals with an IF < 5 (43.5%). (iv) SRs of air pollution focused more on respiratory diseases (31.5%), cardiovascular diseases (28.7%), and all-cause mortality (18.2%), while SRs of temperature were more likely to study all-cause mortality (39.8%), infectious diseases (15.7%), and cardiovascular diseases (13.9%).

Table 1 The characteristics and results of the methodological and reporting quality of the included systematic reviews and meta-analyses
Full size table

Results of methodological quality

The methodological quality of the included studies was suboptimal, with only 0.7% of high quality. The main results of each item were as follows (Additional file 1. Table S2). (i) All the included studies were rated “Yes” for item 1 (PICO: populations, interventions, comparisons, and outcomes) and item 11 (appropriate methods). (ii) Of the included studies, 26.6% justified the study protocol and registration, 5.6% provided a list of the excluded studies and justified the exclusions, but only 3.5% explained their selection of study designs for inclusion in the review. (iii) Only 0.7% of studies reported the source of funding for the studies included in the review (The detailed results can be found in Additional file 1. Table S4).

Results of reporting quality

The reporting quality of the included studies needs further improvement, with only 45.7% having minimal flaws. The main results for each item were as follows (Additional file 1. Table S3). (i) All the included studies reported item 3 (rationale), and item 7 (information sources of methods). All the included studies reported item 2 (abstract), with only 19% of them reporting the structured summary completely and most of the studies reporting it partially. (ii) Only 26.4% of the included studies reported item 5 (protocols and registration). (iii) Less than 70% of the included studies reported on items 12, 15, 19, and 22 (55.6%, 57.3%, 47.9%, and 50.6%, respectively), which required authors to describe methods used and conclusions reached in the assessment of risk of bias within and across studies (The detailed results can be found in Additional file 1. Table S5).

Fig. 1
figure 1

The methodological and reporting quality of the included systematic reviews and Meta-analyses (A. AMSTAR 2 score; B. PRISMA score)

Full size image

The comparison of methodologic and reporting quality of included studies

We conducted a comparison of the methodological and reporting quality of the SRs for air pollution and temperature using the compliance rate for each item (Fig. 1). The SRs of air pollution had generally higher methodological quality than that of temperature, especially in item 5 (Study selection in duplicate) (57.7% v. 46.3%) (P < 0.05), item 6 (Data extraction in duplicate) (51.7% v. 35.2%) (P < 0.01), and item 14 (Satisfactory explanation for and discussion of any heterogeneity) (69.6% v. 45.4%) (P < 0.001). Furthermore, the methodological quality of air pollution SRs was superior than that of temperature SRs in item 9 and 13, but lower than that of temperature SRs in item 12 (P < 0.05). (The detailed results can be found in Additional file 1. Table S2 and Table S3).

The SRs of air pollution also had higher reporting quality than that of temperature, especially in terms of reporting the methods of the risk of bias in individual studies (item 12) (61.5% v. 40.7%) (P < 0.001) and reporting the results of the risk of bias within studies (item 19) (53.5% v. 34.3%) (P < 0.001). Additionally, the reporting quality of air pollution SRs was higher than that of temperature SRs in item 1, 8, 10, 11, 13, 14, 15, 16, 20, 21, 22, 23, and 25, but lower than that of temperature SRs in item 17 (P < 0.05). (The detailed results can be found in Additional file 1. Table S2 and Table S3).

Fig. 2
figure 2

Comparison of the methodological and reporting quality of the included systematic reviews

Full size image

When comparing the overall quality of methodology with that of reporting (Fig. 2), the results of AMSTAR 2 and PRISMA evaluations reached similar conclusions. The three SRs with high methodological quality had minimal flaws in reporting quality. Of the seven SRs with moderate methodological quality, six had minimal flaws in reporting quality and one had minor flaws. Of the 59 SRs with low methodological quality, 84.7% (n = 50) had minimal flaws, 11.9% (n = 7) had minor flaws, and 3.4% (n = 2) had major flaws in reporting quality. Of the 336 SRs with low methodological quality, 37.5% (n = 126) had minimal flaws, 32.4% (n = 109) had minor flaws, and 30.0% (n = 101) had major flaws in reporting quality. Correspondingly, the 103 SRs with major flaws in reporting quality were of low (n = 2, 1.9%) or very low (n = 101, 98.1%) methodological quality. Of the 117 SRs with minor flaws in reporting quality, 0.9% (n = 1) had moderate methodological quality, 6.0% (n = 7) had low methodological quality, and 93.2% (n = 109) had critically low methodological quality. Of the 185 SRs that reported minimal flaws, 1.6% (n = 3) had high methodological quality, 3.2% (n = 6) had moderate methodological quality, 27.0% (n = 50) had low methodological quality, and 68.1% (n = 126) had critically low methodological quality.

Discussion

SRs of the health impacts of ambient air pollution and extreme temperature can provide comprehensive and credible evidence for environmental policymaking and guidelines development [1]. With the increasing burden on human health resulting from air pollution and temperature change, the number of SRs in this field has greatly increased in recent years (61.7% in the last five years). However, this study shows that the methodological and reporting quality of SRs of air pollution and temperature health effects were below an acceptable level, highlighting an urgent need to improve the design and conduct of SRs.

Satisfactory explanation for any heterogeneity should be provided

The methodological quality determines the reliability of SRs and, therefore, determines whether the guidelines and policies developed from these SRs provide actionable recommendations [16, 41]. In this study, only 62.2% of the included studies provided a satisfactory explanation for and discussion of any heterogeneity observed in the results. Strict and uniform inclusion and exclusion criteria should be established to only include studies sufficiently homogeneous in terms of participants, interventions, and outcomes for a Meta-analysis [42]. Both air pollution and extreme temperature contribute to human health burdens, yet different kinds of air pollutants always coexist with varied temperatures [10]. Therefore, the sources of heterogeneity should be reasonably explained and their impact on the results should be thoroughly discussed [42]. In addition, subgroup analysis can be performed by dividing the studies into different subgroups according to their different characteristics such as gender and age to reduce heterogeneity. If the heterogeneity is too high to be resolved, meta-analysis can be abandoned [42].

Protocol registration in advance should be conducted

The reporting quality reflects the transparency of the SRs, which protocol registration can help to improve [19]. The present study shows that only 26.4% of the included studies reported protocol registration information, which aligns with the previous studies [16, 19]. The importance of protocol registration has led to the development of several platforms for it. The WHO clinical trial registration platform went online in 2008. The prospective systematic review registration project at the University of York, UK, was launched in 2011. Furthermore, the international practice guideline registration platform was launched in 2014. Cochrane SRs authors are required to register on the Cochrane Collaboration Network, whereas non-Cochrane SRs authors can register their protocols through the PROSPERO platform or publish their plans in journals [42]. Conducting a protocol and registering it in advance provide not only a clear path for researchers but also a reference for readers.

The risk of bias should be properly assessed

The cornerstones of SRs are their included studies, of which the risk of bias can affect the authenticity and the quality of SRs [41]. Reporting the assessment of the risk of bias within and across studies adds to the reliability of SRs [41]. We found the SRs of air pollution performed better than those of temperature in study selection, data extraction, and the assessment and interpretation of risk of bias (P < 0.05). The mainstream risk of bias assessment tool commonly used for SRs is the Cochrane Risk of Bias (RoB) [43], which is an essential tool for the quality evaluation of Randomized Controlled Trials (RCTs). In line with the previous study, we also found that there is no risk of bias tool applicable to air pollutants SRs, with some studies using The Newcastle-Ottawa Scale (NOS) and some using self-developed items for risk of bias assessment [20]. WHO Global Air Quality Guidelines Risk of Bias Assessment Working Group produced a risk of bias assessment tool for air quality and health epidemiology studies in 2020 [44], aiming to assess the risk of bias of long-term and short-term exposure to air pollution in cohort, case-control, time-series, case-crossover, and panel studies. Yet there is no risk of bias assessment tool specially designed for temperature research. Therefore, the development of a risk of bias assessment tool applicable to the temperature field is urgently needed.

Strengths and limitations

There are some strengths in our study. To our knowledge, this is the first attempt to compare the methodological and reporting quality of SRs for air pollution with those for temperature, and highlighted items with significant issues such as protocol registration and risk of bias assessment. Additionally, this study provided a complete and scientific evaluation of the present methodological and reporting quality of SRs on ambient air pollution and extreme temperature, which is of practical value for policy makers. Well-designed epidemiological studies and SRs are required to better understand the specific health impacts associated with ambient air pollution and extreme temperature.

Some limitations must also be acknowledged. We only included SRs published in English. However, we believe the results of our studies would have remained the same even if SRs in languages other than English had been consulted. Data extraction and quality assessment were performed following the data without further investigating the potential that the authors may have undertaken certain analyses but have not presented them in the paper. It is therefore possible that the methodological quality of the studies could be underestimated.

Recommendations for future research

In recent years, the climate-health link has been the target of a growing body of research and the focus of the general public and the health professionals. To guarantee the scientificity and transparency of the SRs, we believe that apart from following methodological specifications to develop SRs, future researchers should develop, register, or publish the study protocols on public platforms. Additionally, it is urgently needed that a risk-of-bias assessment tool be developed in the future to direct future research.

Conclusions

The methodological and reporting quality of SRs for air pollution were better than those for temperature. However, deficiencies in protocol registration and risk of bias assessment remain issues. Future reviewers should adhere to the methodological specifications and recognize the significance of pre-study protocol planning. Developing a risk-of-bias assessment tool applicable to the temperature field may also improve the quality of SRs in this field.

Data availability

All data generated or analyzed during this study are included in this published article and its supplementary information files.

Abbreviations

SR:

Systematic Review

CINAHL:

Cumulative Index to Nursing and Allied Health Literature

AMSTAR 2:

A Measurement Tool to Assess Systematic Reviews 2

PRISMA:

Preferred Reporting Items of Systematic reviews and Meta-Analyses

SO2 :

Sulfur Dioxide

CO:

Carbon Monoxide

NO2 :

Nitrogen Dioxide

PM:

Particulate Matter

BC:

Black Carbon

UFP:

Ultrafine Particles

O3 :

Ozone

WHO:

World Health Organization

GBD:

Global Burden of Disease

PM:

Particulate Matter

MOOSE:

Meta-Analysis of Observational Studies in Epidemiology

MeSH:

Medical Subject Headings

IF:

Impact Factors

JCR:

Journal Citation Reports

BMJ:

British Medical Journal

RoB:

Risk of Bias

NOS:

Newcastle-Ottawa Scale

RCT:

Randomized Controlled Trial

References

  1. Rocque RJ, Beaudoin C, Ndjaboue R, Cameron L, Poirier-Bergeron L, Poulin-Rheault R-A et al. Health effects of climate change: an overview of systematic reviews. Bmj Open 2021; 11(6).

  2. State of global air 2020. Available from: https://www.stateofglobalair.org/.

  3. Global air quality. guidelines.Particulate matter (PM2.5 and PM10),ozone,nitrogen dioxide,sulfur dioxide and carbon monoxide. Available from: https://apps.who.int/iris/bitstream/handle/10665/345329/9789240034228-eng.pdf.

  4. James SLG, Abate D, Abate KH, Abay SM, Abbafati C, Abbasi N, et al. Global, regional, and national incidence, prevalence, and years lived with disability for 354 Diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the global burden of Disease Study 2017. Lancet. 2018;392(10159):1789–858.

    Article  Google Scholar 

  5. Abbafati C, Abbas KM, Abbasi M, Abbasifard M, Abbasi-Kangevari M, Abbastabar H, et al. Global burden of 369 Diseases and injuries in 204 countries and territories, 1990–2019: a systematic analysis for the global burden of Disease Study 2019. Lancet. 2020;396(10258):1204–22.

    Article  Google Scholar 

  6. Phung D, Thai PK, Guo Y, Morawska L, Rutherford S, Chu C. Ambient temperature and risk of cardiovascular hospitalization: an updated systematic review and meta-analysis. Sci Total Environ. 2016;550:1084–102.

    Article  CAS  PubMed  Google Scholar 

  7. Romanello M, Di Napoli C, Drummond P, Green C, Kennard H, Lampard P, et al. The 2022 report of the Lancet countdown on health and climate change: health at the mercy of fossil fuels. Lancet (London England). 2022;400(10363):1619–54.

    Article  PubMed  Google Scholar 

  8. Weilnhammer V, Schmid J, Mittermeier I, Schreiber F, Jiang L, Pastuhovic V, et al. Extreme weather events in europe and their health consequences - a systematic review. Int J Hyg Environ Health. 2021;233:113688.

    Article  PubMed  Google Scholar 

  9. Chen F, Fan Z, Qiao Z, Cui Y, Zhang M, Zhao X, et al. Does temperature modify the effect of PM10 on mortality? A systematic review and meta-analysis. Environ Pollut. 2017;224:326–35.

    Article  CAS  PubMed  Google Scholar 

  10. Li J, Woodward A, Hou XY, Zhu T, Zhang J, Brown H, et al. Modification of the effects of air pollutants on mortality by temperature: a systematic review and meta-analysis. Sci Total Environ. 2017;575:1556–70.

    Article  CAS  PubMed  Google Scholar 

  11. Song X, Jiang L, Wang S, Tian J, Yang K, Wang X, et al. The impact of main air pollutants on respiratory emergency department visits and the modification effects of temperature in Beijing, China. Environ Sci Pollut Res. 2021;28(6):6990–7000.

    Article  CAS  Google Scholar 

  12. Salameh J-P, Bossuyt PM, McGrath TA, Thombs BD, Hyde CJ, Macaskill P et al. Preferred reporting items for systematic review and meta-analysis of diagnostic test accuracy studies (PRISMA-DTA): explanation, elaboration, and checklist. Bmj-British Med J 2020; 370.

  13. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008.

    Article  PubMed  PubMed Central  Google Scholar 

  14. West S, King V, Carey TS, Lohr KN, McKoy N, Sutton SF et al. Systems to rate the strength of scientific evidence. Evid Rep Technol Assess (Summ) 2002(47):1–11.

  15. Shaffril HAM, Samah AA, Samsuddin SF. Guidelines for developing a systematic literature review for studies related to climate change adaptation. Environ Sci Pollut Res. 2021;28(18):22265–77.

    Article  Google Scholar 

  16. Tian J, Zhang J, Ge L, Yang K, Song F. The methodological and reporting quality of systematic reviews from China and the USA are similar. J Clin Epidemiol. 2017;85:50–8.

    Article  PubMed  Google Scholar 

  17. Cao L, Yao L, Hui X, Li J, Zhang X, Li M et al. ,. Clinical Epidemiology in China series. Paper 3: The methodological and reporting quality of systematic reviews and meta-analyses published by China’ researchers in English-language is higher than those published in Chinese-language. J Clin Epidemiol. 2021; 140:178–188

  18. Gao Y, Cai Y, Yang K, Liu M, Shi S, Chen J, et al. Methodological and reporting quality in non-cochrane systematic review updates could be improved: a comparative study. J Clin Epidemiol. 2020;119:36–46.

    Article  PubMed  Google Scholar 

  19. Page MJ, Moher D. Evaluations of the uptake and impact of the Preferred reporting items for systematic reviews and Meta-analyses (PRISMA) Statement and extensions: a scoping review. Syst Reviews 2017; 6.

  20. Sheehan MC, Lam J, Navas-Acien A, Chang HH. Ambient air pollution epidemiology systematic review and meta-analysis: a review of reporting and methods practice. Environ Int. 2016;92–93:647–56.

    Article  PubMed  Google Scholar 

  21. Sheehan MC, Lam J. Use of systematic review and Meta-analysis in Environmental Health Epidemiology: a systematic review and comparison with guidelines. Curr Environ Health Rep. 2015;2(3):272–83.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Li JL, Ge L, Ma JC, Zeng QL, Yao L, An N et al. Quality of reporting of systematic reviews published in evidence-based Chinese journals. Syst Reviews 2014; 3(58).

  24. Park HY, Bae S, Hong YC. PM10 exposure and non-accidental mortality in Asian populations: a meta-analysis of time-series and case-crossover studies. J Prev Med Public Health. 2013;46(1):10–8.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Yu P, Guo S, Xu R, Ye T, Li S, Sim MR, et al. Cohort studies of long-term exposure to outdoor particulate matter and risks of cancer: a systematic review and meta-analysis. Innov (Camb). 2021;2(3):100143.

    CAS  Google Scholar 

  26. Sun HZ, Yu P, Lan C, Wan MWL, Hickman S, Murulitharan J, et al. Cohort-based long-term ozone exposure-associated mortality risks with adjusted metrics: a systematic review and meta-analysis. Innov (Camb). 2022;3(3):100246.

    CAS  Google Scholar 

  27. Hua W, Li X, Chu X, Cao N, Wu H, Huang R et al. ,. Ambient air pollutants increase the risk of Immunoglobulin E-mediated allergic diseases: a systematic review and meta-analysis. 2021

  28. Ghanizadeh G, Heidari M, Seifi B, Jafari H, Pakjouei S. The Effect of Climate Change on Cardiopulmonary Disease - A systematic review. J Clin Diagn Res. 2017.

  29. Coates SJ, Davis MDP, Andersen LK. Temperature and humidity affect the incidence of hand, foot, and mouth Disease: a systematic review of the literature - a report from the International Society of Dermatology Climate Change Committee. Int J Dermatol. 2019;58(4):388–99.

    Article  PubMed  Google Scholar 

  30. Anita AR, Tan HS, Fatimah AF, Juni EN. PUBLIC HEALTH IMPACTS OF HEAT WAVES: A REVIEW. Int J Public Health Clin Sci. 2018;5(2):68–85.

    Google Scholar 

  31. VianaI DV, Ignotti E. The ocurrence of dengue and weather changes in Brazil: a systematic review. Rev Bras Epidemiol. 2013;16(2):240–56.

    Google Scholar 

  32. Kakaei S, Zakerimoghadam M, Rahmanian M, Abbasi Dolatabadi Z. The impact of Climate Change on Heart Failure: a narrative review study. Shiraz E-Medical Journal. 2021;22(9):e107895.

    Article  Google Scholar 

  33. Perry T, Obolski U, Peretz C. The association between high ambient temperature and mortality in the Mediterranean basin: a systematic review and meta-analysis. medRxiv 2022:01.20.22269580.

  34. Husaini DC, Reneau K, Balam D. Air pollution and public health in Latin America and the Caribbean (LAC): a systematic review with meta-analysis. Beni-Suef Univ J Basic Appl Sci 2022; 11(1).

  35. Ohlwein S, Kappeler R, Kutlar Joss M, Künzli N, Hoffmann B. Health effects of ultrafine particles: a systematic literature review update of epidemiological evidence. Int J Public Health. 2019;64(4):547–59.

    Article  PubMed  Google Scholar 

  36. Bouchriti Y, Haddou MA, Kabbachi B, Achbani A, Cherrat Z, Rida J, et al. Ambient air quality and health impact of exposure to Outdoor Air Pollution in the Moroccan Population: a systematic review. Pollution. 2023;9(2):660–77.

    CAS  Google Scholar 

  37. Lee W-S, Kim W-S, Lim Y-H, Hong Y-C. High temperatures and Kidney Disease morbidity: a systematic review and Meta-analysis. J Prev Med Public Health = Yebang Uihakhoe Chi. 2019;52(1):1–13.

    Article  CAS  PubMed  Google Scholar 

  38. Gao D, Friedman S, Hosler A, Sheridan S, Zhang W, Lin S. Association between extreme ambient heat exposure and diabetes-related hospital admissions and emergency department visits: a systematic review. Hyg Environ Healh Adv 2022; 4.

  39. Pan R, Xie M, Chen M, Zhang Y, Ma J, Zhou J. The impact of heat waves on the mortality of Chinese population: a systematic review and meta-analysis. Med (Baltim). 2023;102(13):e33345.

    Article  Google Scholar 

  40. Juarez PD, Ramesh A, Hood DB, Alcendor DJ, Valdez RB, Aramandla MP, et al. The effects of air pollution, meteorological parameters, and climate change on COVID-19 comorbidity and health disparities: a systematic review. Environ Chem Ecotoxicol. 2022;4:194–210.

    Article  CAS  Google Scholar 

  41. Murad MH, Montori VM, Ioannidis JPA, Jaeschke R, Devereaux PJ, Prasad K, et al. How to read a systematic review and Meta-analysis and apply the results to patient care users’ guides to the Medical Literature. Jama-Journal of the American Medical Association. 2014;312(2):171–9.

    Article  CAS  Google Scholar 

  42. Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated. February 2022) Available from: www.training.cochrane.org/handbook.

  43. Higgins JPT, Sterne JAC, Savovic J, Page MJ, Hrobjartsson A, Boutron I, et al. A revised tool for assessing risk of bias in randomized trials. Cochrane Database of Systematic Reviews. 2016;10:29–31.

    Google Scholar 

  44. Risk of bias assessment instrument for systematic reviews informing WHO global air quality guidelines. Available from: https://www.euro.who.int/en/health-topics/environment-and-health/air-quality/publications/2020/risk-of-bias-assessment-instrument-for-systematic-reviews-informing-who-global-air-quality-guidelines-2020.

Download references

Acknowledgements

Not Applicable.

Funding

This work was supported by the Gansu Province Science and Technology Planning Project (23YFWA0004) and International Course at Lanzhou University (KCJS202311).

Author information

Authors and Affiliations

  1. Evidence-Based Social Science Research Center, Department of Social Medicine and Health Management, School of Public Health, Lanzhou University, Lanzhou, China

    Xuping Song, Qiyin Luo, Yan Ma, Yue Hu, Yunze Han, Rui Wang, Jing Tang, Yiting Guo, Qitao Zhang, Zhongyu Ma, Yunqi Zhang, Xinye Guo, Shumei Fan, Chengcheng Deng, Yaolong Chen & Long Ge

  2. Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, China

    Kehu Yang

  3. Key Laboratory of Evidence Based Medicine & Knowledge Translation of Gansu Province, Lanzhou, China

    Xuping Song, Yaolong Chen, Kehu Yang & Long Ge

  4. College of Atmospheric Sciences, Chengdu University of Information Technology, Chengdu, Sichuan, China

    Shigong Wang

  5. The Fifth Hospital of Xi’an, Xi’an, China

    Liangzhen Jiang

  6. School of Stomatology, Lanzhou University, Lanzhou, China

    Xinyu Fu

  7. Institute of Health Data Science, Lanzhou University, Lanzhou, China

    Xuping Song, Yaolong Chen, Kehu Yang & Long Ge

  8. WHO Collaborating Centre for Guideline Implementation and Knowledge Translation, Lanzhou, China

    Xuping Song, Yaolong Chen, Kehu Yang & Long Ge

  9. McMaster Health Forum, Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, L8S4L8, Canada

    Xuping Song

Authors
  1. Xuping Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qiyin Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liangzhen Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yan Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yue Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yunze Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Rui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jing Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Yiting Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Qitao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Zhongyu Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Yunqi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Xinye Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Shumei Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Chengcheng Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Xinyu Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Yaolong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Kehu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Long Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Shigong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

XS and QL designed the study, analysed the data, and drafted the manuscript. LJ, YC and KY developed the research design. YM, YH, YzH, RW, and XF drafted manuscript. JT,YG, QZ, ZM, YZ, XG, SF, and CD were in charge of data curation and evaluated the quality of included studies. LG and SW critically reviewed and revised the manuscript. All authors critically revised the article for important intellectual content and approved the final version of the manuscript.

Corresponding authors

Correspondence to Long Ge or Shigong Wang.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary Material 2

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Song, X., Luo, Q., Jiang, L. et al. Methodological and reporting quality of systematic reviews on health effects of air pollutants were higher than extreme temperatures: a comparative study. BMC Public Health 23, 2371 (2023). https://doi.org/10.1186/s12889-023-17256-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12889-023-17256-5

Keywords

BMC Public Health

ISSN: 1471-2458

Contact us