- Research in practice
- Open access
- Published:
A roadmap for using DHIS2 data to track progress in key health indicators in the Global South: experience from sub-saharan Africa
BMC Public Health volume 23, Article number: 1030 (2023)
Abstract
High quality health data as collected by health management information systems (HMIS) is an important building block of national health systems. District Health Information System 2 (DHIS2) software is an innovation in data management and monitoring for strengthening HMIS that has been widely implemented in low and middle-income countries in the last decade. However, analysts and decision-makers still face significant challenges in fully utilizing the capabilities of DHIS2 data to pursue national and international health agendas. We aimed to (i) identify the most relevant health indicators captured by DHIS2 for tracking progress towards the Sustainable Development goals in sub-Saharan African countries and (ii) present a clear roadmap for improving DHIS2 data quality and consistency, with a special focus on immediately actionable solutions. We identified that key indicators in child and maternal health (e.g. vaccine coverage, maternal deaths) are currently being tracked in the DHIS2 of most countries, while other indicators (e.g. HIV/AIDS) would benefit from streamlining the number of indicators collected and standardizing case definitions. Common data issues included unreliable denominators for calculation of incidence, differences in reporting among health facilities, and programmatic differences in data quality. We proposed solutions for many common data pitfalls at the analysis level, including standardized data cleaning pipelines, k-means clustering to identify high performing health facilities in terms of data quality, and imputation methods. While we focus on immediately actionable solutions for DHIS2 analysts, improvements at the point of data collection are the most rigorous. By investing in improving data quality and monitoring, countries can leverage the current global attention on health data to strengthen HMIS and progress towards national and international health priorities.
Background
High quality health data, as collected by health management information systems (HMIS), are a key component of planning for utilization of health services, prevention and vaccination campaigns, and even evaluating national health programs [1, 2]. This pressing need for timely and high quality data has become abundantly clear during the coronavirus (COVID-19) pandemic [3]. Investment in HMIS has often languished, however, particularly in low and middle-income countries (LMICs), hobbling the ability of policymakers and governments to use their own data to inform decision-making and increasing reliance on often inaccurate or incomplete estimates of local health needs from international bodies [4, 5]. The need for these data became particularly pressing after the introduction of the United Nations’ 2030 Agenda for Sustainable Development and its associated Sustainable Development Goals (SDGs), which meant that tracking progress towards health and other development goals became more important than ever [5, 6].
District Health Information System 2 (DHIS2) software is a new solution to the siloed data collection and program-dominated reporting that have characterized HMIS in LMICs, as we have previously highlighted in the WHO Bulletin [7]. DHIS2 was developed by the Health Information Systems Programme, with support from the Norwegian Agency for Development Cooperation, the United States President’s Emergency Plan for AIDS Relief, the Global Fund to Fight AIDS, Tuberculosis and Malaria, the United Nations Children’s Fund, and the University of Oslo [8]. The open-source platform includes data validation, visualization and analysis tools, readily allowing for access and manipulation of health data at central and local levels [8]. In particular, the use of electronic forms enables data collection with built-in quality control measures. Since 2011, it has become the most popular HMIS platform, used in over 70 LMICs [8]. Several countries have reported improved data completeness and timeliness after implementing DHIS2 [9,10,11,12,13]. However, despite the important innovation that the platform offers, we showed in our previous viewpoint that thus far DHIS2 data have been underrepresented in the scientific literature [7]. Now, over a decade after the initial development of the DHIS2 software, and during a period of intense focus on pandemic health statistics reporting, is a key time to prioritize timely and high quality collection of the most important health indicators and promote more widespread scientific use of DHIS2 data.
As part of a larger research project called the Health Impact Assessment for Sustainable Development (HIA4SD), we aimed to access and analyze DHIS2 data in four countries in sub-Saharan Africa: Burkina Faso, Ghana, Mozambique, and Tanzania [14]. As we began working with the wider community of researchers and decision-makers using DHIS2, we realized that while much excellent work is being done with DHIS2 data, analysts are encountering similar barriers across many countries. Based on our real-world experiences using DHIS2 data to track key health indicators in and around large infrastructure projects, and conversations with experts in DHIS2 data, we propose solutions to these barriers. Our objectives are twofold: first, to identify the important health indicators captured by DHIS2 that are relevant for tracking progress towards the SDGs in sub-Saharan African countries; and second, to present a clear roadmap for improving DHIS2 data quality and consistency. While we do discuss systemic issues to be solved, our aim is most especially to provide analysts currently using DHIS2 data with immediately actionable solutions for the most prevalent data issues, allowing analysts to produce sound analyses using existing data for use by national-level decision-makers or publication in the scientific literature.
Methods
As part of a larger research for development project (HIA4SD project; https://hia4sd.net) [14], we partnered with local health institutes in Burkina Faso, Ghana, Mozambique, and Tanzania. These institutes usually already had established contacts with the local Ministries of Health (MoH). In some places, our collaborators already had direct access to the national DHIS2 software and data. In others, a data sharing agreement for DHIS2 was established in the framework of the project and MoH analysts extracted the data and shared it with us in aggregate form, usually at the monthly and health facility level where possible. In all countries, we interacted almost exclusively with the central DHIS2 analysts at the MoH or our partner institutes. In addition, we reached out more informally to a larger network of DHIS2 analysts in many more Sub-Saharan African countries through the Swiss Tropical and Public Health Institute in order to expand our knowledge and understanding of the DHIS2 application. A Slack channel was established to promote discussion on these topics. After a period of informal discussion and synthesis, a comprehensive document encompassing the identified problems and solutions was created by the authors.
What are the most important indicators that can realistically be captured by DHIS2 to harmonize across countries?
Each country has its own implementation of DHIS2 software [8], reflecting its own needs and priorities. However, we found some commonalities across all four study countries. Figure 1 presents a list of the health-related SDG indicators and our evaluation of whether they can be adequately captured by the DHIS2 system as it currently stands. In particular, maternal and child health outcomes are captured and reported almost universally, including maternal and child mortality in health facilities, infectious disease, antenatal care, and vaccination coverage. While these indicators are often standard across countries, the data quality varies by indicator and by country, and often even by health facility, as discussed further in the subsequent sections.
Other indicators, such as positive HIV cases, are widely captured and reported, but in a variety of different ways that inhibit comparison across time or countries. In fact, the huge variety of indicators around HIV testing makes it difficult to identify which variable to use as the definitive indicator for HIV case counts (example from one country of the seemingly similar indicators available: cases of HIV/AIDS, patients HIV positive, HIV positive test result). It is unclear whether each of these indicators refers to incident or prevalent cases, for example. Careful documentation of definitions and identification of which indicators are most important for national priorities should be readily available.
Almost universally, DHIS2 implementation would benefit greatly from standardized case definitions of the most important indicators for analysis, whether to track progress towards the SDGs, or to ensure that the specific programmatic concerns of each country are adequately tracked. This will require close collaboration between national policy makers at the MoH and governmental agencies, the DHIS2 implementation team, and staff from key national health programs. The proliferation of data indicators over time is a well-known problem, and regular database maintenance and elimination of unused indicators (after archiving any remaining data) should take place. Agreement on these issues will allow local data collectors to focus their time and resources on uploading the most important indicators at each site.
In general, certain key health indicators are not suitable to be captured in DHIS2 in its current form. In particular, mortality indicators are among the most difficult and inaccurate indicators in routine HMIS. Promising work in the global South has been done with conducting verbal autopsy to supplement the limited data available from HMIS [15]. In addition, data on health systems (e.g. number of health workers, capacity, services) has not often been captured within DHIS2, although these could also be a useful national metric for health. Instead, most countries rely on periodic implementations of the Service Availability and Readiness Assessment (SARA), a health facility assessment tool developed by the WHO. Better integration of these existing data collection systems with data from DHIS2 could be a powerful way to better track progress in health systems without overloading the HMIS.
What are some actionable solutions for achieving better data quality and quantity in key health indicators?
As part of our study, we also became familiar with the process by which health data are collected, entered, and uploaded into the DHIS2 system, which was remarkably similar across countries. Most case reports are initially done on paper by local health staff at the facility, and then at the end of each month these case reports are collated into summary reports and entered into the software. Most countries have procedures for ensuring accuracy by comparing these summary reports with physical entries in the register books and what is entered into the DHIS2 software, often by trained staff that travel between health facilities. These procedures occur anywhere from monthly to once or twice per year, depending on the resources available. Fully digital data collection has traditionally not been seen as possible in many settings due to limited resources such as personnel, information technology (IT) infrastructure, internet connectivity and stable electricity, although case studies such as in Mali have shown that distribution of tablets for data collection can further improve this process [16]. While there are data quality assurance tools available within DHIS2 to check the accuracy of data entry, the degree to which these are used by local data entry staff is often limited. This similarity of data collection and processing across countries due to the similar structure of the health systems has the added advantage that identified solutions are likely to work in many settings.
DHIS2 data quality and quantity can be improved during four main time periods (Table 1). The first is to improve data collection in health facilities through more complete capture and recording of case reports on paper registers. The second is to improve the monthly tallying process during data aggregation. The third is to improve quality control during the data entry and upload process. The fourth is to correct for data shortcomings post-hoc, usually through statistical analysis techniques (e.g. imputing missing data). The periodic data quality assurance procedures carried out in most countries on a quarterly or annual basis at the health facilities represent an important opportunity to implement many of the mitigation activities suggested.
The most rigorous data improvement interventions occur on the level of data collection and entry; hence, we propose solutions to improve data collection processes ranging from digital staff training platforms to automated data reporting and quality control to performance-based funding (Table 1). In particular, regular trainings of data collectors and routine definition and documentation of key variables on the national level by each respective MoH would yield significant improvements in data quality and completeness. It is especially key to train and support the staff who conduct the monthly validation of DHIS2 data entry. Giving regular feedback and access to DHIS2 analysis tools to facility-level managers may also incentivize health facilities to optimize their own data collection processes. The World Health Organization (WHO) has collaborated with DHIS2 to produce standardized digital health toolkits [17], which could play a major role in developing and disseminating data standards across countries. Understanding and implementation of these toolkits should be of high priority for further improving data quality, especially for indicators like HIV where data collection remains unstandardized among countries. However, these solutions are also inevitably the most difficult and time and resource intensive. Therefore, we also propose solutions at the analysis level, with the goal of enabling analysts to utilize DHIS2 data in its current form.
Some problems, such as being able to differentiate between zero cases and missing data, seem to have a relatively straightforward technical fix. This issue was reported as a key frustration for many analysts working with DHIS2 data, and especially limits the ability of analysts working with national level data to use imputation methods to correct for missing data. Other problems, such as the large differences in data quality between different indicators, are often a result of differences in how data are collected by different programs and priorities of countries. The analyst can work around these differences by working with experts in the local health systems to identify and correctly define the appropriate indicators to use. One key takeaway of our research project is that strong partnerships with local data experts are absolutely crucial to utilizing the full potential of DHIS2 data.
Conclusions
Our experience working with DHIS2 data as part of the HIA4SD project indicates that many key health indicators (e.g. child and maternal health indicators) are already well captured by the platform; other indicators, such as HIV/AIDS incidence and prevalence, would benefit from more standardized case definitions and streamlining the number of indicators collected. While variations in facility level reporting, availability of denominator data, and differences in quality between different indicators remain a systemic problem, we have identified the above workarounds for these problems that should be shared more widely with the entire constellation of DHIS2 analysts and data users.
By focusing on improving a more limited number of indicators and resolving known data quality issues, countries utilizing DHIS2 software can dramatically improve their ability to monitor and evaluate progress towards national and international health targets. The global COVID-19 crisis has created a particular incentive and opportunity for LMICs to invest in their HMIS, potentially creating a lasting positive impact on local health capacities, ability to implement and evaluate new health programmes, and real-time monitoring of emergent health conditions. Some countries have already seen the benefits of these investments. During the pandemic, Côte d’Ivoire used its local implementation of DHIS2 to track COVID-19 rumors in real-time [27] and Bangladesh extended pandemic surveillance to collect cancer screening data [28]. These case studies should serve as examples of how LMICs can leverage the flexibility of DHIS2 software to advance their own priorities for their health systems. Small investments in DHIS2 training now could have immediate payoff for resolving the known data quality issues; for example, there is a widely available tool for data quality assurance available for DHIS2 [29] that appears to be underutilized at the country level [23]. The digital health toolkits that the WHO has developed in collaboration with DHIS2 teams should be utilized by countries to maximize data quality at all time periods and standardize data collection and reporting across countries, especially for key indicators such as HIV. A robust HMIS is an essential part of a strong health system and a key part of supporting evidence-based policy and decision-making, and should be both a national and international priority.
On the analyst level, there is no need to continually “reinvent the wheel” in terms of the data approaches to solve known data issues; instead, we propose that this perspective serve as a systematic catalogue of the data techniques that have been used thus far to improve the scientific quality of DHIS2 analyses. Our experience suggests that in-country analysts who are able to contextualize programmatic and technical changes that may affect data analysis and interpretation are indispensable to being able to use DHIS2 data, and should be recognized as such. Ideally, those tasked with analyzing and interpreting DHIS2 data should have access to a standardized list of data solutions such as we propose here, and an international network of analysts knowledgeable about how to work with DHIS2 data. Investment in the human resources around HMIS is no less important than the technical abilities that DHIS2 offers.
Several ideological challenges around using HMIS data to track progress towards the SDGs also remain, and are important to consider. First of all, there is still not a strong culture around data use by those developing, maintaining, and analyzing national level HMIS. Using HMIS data to generate evidence is largely restricted to minimal analysis of a large set of indicators, leading to a surface level understanding of the results. Related to this, most of the users routinely using DHIS2 to generate reports and analyses have minimal analytical skills and little incentive to delve deeply into the complexities of the DHIS2 database to answer more sophisticated questions. The HIA4SD study offers one template for how academic institutes can partner directly with MoH to use DHIS2 data to produce complex analyses and scientific publications. In terms of capacity building, we found partnerships to be particularly effective when the contact person at the MoH was partnered with the academic institute already during their training (e.g. when completing a Masters in Public Health). These types of partnerships have enormous potential in the longer-term to strengthen the culture around data use and analysis using national HMIS systems.
The international rollout of DHIS2 software over the past decade has offered clear opportunities for countries to own their data and lead improvements to national level HMIS. Nevertheless, the use of these data in the academic literature and by policy-makers has lagged. We believe that there is no time like the present to invest further in DHIS2 and bring HMIS in LMICs to the next level. Some fixes will require national level coordination, such as implementing more robust data quality measures and identifying and harmonizing indicators across the platform (e.g. through a core indicator classification system), while remaining sensitive to the needs of individual countries. Other improvements can be implemented at the analyst level, by standardizing data approaches and cleaning techniques across countries. Up-to-date and relevant HMIS data are clearly a current global priority, and countries using DHIS2 software can benefit today and in the future by maximizing its potential.
Availability of data and materials
Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.
Abbreviations
- HMIS:
-
Health Management Information Systems
- LMIC:
-
Low and middle-income countries
- MoH:
-
Ministry of Health
- SDG:
-
Sustainable Development Goal
- DHIS2:
-
District Health Information System 2
- HIA4SD:
-
Health Impact Assessment for Sustainable Development
References
AbouZahr C, Boerma T. Health information systems: the foundations of public health. Bull World Health Organ. 2005;83(8):578–83. Epub 2005/09/27.
Wagenaar BH, Sherr K, Fernandes Q, Wagenaar AC. Using routine health information systems for well-designed health evaluations in low- and middle-income countries. Health Policy Plann. 2015;31(1):129–35.
Amouzou A, Faye C, Wyss K, Boerma T. Strengthening routine health information systems for analysis and data use: a tipping point. BMC Health Serv Res. 2021;21(1):618.
Maïga A, Jiwani SS, Mutua MK, Porth TA, Taylor CM, Asiki G, et al. Generating statistics from health facility data: the state of routine health information systems in Eastern and Southern Africa. 2019;4(5):e001849.
Boerma T, Victora C, Abouzahr C. Monitoring country progress and achievements by making global predictions: is the tail wagging the dog? Lancet. 2018;392(10147):607–9.
Victora CG, Black RE, Boerma JT, Bryce J. Measuring impact in the millennium development goal era and beyond: a new approach to large-scale effectiveness evaluations. Lancet. 2011;377(9759):85–95.
Farnham A, Utzinger J, Kulinkina AV, Winkler MS. Using district health information to monitor sustainable development. Bull World Health Organ. 2020;98(1):69–71. Epub 2019/11/29.
DHIS2. DHIS2 in action. University of Oslo. ; 2021 [cited 2021 15 September ]; Available from: https://dhis2.org/in-action/#map.
Kiberu VM, Matovu JK, Makumbi F, Kyozira C, Mukooyo E, Wanyenze RK. Strengthening district-based health reporting through the district health management information software system: the ugandan experience. BMC Med Inform Decis Mak. 2014;14:40. Epub 2014/06/03.
Karuri J, Waiganjo P, Orwa D, Manya A. DHIS2: the Tool to improve Health Data demand and use in Kenya. J Health Inform Dev Ctries. 2014;8(1). Retrieved from https://www.jhidc.org/index.php/jhidc/article/view/113.
Githinji S, Oyando R, Malinga J, Ejersa W, Soti D, Rono J, et al. Completeness of malaria indicator data reporting via the District Health Information Software 2 in Kenya, 2011–2015. Malar J. 2017;16(1):344.
Shuaib F, Garba AB, Meribole E, Obasi S, Sule A, Nnadi C, et al. Implementing the routine immunisation data module and dashboard of DHIS2 in Nigeria, 2014–2019. BMJ global health. 2020;5(7):e002203.
Thawer SG, Chacky F, Runge M, Reaves E, Mandike R, Lazaro S, et al. Sub-national stratification of malaria risk in mainland Tanzania: a simplified assembly of survey and routine data. Malar J. 2020;19(1):177.
Farnham A, Cossa H, Dietler D, Engebretsen R, Leuenberger A, Lyatuu I, et al. Investigating Health Impacts of natural resource extraction projects in Burkina Faso, Ghana, Mozambique, and Tanzania: protocol for a mixed methods study. JMIR Res protocols. 2020;9(4):e17138–e.
de Savigny D, Riley I, Chandramohan D, Odhiambo F, Nichols E, Notzon S, et al. Integrating community-based verbal autopsy into civil registration and vital statistics (CRVS): system-level considerations. Global Health Action. 2017;10(1):1272882.
Kirk K, McClair TL, Dakouo SP, Abuya T, Sripad P. Introduction of digital reporting platform to integrate community-level data into health information systems is feasible and acceptable among various community health stakeholders: a mixed-methods pilot study in Mopti, Mali. J Glob Health. 2021;11:07003. Epub 2021/04/02.
DHIS2. WHO Health Data Toolkit. [cited 2023 14 April]; Available from: https://dhis2.org/who/.
Gesicho MB, Babic A, Were MC. Health Facility ownership type and performance on HIV Indicator Data reporting in Kenya. Stud Health Technol Inform. 2020;270:1301–2. Epub 2020/06/24.
Sato R, Belel A. Effect of performance-based financing on health service delivery: a case study from Adamawa state. Nigeria Int Health. 2020;13(2):122–9.
Chrysantina A, Sanjaya G, Pinard M, Hanifah Nm. Improving Health Information Management Capacity with Digital Learning platform: the case of DHIS2 Online Academy. Procedia Comput Sci. 2019;161:195–203.
Bhattacharya AA, Allen E, Umar N, Audu A, Felix H,Schellenberg J, et al. Improving the quality of routine maternal and newborn data captured in primary health facilities in Gombe State, northeastern Nigeria: a before-and-after study. 2020;10(12):e038174.
Gesicho MB, Babic A, Were MC. K-Means clustering in Monitoring Facility Reporting of HIV Indicator Data: case of Kenya. Stud Health Technol Inform. 2020;272:143–6. Epub 2020/07/02.
Gesicho MB, Were MC, Babic A. Data cleaning process for HIV-indicator data extracted from DHIS2 national reporting system: a case study of Kenya. BMC Med Inf Decis Mak. 2020;20(1):293.
Ssempiira J, Kissa J, Nambuusi B, Kyozira C, Rutazaana D, Mukooyo E, et al. The effect of case management and vector-control interventions on space-time patterns of malaria incidence in Uganda. Malar J. 2018;17(1):162.
Buzzelli M. Modifiable Areal Unit Problem. Int Encycl Hum Geogr. 2020:169–73. https://doi.org/10.1016/B978-0-08-102295-5.10406-8. Epub 2019 Dec 4.
Tuson M, Yap M, Kok MR, Boruff B, Murray K, Vickery A, et al. Overcoming inefficiencies arising due to the impact of the modifiable areal unit problem on single-aggregation disease maps. Int J Health Geogr. 2020;19(1):40.
Tibbels N, Dosso A, Allen-Valley A, Benie W, Fordham C, Brou JA, et al. Real-Time Tracking of COVID-19 rumors using community-based methods in Côte. d’Ivoire. 2021;9(2):355–64.
Basu P, Lucas E, Zhang L, Muwonge R, Murillo R, Nessa A. Leveraging vertical COVID-19 investments to improve monitoring of cancer screening programme – a case study from Bangladesh. Prev Med. 2021;151:106624.
Haugen J, Hjemås G, Poppe O. Manual for the DHIS2 quality tool. Understanding the basics of improving data quality. 2017. https://docs.dhis2.org/pt/full/use/optional-apps/who-data-quality-toolmanual.html.
Acknowledgements
We would like to acknowledge the invaluable contributions of the editor and peer reviewers in revising this piece for publication.
Funding
Open access funding provided by University of Basel This work part of the r4d programme (www.r4d.ch), which is a joint funding initiative by the Swiss Agency for Development and Cooperation (SDC) and the Swiss National Science Foundation (SNSF) [grant number 194003]. The funding bodies had no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.
Author information
Authors and Affiliations
Contributions
AF drafted the paper, with major inputs from GL. All authors participated in drafting and revising the manuscript and approved the final version before submission.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Farnham, A., Loss, G., Lyatuu, I. et al. A roadmap for using DHIS2 data to track progress in key health indicators in the Global South: experience from sub-saharan Africa. BMC Public Health 23, 1030 (2023). https://doi.org/10.1186/s12889-023-15979-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12889-023-15979-z