Data for decision-making is a common cry in public arenas. However, not all data are the same, and the implications of using the various alternative data sources available can be significant, especially when multiple choices exist, such as in the case of Palestine. Selection of data source can be even more critical when used as inputs into a formal analytic framework, as many policy makers do not see the raw data but only the results of the processing, assumptions, estimates, and analysis. Global agencies and research teams publish consensus estimates of mortality with uncertainty bounds, but estimates of health intervention coverage show more variability and are less widely available in general. The availability of new sources of local and timely data and indicators is likely to increase as countries shift towards digital data and case-based collection methods. The evaluation of these new data sources is critical to assess their potential for improving the care being delivered and to appropriately inform planning processes.
In this analysis, the four data sources yielded notably different results when utilized in LiST. The maternal deaths averted ranged from 5 to 39, or a reduction of maternal mortality from 1 to 6%. At the same time, the composition of interventions to save these lives varied from 100% for pre-eclampsia management to 75% for hypertensive disorders management. These differences would likely result in different policy and practice decisions being taken. Similar, but less dramatic differences could be seen in newborn, stillbirth, and anemia results using the different data sources. Although the absolute differences were relatively small in this particular context, they would be magnified greatly in countries and settings with higher mortality and morbidity rates, or if interventions beyond antenatal care were included.
The power of the Lives Saved Tool can be maximized when data of better quality and quantity are available to populate it. However, in most country settings, several data points are not directly available in either routine reports or household surveys. Drawing data directly from clinical records allows for a more complete and complex picture of antenatal care and covers almost all data needs. Many surveys, such as the MICS, only include data from live-births [17], thus excluding data on women who experienced stillbirths or miscarriages and their potentially complicated pregnancies. Another aspect of clinical data, not present in most survey or routine data sources, is the longitudinal perspective within a pregnancy. Longitudinal analyses across periods of time and healthcare contacts allow the ability to include only managements based upon true conditions, ensuring that only appropriate and correct referrals are included in the calculation rather than all referrals. An ideal data source for complex indicators would be longitudinally collected at the point-of-care to minimize the need for post data-collection processing. This would ensure that both numerators and denominators were collected simultaneously, and mitigate issues from recall bias of either care providers or mothers. In addition, one of the largest criticisms of the Lives Saved Tool is the quality of estimates around maternal mortality. The current use of indirect estimates greatly increases the likely uncertainty around the LiST estimates of maternal mortality. These results should increase the validity and reliability of future analysis with such data, simply because fewer assumptions will be needed.
The paper-based routine health information systems in Palestine, as in many other places, rely on care providers identifying key characteristics about patients and reporting to district and national health authorities, who aggregate and process the data to generate national indicators. The validity of any individual diagnosis is unknown. This additional reporting burden on care providers limits the ability to demand reporting of a comprehensive set of clinical data, and thus results in a reporting system focused on only the highest priority indicators. Complex health conditions and reporting chains can lead to either over- or under-reporting. For example, knowing the number of women referred for diabetes is useful, but does not indicate the proportion of women correctly diagnosed with diabetes or appropriately referred, leaving the system unable to rectify underlying problems. To create more actionable indicators, providers would need to document every diabetes test, the number of women positive and the number referred according to recommended guidelines. This extensive task is not likely to be a valuable use of time in a paper-based system. The routine system in Palestine also relies on reporting by two different levels of clinics (primary and referral), which makes it difficult to ensure that women are correctly included only once, in either the numerator or denominator, potentially leading to biases. Routine reporting data should be limited and focused on critical indicators that cannot be collected easily in clinical data sources or those needed to triangulate with other sources.
Typically, the primary source of coverage data used in LiST is household survey data, such as the MICS presented here. However, very little data are available from these surveys to directly populate antenatal care (or childbirth care) indicators. Information on antenatal services received or antenatal care attendance can be used to indirectly calculate several other indicators. However, these estimates are dependent on maternal recall, which may be biased towards experiences of women with difficult pregnancies who would tend to remember care more completely relative to uneventful pregnancies and deliveries, while excluding pregnancies ending in stillbirth or miscarriage. Although these indirect indicators (together with non-antenatal care indicators) can be useful for planning, these surveys are typically conducted only every five years making their input less timely for shorter-term planning or course-correction. Additional questions should be asked about the utility of these indirect estimates (which were formulated with sub-Saharan African data) when compared to actual values extracted from antenatal care records. If the Kanyangarara formula is applied to the paper-based antenatal records and the eRegistry, respectively, approximately 62 and 61% of women are estimated to be correctly managed for hypertension while the clinical data indicated that only 7 and 10% were correctly managed. The differences were much smaller for the diabetes management indicator which were predicted to be 29 and 31% respectively, while the actual clinical values were 13 and 35%, respectively.
Data extracted from paper-based antenatal care records and the eRegistry contained the greatest quantity of data for direct analysis. They also allowed for computing indicators that most closely matched the ideals of the Lives Saved Tool (Table 3). Although differences in documentation may account for the different values reported, it should also be noted that indicators from the eRegistry document more carefully the details around management, which are not typically recorded in the paper records, and thus should theoretically be a more precise indicator of correct management. The simplified single checkbox of any iron-folate supplementation in paper records may have over-estimated current performance as the LiST analysis estimated more than two-fold higher numbers of anemia cases being averted in the eRegistry-based analysis compared with paper records. Assuming that care providers are correctly completing their documentation, these results should be more valid and more reliable than survey based data or routine reporting with the multiple additional layers of data processing required. They are certainly more direct estimates that have the potential to be more representative of facility care since they also include all pregnancies, not just all live-births.
Extracting data from paper-based records on a regular basis is neither feasible nor sustainable for routine monitoring due to the expense and tardiness of such a system, and without the quality assurance routines used in this study, also by the likelihood of transcription errors. In addition, paper records can be incomplete and do not have built-in validations at data entry, as seen with the tetanus toxoid vaccination data.
Although the development of an eRegistry is time-consuming and resource-intensive, and up-front implementation costs are relatively high, the benefits can be wide-ranging by integrating multiple digital health interventions in a single system. In Palestine, the point-of-care data entry currently serves as an interactive checklist with clinical decision support, with integrated audit and feedback components and a reminder system for pregnant women. On the back-end, the system routinely generates key indicators at national, sub-national and clinic levels without requiring burdensome reporting.
A limitation of the eRegistry system in Palestine is that it is currently only available in public sector facilities and does not include private or non-governmental organization (NGO) facilities, nor public hospitals. Population coverage cannot be measured with the eRegistry data in this setting. Although the lack of data from private and NGO facilities does not affect the analysis of care delivered at public facilities, LiST analysis might predict larger health improvements than actually could occur, due to missing data on referred patients who seek care in external facilities. At the same time, population based surveys can provide the data needed to understand the flow of patients between public and private or NGO sectors and thus act as a calibration of the clinical data, in conjunction with routine reported data. The lack of data from any of the hospitals also limited the ability to define the interventions in terms of full quality of care at referral centers. However, it is likely that adding this information would only decrease the proportion of women correctly managed.