Why choice of metric matters in public health analyses: a case study of the attribution of credit for the decline in coronary heart disease mortality in the US and other populations
© Gouda et al; BioMed Central Ltd. 2012
Received: 16 September 2011
Accepted: 28 January 2012
Published: 28 January 2012
Reasons for the widespread declines in coronary heart disease (CHD) mortality in high income countries are controversial. Here we explore how the type of metric chosen for the analyses of these declines affects the answer obtained.
The analyses we reviewed were performed using IMPACT, a large Excel based model of the determinants of temporal change in mortality from CHD. Assessments of the decline in CHD mortality in the USA between 1980 and 2000 served as the central case study.
Analyses based in the metric of number of deaths prevented attributed about half the decline to treatments (including preventive medications) and half to favourable shifts in risk factors. However, when mortality change was expressed in the metric of life-years-gained, the share attributed to risk factor change rose to 65%. This happened because risk factor changes were modelled as slowing disease progression, such that the hypothetical deaths averted resulted in longer average remaining lifetimes gained than the deaths averted by better treatments. This result was robust to a range of plausible assumptions on the relative effect sizes of changes in treatments and risk factors.
Time-based metrics (such as life years) are generally preferable because they direct attention to the changes in the natural history of disease that are produced by changes in key health determinants. The life-years attached to each death averted will also weight deaths in a way that better reflects social preferences.
KeywordsComparative Effectiveness Research Policy analysis Determinants of Mortality Epidemiologic Methods Coronary Heart Disease
Public health policy is powerfully influenced by assumptions about past successes. In the last decades of the 20th century, most high income countries experienced substantial declines in premature deaths from major vascular diseases. Interpretation of the reasons for these declines remains relevant to contemporary deliberations on public health policy.
The coronary heart disease model, IMPACT, which was first developed by Capewell and colleagues in the late 1990s has been used to estimate the relative contributions of risk factor changes and clinical treatments to observed declines in coronary heart disease (CHD) mortality. As initially developed, IMPACT expressed mortality declines in the metric of 'deaths prevented or postponed' (DPP). Assessments, using DPP, have been published of the relative contribution of risk factor changes and better treatment to CHD declines in Scotland , England and Wales , Ireland , the USA , Poland (unpublished), New Zealand , Finland  China (Beijing)  and elsewhere.
Pairs of studies: DPP versus LYG
IMPACT is a very large model implemented in Excel. It uses data on age and sex specific levels of up to 6 population-based risk factors as well as nine specific treatments for nine disease states, all stratified by age and sex.
Results of four pairs of studies using the IMPACT model are shown in Figure 1. The studies using DPP attributed between one half and two thirds of the gains from 1970 to 2000 to favourable shifts in risk factor levels. However, the corresponding analyses using LYG have attributed much more of the gains (up to 79%) to RF changes.
Despite the sensitivity of results to the choice of metric, there has been little published discussion of the respective merits of models based in these alternative metrics. Here we explore why the results obtained in these analyses turn out to be so sensitive to the choice of metric. We also assess the robustness of the associated assumptions. Our review concentrates mainly on the decline in the population risks of CHD death in the USA between 1980 and 2000 [4, 11] and addresses two questions: how do the differences in the attribution of credit come about? And do the extra assumptions required for the LYG analyses make them less robust? In the discussion we reflect on the wider implications of this case study.
Methods and Results
How does the difference in attributed credit come about?
Within IMPACT, reduced population risks of CHD death derive from two sources:
1 Better treatment: By definition this only applies to those patients who come to clinical attention, either because of the onset of clinical CHD or the detection of elevated risk factors. Treatment benefits derive either from reduced case fatality due to clinical management of an acute or chronic episode of CHD or from the therapeutic (pharmacological) reduction in elevated blood pressure or blood cholesterol levels (statin-induced LDL-C lowering for example). The gains from preventive treatments (such as statins) are shown distinct from risk factor improvement gains, (such as following reduction in population levels of cholesterol). To gain credit for a 'death prevented or postponed' an advance in treatment has to result in survival for at least 12 months longer than expected under baseline treatment. For the corresponding analyses using LYG, each death averted by advances in treatment is assigned an expected remaining life time (derivation discussed further below). The sum of these provides the LYG from treatment advances. When calculating LYG, the expected remaining lifetime is estimated using mortality rates observed in those surviving beyond the first thirty days after the onset of clinical CHD.
* This is a simplification of the actual model, in which transitions between a larger range of states are modelled
Key for Figure 2
1- General US life expectancy
2- Mid-way between general US life expectancy and survival rate for post-AMI patient group
3- Equivalent to the survival rate for the post-AMI patient group
4- Patient group specific survival rates
Results of pairs of studies using the IMPACT model: for each of 4 countries one study was performed using Deaths Prevented or Postponed (DPPs) and one using Life Years Gained (LYG).
DPPs (% share)
LYG (% share)
LYG per DPP
Favourable risk factor changes
Favourable risk factor changes
Favourable risk factor changes
Favourable risk factor changes
* All LYG are attributed to the age of averted death, not to the age at which the life years would have been lived
Where do the gained years come from?
The treatment arm of the US IMPACT model estimates patients' median survival using population-based data from unselected cohorts of MediCare patients following their hospital admission for acute myocardial infarction, heart failure, or revascularization. Additional age-specific median survival data for unstable angina patients were obtained from a large retrospective cohort study of unselected patients in the United Kingdom [16–18], since MediCare only included US patients aged over 65.
Life expectancies (ex) assigned to US men and women in 2000, comparing those who have not progressed to clinical CHD ('healthy') with those who have survived an acute myocardial infarction without the onset of heart failure and those who progress to heart failure
Life expectancy at exact age (ex)
Those who have survived an AMI by 30 days (excluding those with HF*)2
Those who have survived an AMI by 30 days but remain in HF* 3
How robust are the model extensions required for the LYG metric?
Survival functions assigned to specified health states: sources, limitations and strengths
Limitations of assumption
Strengths of assumption
CHD patients. Heart failure (HF) in the community is assumed to result in one third the fatality as HF in hospital and hypertension life expectancy is 80% that of US general population
May overestimate the life years lost due to deaths caused by HF in the community and hypertension.
May overestimate the impact of treatments preventing and postponing deaths due to HF in the community and hypertension therefore making the estimated relative contribution of risk factors conservative.
CHD patients. Median survival was assumed to be that of the post-AMI patients
May underestimate the benefit of reductions in risk factor prevalence
Provides a conservative estimate
Healthy population. survival was assumed to be that of general US population
US Bureau of the Census 
May both underestimate and overestimate the benefit or harms in terms of survival for each of the changes in risk factor prevalence
Avoids methodological issues of non-additivity and double-counting
Asymptomatic CHD. Survival was estimated as half-way between that of post-AMI patient and General US life expectancy
Arbitrary and may underestimate the benefit of reductions in risk factor prevalence
Provides a conservative estimate
For all healthy people who never experience a CHD event, the model uses a universal weight, the general US life expectancy (weight 1 in Figure 2). There are two potential problems with this. First, the general US population is made up of people who are both at risk of a CHD death and those that are not at risk. The impact of decreasing one's exposure to a risk factor or risk factors, however, might be expected to improve one's life expectancy above that of the general population. The assumption used here in the IMPACT model, therefore, is likely to lead to conservative estimates of the gains due to risk factor changes amongst the healthy population.
Second, a universal life expectancy applied to deaths averted amongst the 'healthy' populations due to risk factors changes means that the specific benefits gained from each risk factor change are lost. There are two reasons why this one weight is employed however. First the IMPACT model calculates the number of DPPs according to specific risk profiles and the prevalence of each risk factor. Any attempt to account for these differences again would potentially double count the impact.
A separate but related issue arises from risk factors being correlated  so the impact of changes in individual risk factors is not additive. Thus, employing a common life expectancy for all those who avert a CHD death due to risk factor changes avoids the methodological problems that arise from non-additivity.
For the group of people with asymptomatic and/or undiagnosed CHD, the model uses a weight that is mid-way between the general US life expectancy and the survival rates of post-AMI patients (weight 2 in Figure 2). Furthermore, people diagnosed with CHD may gain survival time either from better treatment or from favourable risk factor changes. This group are given a survival weight equivalent to that of the post-AMI patient group (weight 3 in Figure 2).
These assumptions suggest that these groups of people have a small advantage in terms of survival and, again, probably provide a conservative assessment of the impact of risk factors on these groups of people.
Lastly, patient group specific survival rates are used to weight the death prevented or postponed from a treatment (weight 4 in Figure 2). Weight 4 is based on case fatality data for most of the patient groups included in the model. Two main groups, however, community heart failure and hypertension, lack sufficient data to inform on survival. To estimate survival for these two groups the model uses the assumptions that those experiencing heart failure in the community have one third of the case fatality of those who suffer heart failure in the hospital and those with hypertension will have 20% less life expectancy than the average life expectancy. Little literature exists to inform these estimates and these assumptions may slightly overestimate the number of years lost due to these conditions .
To answer the question posed above, therefore, all groups of people that may have avoided or postponed a CHD death have been accounted for in the US IMPACT model and the assumptions used to extend this analysis from the event-based metric DPP to the time-based metric LYG may at times seem speculative and even arbitrary, but the estimates are typically reasonable and often conservative. In other words, the model is likely to underestimate gains in life years from favourable changes in risk factors.
Changes in disease determinants gain policy relevance from their effects on the population's burden of disease. This applies to both risk factor changes and to therapies. We have shown how these effects are better captured when expressed in a time-based metric such as life years gained (LYG).
Extending the IMPACT model to enable outputs in LYG required additional assumptions but these turned out to be relatively robust and unlikely to seriously offset the wider advantages of the LYG model.
Although results from analyses based in LYG differ importantly in their policy implications from analyses based in DPP, it is striking that there has been little discussion of the respective merits of public health models employing these two alternative metrics. Given that public health modelling influences public health policy, it is important to 'get it right'. Criteria are therefore needed for the critical appraisal of such models. From our demonstration that choice of metric can be highly consequential in such models it follows that reports of modelling exercises should explicitly justify their choice of metric.
Time-based metrics will usually be better because:
a They direct attention to changes in the natural history of the disease produced by the change in health determinants This aids understanding of 'what is going on' in the population's health experience;
b Life years gained relate more closely to the social value of the changes in population health. Deaths, considered as instantaneous events, are weightless. It is the life foregone that matters.
Some limitations of this investigation should be noted. Furthermore, the many assumptions and limitations of the US IMPACT model have previously been noted [4, 11]. In this review of the IMPACT model we have therefore focussed mainly on the assumptions involved in estimating LYG due to CHD treatments and risk factor changes, and not on those involved in estimating the underlying deaths postponed or prevented (DPPs). Clearly, as the life years gained are calculated as weights for each DPP, assumptions which weaken the DPP model will, therefore, impact upon the results of the LYG model as well. However, it was not our purpose to evaluate the specific results of either model but rather to assess the use of time-based outcome (LYG) versus an event-based outcome (DPP). Another limitation of our study is that we have not quantified the uncertainty around each assumption in this paper and this may be warranted in future efforts in order to identify where to focus further detailed work and clarifications.
Lastly, here we have considered just two metrics for capturing changes in disease occurrence. Our argument generalises to choice from among the full spectrum of available metrics - from the purely objective metrics of aetiologic epidemiology, to metrics based in objective phenomena but weighted to take account of social values (such as life years lost and gained), to hybrid measures such as the DALY which entail some formal valuing of health states but which maintain strong links to the 'objectivist' categories of medical science, to metrics located within the subjectivist theory of value of economics ('extra-welfarist' QALYs and 'welfarist' QALYs ). Methodological pluralism can enrich public health. No one metric meets all purposes, so choices from among them should be deliberate and justified.
In conclusion, time-based metrics (such as life years) are generally preferable to simply enumerating deaths prevented, because life years direct attention to the changes in the natural history of disease that are produced by changes in key health determinants. The life-years metric also better reflects social preferences.
This work was supported by a Medical Research Council PhD Studentship.
- Capewell S, Morrison CE, McMurray JJ: Contribution of modern cardiovascular treatment and risk factor changes to the decline in coronary heart disease mortality in Scotland between 1975 and 1994. Heart. 1999, 81: 380-386.View ArticlePubMedPubMed CentralGoogle Scholar
- Unal B, Critchley JA, Capewell S: Explaining the Decline in Coronary Heart Disease Mortality in England and wales between 1981 and 2000. Circulation. 2004, 109: 1101-1107. 10.1161/01.CIR.0000118498.35499.B2.View ArticlePubMedGoogle Scholar
- Bennett K, Kabir Z, Unal B, et al: Explaining the recent decrease in coronary heart disease mortality rates in Ireland, 1985-2000. J Epidemiol Commun H. 2006, 60: 322-327. 10.1136/jech.2005.038638.View ArticleGoogle Scholar
- Ford E, Ajani UAA, Croft JB, Critchley JA, Labarthe DR, Kottke TE, Giles WH, Capewell S: Explaining the Decrease in U.S. Deaths from Coronary Disease, 1980-2000. N Engl J Med. 2007, 356: 2388-2398. 10.1056/NEJMsa053935.View ArticlePubMedGoogle Scholar
- Capewell S, Beaglehole R, Seddon M, McMurray J: Explanation for the decline in coronary heart disease mortality rates in Auckland, New Zealand, between 1982 and 1993. Circulation. 2000, 102: 1511-1516.View ArticlePubMedGoogle Scholar
- Laatikainen T, Critchley J, Vartiainen E, Salomaa V, Ketonen M, Capewell S: Explaining the decline in coronary heart disease mortality in Finland between 1982 and 1997. Am J Epidemiol. 2005, 162: 764-773. 10.1093/aje/kwi274.View ArticlePubMedGoogle Scholar
- Critchley J, Liu J, Zhao D, Wei W, Capewell S: Explaining the increase in coronary heart disease mortality in Beijing between 1984 and 1999. Circulation. 2004, 110: 1236-1244. 10.1161/01.CIR.0000140668.91896.AE.View ArticlePubMedGoogle Scholar
- Unal B, Critchley JA, Fidan D, Capewell S: Life-years gained from modern cardiological treatments and population risk factor changes in England and Wales, 1981-2000. Am J Public Health. 2005, 95: 103-108. 10.2105/AJPH.2003.029579.View ArticlePubMedPubMed CentralGoogle Scholar
- Critchley JA, Capewell S, Unal B: Life-years gained from coronary heart disease mortality reduction in Scotland: prevention or treatment?. J Clin Epidemiol. 2003, 56: 583-590. 10.1016/S0895-4356(03)00059-3.View ArticlePubMedGoogle Scholar
- Kabir Z, Bennett K, Shelley E, et al: Life-years-gained from population risk factor changes and modern cardiology treatments in Ireland. Eur J Public Health. 2006, 17: 193-198.View ArticlePubMedGoogle Scholar
- Capewell S, Hayes DK, Ford ES, et al: Life-years gained among US adults from modern treatments and changes in the prevalence of 6 coronary heart disease risk factors between 1980 and 2000. Am J Epidemiol. 2009, 170: 229-236. 10.1093/aje/kwp150.View ArticlePubMedGoogle Scholar
- Wannamethee G, Whincup PH, Shaper AG, Walker M, MacFarlane PW: Factors determining case fatality in myocardial infarction "who dies in a heart attack"?. Br Heart J. 1995, 74: 324-331. 10.1136/hrt.74.3.324.View ArticlePubMedPubMed CentralGoogle Scholar
- Levy Y, Ellis TJ: A systems approach to conduct an effective literature review in support of information systems research. Informing Science Journal. 2006, 9: 181-211.Google Scholar
- Njolstad I, Arnesen E: Preinfarction blood pressure and smoking are determinants for a fatal outcome of myocardial infarction: a prospective analysis from the Finnmark Study. Arch Intern Med. 1998, 158: 1326-1332. 10.1001/archinte.158.12.1326.View ArticlePubMedGoogle Scholar
- Chun BY, Dobson AJ, Heller RF: The impact of diabetes on survival among patients with first myocardial infarction. Diabetes Care. 1997, 20: 704-708. 10.2337/diacare.20.5.704.View ArticlePubMedGoogle Scholar
- Gambhir M, Basanez MG, Turner F, Kumaresan J, Grassly NC: Trachoma: transmission, infection, and control. Lancet Infect Dis. 2007, 7: 420-427. 10.1016/S1473-3099(07)70137-8.View ArticlePubMedGoogle Scholar
- U.S. Census Bureau. 2000, [http://www.census.gov/]
- World Health Organisation: World Health Report 2004. Changing History. 2004Google Scholar
- Zhang Y: Cardiovascular diseases in American women. Nutr Metab Cardiovasc Dis. 2010, 20: 386-393. 10.1016/j.numecd.2010.02.001.View ArticlePubMedPubMed CentralGoogle Scholar
- Franco OH, Peeters A, Bonneux L, de Laet C: Blood pressure in adulthood and life expectancy with cardiovascular disease in men and women: life course analysis. Hypertension. 2005, 46: 280-286. 10.1161/01.HYP.0000173433.67426.9b.View ArticlePubMedGoogle Scholar
- Coast J, Smith RD, Lorgelly P: Welfarism, extra-welfarism and capability: the spread of ideas in health economics. Soc Sci Med. 2008, 67: 1190-1198. 10.1016/j.socscimed.2008.06.027.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/12/88/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.