Main findings
The observed changes in the procedure rates over time were not consistent with the decline in rates that would be expected if English NHS trusts had responded to the NICE “do not do” guidance. Between 1998 and 2010, the proportion of deliveries performed by planned caesarean section increased in all women, with a steeper increase in women with hepatitis, against best-practice guidance.
Numbers of male varicocele procedures were low and changed little during the study. The low procedure rate may be a reflection of many clinicians being aware of, and largely complying with, best practice before NICE guidance was issued. However, as a ‘do not do’ guidance, a decline even in the already low rate would have been anticipated. Endometrial biopsy rates, on the other hand, have shown an unexpected general increase in women less than 40 years, with only a transient slowing between 2003 and 2007. This increase is present in both women above and below 40 years and when infertility was used in primary and secondary diagnosis codes. This increase is contrary to guidance and shows no association with the timing of the fertility guidance. Overall, there is no good evidence that the release of the reminders had any effect on the trends in clinical practice for any of the procedures studied.
Strengths and limitations
This work should be interpreted in light of some limitations, including difficulties extrapolating what trends would have occurred in procedure rates had guidance not been issued. We have partially overcome this for the caesarean section analysis by looking at women without hepatitis as an observational ‘control’ group. In addition, by assessing three very different procedures we hope to reduce the risk of attributing change to guidance that may be secondary to other secular trends. Any change in procedure trend, would not necessarily be caused by NICE guidance, only temporally associated with it. We are also unable to detect whether individual NHS trusts are enacting NICE guidance, relying instead on nationally collected evidence, which has shown that the guidance in these clinical areas has not appreciably influenced the trend in procedure performance. Any inference from this national data to the local trust level would be subject to ecological fallacy. Since the intention of the recommendation reminders and ‘do not do’ guidance is to effect a reduction in clinical procedure uptake, however, we can conclude, based on our study, that the guidelines appear not to have had an effect, since none of the procedures declined appreciably.
NICE does not record the date when recommendation reminders are published, which is an additional limitation, but since most observed changes in practice occurred before the NICE clinical guidance release (February and April 2004) and therefore, certainly before the recommendation reminder which follows the guidance, this would not alter our interpretation. There were two exceptions where there were changes in trend after guidance was issued: endometrial biopsies, which saw an increase in rate in 2007, the opposite of guidance and the fall in unplanned caesareans in women without hepatitis, which was not the focus of the caesarean guidelines or recommendation reminders. We are unable to account for any changes over time in the accuracy of diagnostic coding, but the consistency of our findings across three different procedures suggests that our results and interpretation are probably not due to coding anomalies changing over time.
More specific limitations, relevant to the clinical problems, include the fact that a caesarean section that occurs in a woman with hepatitis may not be solely because of the hepatitis B or C, but for a valid clinical reason, such as a breech presentation. We have therefore compared rates of planned and unplanned procedures in women with and without hepatitis to extrapolate, at an ecological level any trend to offset this concern. While the hepatitis C recommendation is a blanket ‘do not do’ guidance, the recommendation reminder for hepatitis B advises against planned caesarean section in women with hepatitis B on the presumption that the neonate will receive vaccination and immunoglobulin after birth. It is unlikely, therefore that a planned caesarean would be arranged to prevent mother-to-child transmission of hepatitis B in the UK, where the vaccine and immunoglobulin are freely available after birth.
To assess the quality of the HES data recording, we have compared the number of pregnant women with hepatitis B based on antenatal screening, with the number of hepatitis positive pregnant women in HES. The total number of deliveries in women with hepatitis coded in HES data was lower than the expected number of women with hepatitis B, based on antenatal screening (hepatitis B prevalence at antenatal clinic: 0.35% 2008; women delivering with hepatitis according to HES data: 0.11% in 2008) [14]. This raises questions as to the completeness of HES recording. This potential under-recording in HES could only affect our results if real reductions in caesarean section rates amongst women with hepatitis B were masked because recording of hepatitis B in women with caesarean section dramatically improved after 2004. The total numbers of women giving birth coded in HES is comparable with national epidemiological surveillance figures over the time period.
Our results show two procedures which have clear upward trends in procedure uptake, despite guidance to the contrary. The increasing trend in endometrial biopsies in women with infertility cannot be explained by competing evidence outside the published guidelines since there is good consensus in the peer-reviewed literature that endometrial biopsies are an ineffective means of predicting fertility and therefore, should not be conducted outside a research context [15]. We have attempted to control for artefactual explanations for the rise in endometrial biopsies by stratifying women into above and below 40 years and by limiting the analysis to only those women with infertility as their primary diagnosis, in case the biopsies are indicated for cancer investigation rather than infertility. Despite best evidence, the continued rise in planned caesarean section rates in women with hepatitis, compared to women without hepatitis, illustrates the potential difficulties in implementing disinvestment in this and other clinical guidelines where it is possible clinicians may dispense with NICE guidance, in favour of alternative evidence sources, such as published systematic reviews or local experience to dictate best practice. Unlike the consensus in endometrial biopsy indications amongst experts and the literature, there has been some debate in the literature for caesarean sections in women with hepatitis [16]. However, the updated NICE guidelines (2011) [17] uphold the original position that planned caesarean sections are not indicated for hepatitis status alone.
This work provides a clear evaluation of trends in three sub populations, in which procedures were identified as areas for potential disinvestment. Successful disinvestment from ineffective care is crucial if the NHS is to respond successfully to the current pressures on public funding. Our analysis uses twelve years of HES data [18] to describe changes in clinical practice that might be associated with NICE guidance for evidence based practice. Previous publications evaluating the implementation of NICE guidance, both those commissioned by NICE [8] and reported independently [10] were in 2005 and 2004 respectively and therefore, this is a timely update in light of the changing responsibilities of NICE [19]. The past literature focuses on NICE ‘technology appraisals’, which historically carried a mandatory funding requirement for commissioners, rather than the non-mandatory clinical guidelines [8, 10]. Numerous evaluations of NICE implementation have revealed ‘under implementation’ of guidance [4, 8]. Of those specifically assessing disinvestment decisions (two of the 45 implementation uptake reports) one has shown a decline in accordance with NICE guidance [20], the other a continued increase in drug prescribing, despite guidance [21]. Therefore, NICE may not only produce an insufficient number of disinvestment guidelines [22], but also have minimal evidence of their implementation.
