The impact of the introduction of liquid based cytology on the variation in the proportion of inadequate samples between GP practices
© Harrison et al; licensee BioMed Central Ltd. 2007
Received: 05 April 2007
Accepted: 01 August 2007
Published: 01 August 2007
Historically there has been a wide variation in the proportion of inadequate smears between general practices. Cervical screening in the UK is undergoing a fundamental change by moving from conventional to liquid based cytology (LBC). The main driver for this change has been a predicted reduction in the proportions of inadequate samples. This study investigates the effect of LBC on the variation in the proportion of inadequate samples between general practices using Shewhart's theory of variation and control charts.
Routinely collected cervical cytology data was obtained for all general practices in two localities in South Staffordshire for periods before and after the introduction of liquid based cytology. Control charts of the proportion of inadequate smears were plotted for the practices stratified by laboratory. A standardised measure of variation for all of the practices in each laboratory and each time period was also calculated.
Following the introduction of liquid based cytology the overall proportion of inadequate samples in the two localities fell from 11.8 to 1.3% (p < 0.05). This fall was associated with a reduction in the average variation between the GP practices in the two localities from 1.6 to 1.0 standard deviations. There has also been a reduction in the number of practices showing special cause variation from eight to one following the introduction of liquid based cytology.
A reduction in the proportion of inadequate samples has been realised in these localities. The reduction in the overall proportion of inadequate samples has also been accompanied by a reduction in variation between GP practices.
In 2005/06 3.6 million women were screened for cervical cancer in England, 3.36 million following a formal invitation from the screening programme, generating just under 4.0 million cervical smears . Women who have a smear which is identified as inadequate by the reporting cytology laboratory must be retested, in line with national guidelines, and women who have three successive inadequate smears are referred for colposcopy . Inadequate smears are a source of avoidable distress to women, and a potential waste of resources in general practices, clinics and cytology laboratories .
A study we undertook in South Staffordshire in 2000–01 identified two sources of variation in the proportion of inadequate smears; that associated with laboratories and that associated with GP practices . Our study showed that there was wide variation in the proportion of inadequate smears amongst 100 general practices, with 23% showing evidence of special cause variation which merited further investigation to identify possible causes. However, the vast majority of general practices (77%) were consistent with common cause variation, which, according to Shewhart's theory of variation, is best addressed by introducing fundamental changes to the underlying process.
Since then a fundamental change has been made in the process of collecting and analysing cervical cytology samples in two of the four localities in the original study, by introducing Liquid Based Cytology (LBC), a new method of cervical cell sample preparation. The main driver for this change was a reduction in the proportion of inadequate samples in pilot studies [4, 5].
The current study repeated the analysis carried out previously on this subset of data. The aim is to assess whether the move to LBC has affected the variation in the proportion of inadequate samples between practices as well as the overall proportion.
Cervical cytology data were obtained for all 45 general practices in two localities in South Staffordshire for the calendar years 2000 and 2001. The data included the number of samples taken and the number identified as inadequate. The corresponding data for the 45 practices in the two localities was obtained as six month aggregates for October 2005 – September 2006 following introduction of LBC into these areas. Both the laboratories serving the area used the SurePath® LBC system. A number of practice changes have occurred over this period resulting in slightly different numbers using each laboratory in the two study periods.
We undertook analyses of variation by general practice using P-charts. P-charts are one member of the family of control charts and are designed to be used with binomial data (inadequate smear – yes/no) which is expressed as a proportion of the sample size. We sorted our data by the total number of samples (ie sample size) and plotted our P-charts with respect to this order. This has two advantages. Firstly, the resulting P-charts show the impact of sample size on the control limits, and secondly, the resulting control limits are easy on the eye because they appear like a funnel. Such plots have been advocated in health care .
Where, for each laboratory in each time period, Pobs is the proportion of inadequate samples observed for each individual practice, Pmean the overall mean proportion of inadequate samples for all the practices, and Nobs the number of samples taken for each individual practice. The mean of the standardised variation for all of the practices in each laboratory and each time period was then calculated. Analyses were performed in Microsoft Excel and Stata.
The study was approved by the Pan South Staffordshire Research Management and Governance Office. The study protocol was also submitted to South Staffordshire Local Research Ethics Committee who ruled that ethics committee approval was not necessary.
Overall the proportion of inadequate samples
Variation between GP practices
There has been a reduction in the average variation between the GP practices in the two localities from 1.6 to 1.0 SD in the study period. This has reduced from 1.49 to 1.10 SD for the practices served by Laboratory 1 and from 2.70 to.0.45 SD for the practices served by Laboratory 2.
There has also been a reduction in the number of practices showing special cause variation from eight in the period when conventional cytology was employed to one following the introduction of LBC. None of the practices showing special cause variation in the original study show special cause variation in the current analysis.
The expected reduction in the proportion of inadequate samples following the introduction of LBC has initially been realised in these localities. A systematic review commissioned before the decision (by the National Screening Committee) to roll out LBC found an overall proportion of inadequate samples of 0.8% (95% CI 0.1% to 5.5%), which is comparable to the 1.3% in this study (95% CI 1.2% to 1.5%) .
Our findings are contrary to a recent systematic review of primary studies comparing conventional and LBC techniques that found a median difference in the percentage of unsatisfactory slides of only 0.17% (IQR 0.98% to -0.37%) . This difference seems mainly due to a higher proportion of inadequate samples for conventional cytology in practice compared to the primary studies ; it remains to be seen if the low proportion of inadequate samples for LBC can be maintained over the longer term.
The introduction of LBC in these two localities has also been associated with a reduction in the variation in the proportion of inadequate samples between GP practices. Because of the relatively low number of samples taken after the introduction of LBC, some practices with zero inadequate samples can still fall above the lower control limit. However, this affects only one practice served by each laboratory so any effect would be minimal. There is further evidence that the introduction of LBC has resulted in a reduction in variation with only one practice now displaying special cause variation compared to the eight practices previously.
It seems therefore that the practices in these two localities are now displaying a more consistent process. This is perhaps not surprising as the introduction of the new technique was accompanied by training for all primary care staff involved in the process as part of local clinical governance arrangements. Training was carried in a centralised fashion by the PCT Cervical Screening Co-ordinators using educational material provided by SurePath®. Attendance at a training session was mandatory prior to any primary care practitioner (GP or nurse) undertaking cervical sample-taking with LBC.
Our original study noted a wide variation in the proportion of inadequate samples between all 156 English laboratories . More recently a study has compared the variability of inadequacy rates in eleven Welsh laboratories following the introduction of LBC using longitudinal control charts . They reported a reduction in variability between the laboratories. The results that we present here indicate this reduction may be mirrored at a practice level, suggesting that an improvement has been made in the overall process.
There are a number of limitations in our study. As it is a before/after comparison changes in other factors, such as the demographics of the population screened may have contributed to the results observed. We have demonstrated a large absolute and relative drop in the proportion of inadequate samples, both overall and for each laboratory. However, due to limitations in the data we were unable to adjust this by practice or by age. Nevertheless, the size of the change suggests that confounding factors are unlikely to explain it entirely. We are also unable to estimate how much of the reduction in both the proportion of inadequate samples and the variation between practices is due to LBC as a technique or due to other factors such as mandatory training.
The proportion of inadequate samples are not the only factor to consider when measuring quality improvement in cervical screening. Diagnostic accuracy is also important. In our original study we found no relationship between the proportion of smears reported as inadequate and the positive predictive value of a smear in data from English laboratories . We have not had the opportunity to measure any change in the detection of abnormalities in the current data, though recent reports from two large medium quality studies (based on the criteria used by Davey et al ) have indicated that LBC has been associated with an increase in diagnostic accuracy [10, 11].
It is clear that the reduction in the proportion of cervical screening inadequate samples has been realised, at least initially, in these localities. The reduction in the overall proportion of inadequate samples has also been accompanied by a reduction in variation between GP practices.
The challenge now is to maintain the low proportion of inadequate samples in the longer term.
Thanks to the Cytology Departments at Derby Hospitals NHS Foundation Trust and Good Hope Hospital NHS Trust for supplying data, and to the Cervical Cytology Steering Group for the former Burntwood, Lichfield and Tamworth and East Staffordshire PCTs for their support.
- Cervical Screening Programme England 2005-06. 2006, Leeds, The Information Centre
- Johnson J, Patnick J: Achievable standards, benchmarks for reporting and criteria for evaluating cervical cytopathology. 2000, Sheffield, NHSCSP Publications, 2Google Scholar
- Harrison WN, Mohammed MA, Wall MK, Marshall TP: Analysis of inadequate cervical smears using Shewhart control charts. BMC Public Health. 2004, 4: 25-10.1186/1471-2458-4-25.View ArticlePubMedPubMed CentralGoogle Scholar
- National Institute for Clinical Excellence: Technology Appraisal 69. Guidance on the use of liquid-based cytology for cervical screening. 2003, London, National Institute for Clinical ExcellenceGoogle Scholar
- Moss SM, Gray A, Marteau T, Legood R, Henstock E, Maissi E: Evaluation of HPV/LBC Cervical Screening Pilot Studies. Report to the Department of Health. 2004, London, Department of HealthGoogle Scholar
- Spiegelhalter D: Funnel plots for institutional comparison. Qual Saf Health Care. 2002, 11: 390-39a. 10.1136/qhc.11.4.390-a.View ArticlePubMedPubMed CentralGoogle Scholar
- Karnon J, Peters J, Chilcot J, McGoogan E: Liquid-based cytology in cervical screening: an updated rapid and systematic review. 2003, Sheffield, The School of Health and Related ResearchGoogle Scholar
- Davey E, Barratt A, Irwig L, Chan SF, Macaskill P, Mannes P, Saville AM: Effect of study design and quality on unsatisfactory rates, cytology classifications, and accuracy in liquid-based versus conventional cervical cytology: a systematic review. Lancet. 2006, 367: 122-132. 10.1016/S0140-6736(06)67961-0.View ArticlePubMedGoogle Scholar
- Fox R, Nix AB, Fielder H: A comparison of variability in Papanicolaou and liquid-based cytology inadequacy rates using Shewhart control charts. Cytopathology. 2006, 17: 175-181. 10.1111/j.1365-2303.2006.00302.x.View ArticlePubMedGoogle Scholar
- Schledermann D, Ejersbo D, Hoelund B: Improvement of diagnostic accuracy and screening conditions with liquid-based cytology. Diagnostic Cytopathology. 2006, 34: 780-785. 10.1002/dc.20515.View ArticlePubMedGoogle Scholar
- Kirschner B, Simonsen K, Junge J: Comparison of conventional Papanicolaou smear and SurePath liquid-based cytology in the Copenhagen population screening programme for cervical cancer. Cytopathology. 2006, 17: 187-194. 10.1111/j.1365-2303.2006.00384.x.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/7/191/prepub