Surgical audit: statistical lessons from Nightingale and Codman
Abstract
There is a long history of interest in examining and comparing surgical outcomes. The ‘epidemiological’ approach was initiated by Florence Nightingale in her suggestion for uniform surgical statistics, and she clearly predicted the problems that are associated with collecting, analysing and interpreting such data. Unfortunately those responsible for implementing and reporting her scheme appeared not to have shared her insight. The contrasting ‘clinical’ approach was championed by Ernest Codman in his search for full and honest appraisals of surgical errors. Once again, despite initial enthusiasm, others had great difficulty in following his example, although we discuss a recent instance of a reflective analysis of an individual surgeon's performance. We conclude by suggesting that a synthesis between these approaches is appropriate, but we follow others in warning of the inevitable extra‐statistical difficulties that will arise.
Citing Literature
Number of times cited according to CrossRef: 43
- Haiying Sun, Ruoqi Ning, Dong Xu, Fangxu Tang, Reply from Sun et al, Journal of the American Geriatrics Society, 10.1111/jgs.16760, 0, 0, (2020).
- David M. Shahian, Jeffrey P. Jacobs, Health Services Information: Lessons Learned from the Society of Thoracic Surgeons National Database, Health Services Evaluation, 10.1007/978-1-4939-8715-3_11, (217-239), (2019).
- Elizabeth Cecil, Alex Bottle, Aneez Esmail, Charles Vincent, Paul Aylin, What is the relationship between mortality alerts and other indicators of quality of care? A national cross-sectional study, Journal of Health Services Research & Policy, 10.1177/1355819619847689, (135581961984768), (2019).
- Elizabeth Cecil, Samantha Wilkinson, Alex Bottle, Aneez Esmail, Charles Vincent, Paul P Aylin, National hospital mortality surveillance system: a descriptive analysis, BMJ Quality & Safety, 10.1136/bmjqs-2018-008364, 27, 12, (974-981), (2018).
- Paul Aylin, Alex Bottle, Susan Burnett, Elizabeth Cecil, Kathryn L Charles, Paul Dawson, Danielle D’Lima, Aneez Esmail, Charles Vincent, Samantha Wilkinson, Jonathan Benn, Evaluation of a national surveillance system for mortality alerts: a mixed-methods study, Health Services and Delivery Research, 10.3310/hsdr06070, 6, 7, (1-314), (2018).
- Noel-Ann Bradshaw, Florence Nightingale (1820–1910): A Pioneer of Data Visualisation, Women in Mathematics, 10.1007/978-3-319-66694-5_11, (197-217), (2017).
- John Maindonald, Alice M. Richardson, This Passionate Study: A Dialogue with Florence Nightingale, Journal of Statistics Education, 10.1080/10691898.2004.11910718, 12, 1, (2017).
- David J. Ballard, Barbara Spreadbury, Robert S. Hopkins, Health Care Quality Improvement Across the Baylor Health Care System: The First Century, Baylor University Medical Center Proceedings, 10.1080/08998280.2004.11927982, 17, 3, (277-288), (2017).
- Jan Beyersmann, Christine Schrade, Florence Nightingale, William Farr and competing risks, Journal of the Royal Statistical Society: Series A (Statistics in Society), 10.1111/rssa.12187, 180, 1, (285-293), (2016).
- Bradley N Manktelow, Sarah E Seaton, T Alun Evans, Funnel plot control limits to identify poorly performing healthcare providers when there is uncertainty in the value of the benchmark, Statistical Methods in Medical Research, 10.1177/0962280214530281, 25, 6, (2670-2684), (2016).
- David M. Shahian, Jeffrey P. Jacobs, Health Services Information: Lessons Learned from the Society of Thoracic Surgeons National Database, Data and Measures in Health Services Research, 10.1007/978-1-4899-7673-4, (1-24), (2016).
- Jessica Hullman, Paul Resnick, Eytan Adar, Hypothetical Outcome Plots Outperform Error Bars and Violin Plots for Inferences about Reliability of Variable Ordering, PLOS ONE, 10.1371/journal.pone.0142444, 10, 11, (e0142444), (2015).
- M. A. Mohammed, R. Lilford, G. Rudge, R. Holder, A. Stevens, The findings of the Mid-Staffordshire Inquiry do not uphold the use of hospital standardized mortality ratios as a screening test for 'bad' hospitals, QJM, 10.1093/qjmed/hct101, 106, 9, (849-854), (2013).
- Andrea Manca, Andrew R. Willan, “Lost in Translation”, PharmacoEconomics Italian Research Articles, 10.1007/BF03320676, 12, 3, (183-202), (2013).
- Mohammed A Mohammed, Bradley N Manktelow, Timothy P Hofer, Comparison of four methods for deriving hospital standardised mortality ratios from a single hierarchical logistic regression model, Statistical Methods in Medical Research, 10.1177/0962280212465165, 25, 2, (706-715), (2012).
- Sarah E Seaton, Bradley N Manktelow, The probability of being identified as an outlier with commonly used funnel plot control limits for the standardised mortality ratio, BMC Medical Research Methodology, 10.1186/1471-2288-12-98, 12, 1, (2012).
- David M. Shahian, Fred H. Edwards, Jeffrey P. Jacobs, Richard L. Prager, Sharon-Lise T. Normand, Cynthia M. Shewan, Sean M. O'Brien, Eric D. Peterson, Frederick L. Grover, Public Reporting of Cardiac Surgery Performance: Part 1—History, Rationale, Consequences, The Annals of Thoracic Surgery, 10.1016/j.athoracsur.2011.06.100, 92, 3, (S2-S11), (2011).
- I. Alan Fein, Quality, patient safety, and culture: ‘We have met the enemy and he is us’—Pogo (Walt Kelly, 1971)*, Critical Care Medicine, 10.1097/CCM.0b013e31820a5183, 39, 5, (1196-1197), (2011).
- Maya Aravind, Kevin C. Chung, Evidence-Based Medicine and Hospital Reform: Tracing Origins Back to Florence Nightingale, Plastic and Reconstructive Surgery, 10.1097/PRS.0b013e3181c2bb89, 125, 1, (403-409), (2010).
- John L Moran, Patricia J Solomon, Global quantitative indices reflecting provider process-of-care: data-base derivation, BMC Medical Research Methodology, 10.1186/1471-2288-10-32, 10, 1, (2010).
- Peter Congdon, References, Applied Bayesian Hierarchical Methods, 10.1201/9781584887218, (495-500), (2010).
- Gwyn Bevan, Performance Measurement of “Knights” and “Knaves”: Differences in Approaches and Impacts in British Countries after Devolution, Journal of Comparative Policy Analysis: Research and Practice, 10.1080/13876980903076187, 12, 1-2, (33-56), (2010).
- Aris Perperoglou, Paul H. C. Eilers, Penalized regression with individual deviance effects, Computational Statistics, 10.1007/s00180-009-0180-x, 25, 2, (341-361), (2009).
- Stephen Bolsin, Mark Colson, IT Benefits in Healthcare Performance and Safety, Handbook of Research on Information Technology Management and Clinical Data Administration in Healthcare, 10.4018/978-1-60566-356-2, (798-813), (2009).
- Giampaolo Ugolini, Giancarlo Rosati, Isacco Montroni, Simone Zanotti, Alessio Manaresi, Luca Giampaolo, Mario Taffurelli, Victor Pricolo, An easy-to-use solution for clinical audit in colorectal cancer surgery, Surgery, 10.1016/j.surg.2008.07.012, 145, 1, (86-92), (2009).
- Gwyn Bevan, Richard Hamblin, Hitting and missing targets by ambulance services for emergency calls: effects of different systems of performance measurement within the UK, Journal of the Royal Statistical Society: Series A (Statistics in Society), 10.1111/j.1467-985X.2008.00557.x, 172, 1, (161-190), (2008).
- Christopher H Jackson, Displaying Uncertainty With Shading, The American Statistician, 10.1198/000313008X370843, 62, 4, (340-347), (2008).
- A. Laurie W. Shroyer, Gerald O. McDonald, Brandie D. Wagner, Randy Johnson, Lisa M. Schade, Missy R. Bell, Frederick L. Grover, Improving Quality of Care in Cardiac Surgery: Evaluating Risk Factors, Processes of Care, Structures of Care, and Outcomes, Seminars in Cardiothoracic and Vascular Anesthesia, 10.1177/1089253208323060, 12, 3, (140-152), (2008).
- Hayley E. Jones, David I. Ohlssen, David J. Spiegelhalter, Use of the false discovery rate when comparing multiple health care providers, Journal of Clinical Epidemiology, 10.1016/j.jclinepi.2007.04.017, 61, 3, (232-240.e2), (2008).
- David I. Ohlssen, Linda D. Sharples, David J. Spiegelhalter, A hierarchical modelling framework for identifying unusual performance in health care providers, Journal of the Royal Statistical Society: Series A (Statistics in Society), 10.1111/j.1467-985X.2007.00487.x, 170, 4, (865-890), (2007).
- Nicholas T. Longford, Donald B. Rubin, Performance Assessment and League Tables: Comparing Like with Like, SSRN Electronic Journal, 10.2139/ssrn.1002570, (2007).
- David M. Shahian, Fred H. Edwards, Victor A. Ferraris, Constance K. Haan, Jeffrey B. Rich, Sharon-Lise T. Normand, Elizabeth R. DeLong, Sean M. O’Brien, Cynthia M. Shewan, Rachel S. Dokholyan, Eric D. Peterson, Quality Measurement in Adult Cardiac Surgery: Part 1—Conceptual Framework and Measure Selection, The Annals of Thoracic Surgery, 10.1016/j.athoracsur.2007.01.053, 83, 4, (S3-S12), (2007).
- Bruce E. Keogh, Benjamin Bridgewater, Toward Public Disclosure of Surgical Results: Experience of Cardiac Surgery in the United Kingdom, Thoracic Surgery Clinics, 10.1016/j.thorsurg.2007.08.001, 17, 3, (403-411), (2007).
- Brad C. Meyer, Debra S. Bishop, Florence Nightingale: nineteenth century apostle of quality, Journal of Management History, 10.1108/17511340710754699, 13, 3, (240-254), (2007).
- Andrea Manca, Andrew R Willan, ???Lost in Translation???, PharmacoEconomics, 10.2165/00019053-200624110-00007, 24, 11, (1101-1119), (2006).
- Paulo Sousa, Matthew Bazeley, Sören Johansson, Helle Wijk, The use of national registries data in three European countries in order to improve health care quality, International Journal of Health Care Quality Assurance, 10.1108/09526860610704178, 19, 7, (551-560), (2006).
- Peter C. Smith, Performance Measurement in Health Care: History, Challenges and Prospects, Public Money and Management, 10.1111/j.1467-9302.2005.00476.x, 25, 4, (213-220), (2005).
- Sheila M. Bird, Cox Sir David, Vern T. Farewell, Goldstein Harvey, Holt Tim, Smith Peter C., Performance indicators: good, bad, and ugly, Journal of the Royal Statistical Society: Series A (Statistics in Society), 10.1111/j.1467-985X.2004.00333.x, 168, 1, (1-27), (2004).
- Richard Lilford, Mohammed A Mohammed, David Spiegelhalter, Richard Thomson, Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma, The Lancet, 10.1016/S0140-6736(04)15901-1, 363, 9415, (1147-1154), (2004).
- David J. Spiegelhalter, Paul Aylin, Nicola G. Best, Stephen J. W. Evans, Gordon D. Murray, Commissioned analysis of surgical performance using routine data: lessons from the Bristol inquiry, Journal of the Royal Statistical Society: Series A (Statistics in Society), 10.1111/1467-985X.02021, 165, 2, (191-221), (2002).
- R. N. Curnow, Discussion on the paper by Spiegelhalter, Aylin, Best, Evans and Murray, Journal of the Royal Statistical Society: Series A (Statistics in Society), 10.1111/1467-985X.t01-1-02021, 165, 2, (221-231), (2002).
- M. Stone, Florence Nightingale, Statisticians of the Centuries, 10.1007/978-1-4613-0179-0, (171-175), (2001).
- Simon J. Mackenzie, Stephen W. Kendrick, J. Cameron Howie, From severity scores to health gain–a difficult road but one worth traveling, Current Opinion in Critical Care, 10.1097/00075198-200006000-00007, 6, 3, (181-186), (2000).




